GigaBASE is most efficient for applications fetching records using indices or direct object references. High speed of query execution is provided by elimination of data transfer overhead and very effective locking implementation. Synchronization of concurrent database access is implemented in GigaBASE by means of atomic instructions, adding almost no overhead to query processing. GigaBASE uses modified B-tree indices to provide fast access to disk resident data (with minimal disk read operations).
GigaBASE supports transactions, online backups and automatic recovery after system crash. Transaction commit protocol is based on shadow pages algorithm, performing atomic update of database. Recovery can be done very fast, providing high availability for critical applications. Moreover, elimination of transaction logs improves total system performance and leads to more effective usage of system resources.
GigaBASE is application-oriented database. Database tables are constructed using information about application classes. GigaBASE supports automatic scheme evaluation, allowing you to do changes only in one place - in your application classes. GigaBASE provides flexible and convenient interface for retrieving data from database. SQL-like query language is used to specify queries, and such post-relational capabilities as non-atomic fields, nested arrays, user-defined types and methods, direct interobject references simplifies design of database application and makes them more efficient.
GigaBASE is able to efficiently handle databases with several millions objects and up to terabyte size even at computers having not so much physical memory. Page pool using LRU strategy for page replacement and B-tree indices minimize number of disk operations and so provide high system performance.
start from follow by performs recursive records
traversal using references.
The following rules in BNF-like notation specifies grammar of GigaBASE query language search predicate:
| Example | Meaning |
|---|---|
| expression | non-terminals |
| not | terminals |
| | | disjoint alternatives |
| [not] | optional part |
| {1..9} | repeat zero or more times |
select-condition ::= [ expression ] [ traverse ] [ order ] [ limit )
expression ::= disjunction
disjunction ::= conjunction
| conjunction or disjunction
conjunction ::= comparison
| comparison and conjunction
comparison ::= operand = operand
| operand != operand
| operand <> operand
| operand < operand
| operand <= operand
| operand > operand
| operand >= operand
| operand (not) like operand
| operand (not) like operand escape string
| operand (not) match operand
| operand (not) in operand
| operand (not) in expressions-list
| operand (not) between operand and operand
| operand is (not) null
operand ::= addition
additions ::= multiplication
| addition + multiplication
| addition || multiplication
| addition - multiplication
multiplication ::= power
| multiplication * power
| multiplication / power
power ::= term
| term ^ power
term ::= identifier | number | string
| true | false | null
| current | first | last
| ( expression )
| not comparison
| - term
| term [ expression ]
| identifier . term
| function term
| exists identifier : term
| parameter
function ::= abs | length | lower | upper
| integer | real | string | user-function
string ::= ' { { any-character-except-quote } ('') } '
expressions-list ::= ( expression { , expression } )
order ::= order by sort-list
sort-list ::= field-order { , field-order }
field-order ::= [length] field (asc | desc)
field ::= identifier { . identifier }
traverse ::= start from field [ follow by ( next | previous | fields-list ) ]
limit::= limit [ start-position , ] max-selected
max-selected ::= integer | parameter
start-position ::= integer | parameter
fields-list ::= field { , field }
user-function ::= identifier
Identifiers are case sensitive, begin with a..z, A..Z, '_' or '$' character, contain only a-z, A..Z, 0..9 '_' or '$' characters, and do not duplicate a SQL reserved words.
| abs | and | area | asc | between |
| by | current | desc | escape | exists |
| false | first | follow | from | in |
| integer | is | length | like | last |
| limit | lower | match | not | null |
| or | overlaps | real | rectangle | start |
| string | true | upper |
ANSI-standard comments may also be used. All character from double-hyphen to the end of the line are ignored.
GigaBASE extends ANSI standard SQL operations by supporting bit manipulation
operations. Operators and/or can be applied not only
to boolean operands but also to operands of integer type. Result of applying
and/or operator to integer operands is integer
value with bits set by bit-AND/bit-OR operation. Bits operations can be used
for efficient implementation of small sets. Also rasing to a power
operation ^ is supported by GigaBASE for integer and floating point
types.
Starting from 2.51 version GigaBASE supports LIMIT [S,]N
construction in queries. It is more useful, flexible and efficient way to
restrict number of selected records than limit specified for the cursor.
First parameter S (if present)
specifies number of record to be skipped (so S records
which match search criteria will not be selected). Parameter N
specifies maximal number of records to be selected. So if you want to show
first ten results of the query at the screen, you should append
limit 0,10 to the query. If you want to show next 10 results,
then append limit 10,10...
It is possible to use parameters of int4 type instead of
constants for specifying S and N.
In this case, the same precompiled statement can be used
for fetching parts of the list.
Limit construction correctly works when order by clause is present.
If there is index for variable in order part, then records are inspected in requested order,
so sort is not needed and after fetching N records we can finish
query execution. Otherwise, all records will be selected, sorted and then
only records from [S,N] interval are left and other are removed from
the selection.
company.address.city
Structure fields can be indexed and used in order by
specification. Structures can contain other structures as their components
and there are no limitations on nesting level.
Programmer can define methods for structures, which can be used
in queries with the same syntax as normal structure components.
Such methods should have no arguments except pointer to the object to which
they belong (this pointer in C++), and should return
atomic value (of boolean, numeric, string or reference type).
Also method should not change object instance (immutable method).
If method returns string, then this string should be allocated using
new char operator, because it will be deleted after copying of
its value.
So user-defined methods can be used for creation virtual components -
components which are not stored in database, but instead if this are calculated
using values of other components. For example, GigaBASE dbDateTime
type contains only integer timestamp component and such methods
as dbDateTime::year(), dbDateTime::month()...
So it is possible to specify queries like: "delivery.year = 1999"
in application, where delivery record field has
dbDateTime type. Methods are executed in the context of
application, where they are defined, and are not available to other
applications and interactive SQL.
length() function.
[] operator.
If index expression is out of array range, then exception will be raised.
in can be used for checking if array contains
value specified by left operand. This operation can be used only for arrays of
atomic types: with boolean, numeric, reference or string components.
update method which creates copy of the array and returns
non-constant reference.
exists
operator. Variable specified after exists keyword can be used
as index in arrays in the expression preceded by exists
quantor. This index variable will iterate through all possible array
index values, until value of expression will become true or
index runs out of range. Condition
exists i: (contract[i].company.location = 'US')
will select all details which are shipped by companies
located in US, while query
not exists i: (contract[i].company.location = 'US')
will select all details which are shipped only from companies outside US.
Nested exists clauses are allowed. Using of nested
exists quantors is equivalent to nested loops using correspondent
index variables. For example query
exists colon: (exists row: (matrix[colon][row] = 0))
will select all records, containing 0 in elements of matrix
field, which has type array of array of integer.
This construction is equivalent to the following
two nested loops:
bool result = false;
for (int colon = 0; colon < matrix.length(); colon++) {
for (int row = 0; row < matrix[colon].length(); row++) {
if (matrix[colon][row] == 0) {
result = true;
break;
}
}
}
Order of using indices is significant! Result of the following query execution
exists row: (exists colon: (matrix[colon][row] = 0))
will be completely different with result of previous query. The program can
simply hang in last case due to infinite loop for empty matrices.
char in C) and
byte-by-byte comparison of strings ignoring locality settings.
Construction like can be used for
matching string with a pattern containing special wildcard characters
'%' and '_'. Character '_' matches any single character, while character
'%' matches any number of characters (including 0). Extended form of
like operator with escape part can be used
to handle characters '%' and '_' in the pattern as normal characters if
they are preceded by special escape character, specified after
escape keyword.
If you rebuild GigaBASE with USE_REGEX macro, then you can use
match operator implementing standard regular expressions
(based on GNU regex library). Second operand of this operator specified
regular expression to be matched and should be string literal.
It is possible to search substring within string by in
operator. Expression ('blue' in color) will be true
for all records which color fields contains 'blue' word.
If length of searched string is greater than some threshold value
(currently 512), then Boyer-Moore substring search algorithm is used instead
of straightforward search implementation.
Strings can be concatenated by + or || operators.
Last one was added only for compatibility with ANSI SQL standard.
As far as GigaBASE doesn't support implicit conversion to string type in
expressions, semantic of operator + can be redefined for
strings.
company.address.city = 'Chicago'
will access record referenced by company component of
Contract record and extract city component of
address field of referenced record from Supplier
table.
References can be checked for null by is null
or is not null predicates. Also references can be compared for
equality with each other as well as with special null
keyword. When null reference is dereferenced, exception is raised
by GigaBASE.
There is special keyword current, which can be used to get
reference to current record during table search. Usually current
keyword is used for comparison of current record identifier with
other references or locating it within array of references.
For example, the following query will search in Contract
table for all active contracts
(assuming that field canceledContracts has
dbArray< dbReference<Contract> > type):
current not in supplier.canceledContracts
GigaBASE provides special construction for recursive traverse of records by references:
start from root-references
[ follow by ( next | previous | list-of-reference-fields ) ]
First part of this construction is used to specify root objects.
Nonterminal root-references should be variable of reference or
array of reference type. Two special keyword first and
last can be used here, locating first/last record in the table
correspondingly.
If you want to check for some condition all records
referenced by array of references or single reference field, then this
construction can be used without follow by part.If you specify follow by part, then GigaBASE will recursively traverse table records starting from root references and using list of reference fields list-of-reference-fields for transition between records. list-of-reference-fields should consists of fields of reference or array of reference type. Alternatively you can sepcify next or previous preudofields, which refer to next or previous record in the table (all records in GigaBASE table are linked in L2 list, new records are appended at the end of the list).
Traverse is done in depth first top-left-right order (first we visit parent node and then siblings in left-to-right order). Recursion is terminated when null reference is accessed or already visited record is referenced. For example the following query will search tree records with weight larger than 1 in TLR order:
"weight > 1 start from first follow by left, right"
For the following tree:
A:1.1
B:2.0 C:1.5
D:1.3 E:1.8 F:1.2 G:0.8
result of query execution will be:
('A', 1.1), ('B', 2.0), ('D', 1.3), ('E', 1.8), ('C', 1.5), ('F', 1.2)
As was already mentioned GigaBASE always manipulates with objects and doesn't accept joins.
Joins can be implemented using references. Consider the classical
Supplier-Shipment-Detail examples:
struct Detail {
char const* name;
double weight;
TYPE_DESCRIPTOR((KEY(name, INDEXED), FIELD(weight)));
};
struct Supplier {
char const* company;
char const* address;
TYPE_DESCRIPTOR((KEY(company, INDEXED), FIELD(address)));
};
struct Shipment {
dbReference<Detail> detail;
dbReference<Supplier> supplier;
int4 price;
int4 quantity;
dbDateTime delivery;
TYPE_DESCRIPTOR((KEY(detail, HASHED), KEY(supplier, HASHED),
FIELD(price), FIELD(quantity), FIELD(delivery)));
};
We want to get information about delivery of some concrete details from some concrete
suppliers. In relational database this query will be written something like this:
select from Supplier,Shipment,Detail where
Supplier.SID = Shipment.SID and Shipment.DID = Detail.DID
and Supplier.company like ? and Supplier.address like ?
and Detail.name like ?
In GigaBASE this request should be written as:
dbQuery q = "detail.name like",name,"and supplier.company like",company,
"and supplier.address like",address,"order by price";
GigaBASE will first perform index search in the table Detail for details
matching the search condition. Then it performs another index search to locate shipment
records referencing selected details. Then sequential search is used to check the rest of
select predicate.
rectangle type.
By default it has dimension 2 and integer coordinate types. It is possible to easily change
dimension or use floating point coordinates, but recompilation of GigaBASE is needed in
this case.It is possible to use this type not only in geographical system for representing spatial objects but also in many other cases when date is organized as hyper-cube and queries specify range of values for each dimension, for example:
select * from CAR where price between 20000 and 30000
and producedYear between 1997 and 1999
and mileage between 50000 and 100000;
If <price, producedYear, milage> are dimensions of the rectangle,
then this query can be executed using only one indexed search in R-Tree.Rectangle fields can be indexed - R-Tree index is used in this case The R-tree provides fast access to spatial data. Basically the idea behind the R-Tree and the B-Tree are the same: use a hierarchical structure with a high branching factor to reduce the number of disk accesses. The R-tree is the extension of the B_tree for a multidimensional object. A geometric object is represented by its minimum bounding rectangle (MBR). Non-leaf nodes contain entries of the form (R,ptr) where ptr is a pointer to a child node in the R-tree; R is the MBR that covers all rectangles in the child node. Leaf nodes contain entries of the form (obj-id, R) where obj-id is a pointer to the object, and R is the MBR of the object. The main innovation in the R-tree is that the father nodes are allowed to overlap. By this means the R-tree guarantees at least 50% space utilization and remains balanced. The first R-tree implementation was proposed by Guttman. The Gigabase R-Tree class is based on Guttman's implementation with a quadratic split algorithm. The quadratic split algorithm is the one that achieves the best trade-off between splitting time and search performance.
Rectangle class provides method for calculating distance between two rectangle, rectangle area, checking whether two rectangle overlap or one contains another. Rectangle is specified by coordinates of two it's vertices. Rectangle class contains array of coordinates. Coordinates of first vertex are placed at the beginning of array, and then - coordinates of another vertex. Each coordinate of first vertex should not be large than correspondent coordinate of the second vertex. Singular rectangle with one or more coordinates of first vertex equals to the correspondent coordinate of the second vertex are allowed - this is a way of storing points in the database.
SubSQL provides some special operators for dealing with rectangle. First of all - all comparison operators can be used with the following semantic:
a == b | Rectangle a is the same as rectangle b |
a != b | Rectangle a is not the same as rectangle b |
a <= b | Rectangle b contains rectangle a |
a < b | Rectangle b contains rectangle a and them are not the same |
a >= b | Rectangle a contains rectangle b and are not the same |
a > b | Rectangle b contains rectangle a and them are not the same |
Also SubSQL provides overlaps and in operators.
First checks if two rectangles overlap, the second is equivalent to <= operator.
Rectangles can be added - result is minimal rectangle containing both rectangles-operands.
It is possible to access rectangle as array of coordinates - using [index]
notation. Coordinates in query are always returned as real numbers.
Optimizer is able to use spatial index for all comparison operators (except !=)
and for overlaps operator.
| Name | Argument type | Return type | Description |
|---|---|---|---|
| abs | integer | integer | absolute value of the argument |
| abs | real | real | absolute value of the argument |
| area | rectangle | real | area of the rectangle |
| integer | real | integer | conversion of real to integer |
| length | array | integer | number of elements in array |
| lower | string | string | lowercase string |
| real | integer | real | conversion of integer to real |
| string | integer | string | conversion of integer to string |
| string | real | string | conversion of real to string |
| upper | string | string | uppercase string |
GigaBASE allows user to define its own functions and operators. Function should have at least one but no more than 3 parameters of string, integer, boolean, reference or user defined (raw binary) type. It should return value of integer, real, string or boolean type.
User functions should be registered by the USER_FUNC(f) macro,
which creates a static object of the dbUserFunction class, binding
the function pointer and the function name.
There are two ways of implementing these functions in application.
First can be used only for functions with one argument. This argument should be of int8, real8,
char_t* types. And the function return type should be int8, real8, char_t* or bool.
If function has more than one parameters or it can accept parameters of different types (polymorphism)
then parameters should be passed as reference to dbUserFunctionArgument structure.
This structure contains type field, which value can be used in function implementation to
detect type of passed argument and union with argument value.
The following table contains mapping between argument types and where the value should be taken from:
| Argument type | Argument value | Argument value type |
|---|---|---|
dbUserFunctionArgument::atInteger | u.intValue | int8 |
dbUserFunctionArgument::atBoolean | u.boolValue | bool |
dbUserFunctionArgument::atString | u.strValue | char const* |
dbUserFunctionArgument::atReal | u.realValue | real8 |
dbUserFunctionArgument::atReference | u.oidValue | oid_t |
dbUserFunctionArgument::atRawBinary | u.rawValue | void* |
For example the following
statements make it possible to use the sin function in SQL
statements:
#include <math.h>
...
USER_FUNC(sin);
Functions can be used only
within the application, where they are defined. Functions are not accessible
from other applications and interactive SQL. If a function returns a string
type , the returned string should be copied by means of the operator
new, because
GigaBASE will call the destructor after copying the returned value.In GigaBASE, the function argument can (but not necessarily must) be enclosed in parentheses. So both of the following expressions are valid:
'$' + string(abs(x))
length string y
Functions with two argument can be also used as operators. Consider the following example,
in which function contains which performs case insensitive search for substring is defined:
bool contains(dbUserFunctionArgument& arg1, dbUserFunctionArgument& arg2) {
assert(arg1.type == dbUserFunctionArgument::atString
&& arg2.type == dbUserFunctionArgument::atString);
return stristr(arg1.u.strValue, arg2.u.strValue) != NULL;
}
USER_FUNC(contains);
dbQuery q1, q2;
q1 = "select * from TestTable where name contains 'xyz'";
q2 = "select * from TestTable where contains(name, 'xyz')";
In this example, queries q1 and q2 are equivalent.
dbQuery q;
dbCursor<Contract> contracts;
dbCursor<Supplier> suppliers;
int price, quantity;
q = "(price >=",price,"or quantity >=",quantity,
") and delivery.year=1999";
// input price and quantity values
if (contracts.select(q) != 0) {
do {
printf("%s\n", suppliers.at(contracts->supplier)->company);
} while (contracts.next());
}
| Type | Description |
|---|---|
| bool | boolean type (true,false) |
| int1 | one byte signed integer (-128..127) |
| int2 | two bytes signed integer (-32768..32767) |
| int4 | four bytes signed integer (-2147483648..2147483647) |
| int8 | eight bytes signed integer (-2**63..2**63-1) |
| real4 | four bytes ANSI floating point type |
| real8 | eight bytes ANSI double precision floating point type |
| char const* | zero terminated string |
| dbReference<T> | reference to class T |
| dbArray<T> | dynamic array of elements of type T |
In addition to types specified in the table above, GigaBASE records can also contain nested structures of these components. GigaBASE doesn't support unsigned types to simplify query language, eliminate bugs caused by sign/unsigned comparison and reduce size of database engine.
Unfortunately C++ provides no way to get metainformation about a class at runtime (RTTI is not supported by all compilers and also doesn't provide enough information). That is why programmer has to explicitly enumerate class fields to be included in database table (it also makes mapping between classes and tables more flexible). GigaBASE provides a set of macros and classes to make such mapping as simple as possible.
Each C++ class or structure, which will be used in database, should
contain special method describing its fields. Macro
TYPE_DESCRIPTOR(field_list) will construct
this method. The single argument of this macro is enclosed in parentheses list
of class fields descriptors. If you want to define some methods for the class
and make them available for database, then macro
CLASS_DESCRIPTOR(name, field_list)
should be used instead of TYPE_DESCRIPTOR. Class name is needed
to get references to member functions.
The following macros can be used for construction field descriptors:
INDEXED
HASHED
HASHED is equivalent to using
INDEXED index type.
CASE_INSENSITIVE
UNIQUE
OPTIMIZE_DUPLICATES
AUTOINCREMENT
int4 type and
make GigaBASE to assign unique value to this field when record is inserted in the database.
order by clause. Comparison is performed by means of
comparator function provided by programmer. Comparator functions receives three
arguments: two pointers to the compared raw binary objects and size of binary object.
The semantic of index_type is the same as of KEY macro.
UDT macro with memcmp
used as comparator.
UDT macro
for raw binary fields with predefined comparator memcmp and without indices.
inverse_reference is field of referenced table
containing inverse reference(s) to the current table. Inverse references
are automatically updated by GigaBASE and also are used for query optimization
(see Inverse references).
Although only atomic fields can be indexed, index type can be also specified for structures. Index will be created for component of the structure only if such type of index is specified in the index type mask of the structure. It makes possible to programmers to enable or disable indices for structure fields depending on the role of the structure in the record.
The following example illustrates creation of type descriptor:
class dbDateTime {
int4 stamp;
public:
int year() {
return localtime((time_t*)&stamp)->tm_year + 1900;
}
...
CLASS_DESCRIPTOR(dbDateTime,
(KEY(stamp,INDEXED),
METHOD(year), METHOD(month), METHOD(day),
METHOD(dayOfYear), METHOD(dayOfWeek),
METHOD(hour), METHOD(minute), METHOD(second)));
};
class Detail {
public:
char const* name;
char const* material;
char const* color;
real4 weight;
dbArray< dbReference<Contract> > contracts;
TYPE_DESCRIPTOR((KEY(name, INDEXED),
KEY(material, INDEXED),
KEY(color, INDEXED),
KEY(weight, INDEXED),
RELATION(contracts, detail)));
};
class Contract {
public:
dbDateTime delivery;
int4 quantity;
int8 price;
dbReference<Detail> detail;
dbReference<Supplier> supplier;
TYPE_DESCRIPTOR((KEY(delivery, INDEXED),
KEY(quantity, INDEXED),
KEY(price, INDEXED),
RELATION(detail, contracts),
RELATION(supplier, contracts)));
};
Type descriptors should be defined for all classes used in database.
In addition to defining type descriptors, it is necessary to establish
mapping between C++ classes and database tables. Macro
REGISTER(name) will do it. Unlike
TYPE_DESCRIPTOR, REGISTER macro should
be used in implementation file and not in header file. It constructs
descriptor of the table associated with the class. If you are going to work
with multiple databases from one application, it is possible to register
table in concrete database by means of
REGISTER_IN(name,database) macro.
Parameter database of this macro should be pointer to
dbDatabase object. Below is example of registration tables
in database:
REGISTER(Detail); REGISTER(Supplier); REGISTER(Contract);Table (and correspondent class) can be used only with one database at each moment of time. When you open database, GigaBASE imports all classes defined in application in database. If class with the same name already exists in database, its descriptor stored in the database is compared with descriptor of this class in application. If there are differences in class definitions, GigaBASE tries to convert records from the table to new format. Any kind of conversions between numeric types (integer to real, real to integer, with extension or truncation, are allowed). Also addition of new fields can be easily handled. But removing of the fields is only possible for empty tables (to avoid accidental data destruction).
After loading all class descriptors, GigaBASE checks if all indices specified in the application class descriptor are already present in database, constructing new indices and removing indices, which are no more used. Reformatting of table and adding/removing indices is only possible when there is no more than one application accessing database. So when first application is attached to database, it can perform table conversion. All other application can only add new classes to database, but not change existed ones.
There is one special preexisted table in database - Metatable,
which contains information about other tables in database. C++ programmer
need not to access this table, because format of database tables is specified
by C++ classes. But in interactive SQL program it is possible to examine
this table to get information about record fields.
Starting from version 2.45 GigaBASE supports autoincrement fields (fields unique value to which are assigned automaticaly by database). To be able to use them you should:
-DAUTOINCREMENT_SUPPROT flags
(add this flag to DEFS variables in GigaBASE makefile).DAUTOINCREMENT_SUPPORT.
dbTableDescriptor::initialAutoincrementCount.
It will be shared between all tables, so all table will have the same initial value of
autoincrement counter.
AUTOINCREMENT flag:
class Record {
int4 rid;
char const* name;
...
TYPE_DESCRIPTOR((KEY(rid, AUTOINCREMENT|INDEXED), FIELD(name), ...));
}
Record rec;
// no rec.rid should be specified
rec.name = "John Smith";
insert(rec);
// rec.rid now assigned unique value
int newRecordId = rec.rid; // and can be used to reference this record
= and , C++ operators
to construct query statement with parameters. Parameters can be specified
directly in places where they are used, eliminating any mapping between
parameters placeholders and C variables. In the following example of query
pointers to the parameters price and quantity
are stored in the query, so that query can be executed several times
with different values of parameters. C++ overloaded functions make it possible
to automatically determine type of parameter, requiring no extra information
to be supplied by programmer (so programmer has no possibility to make a bug).
dbQuery q;
int price, quantity;
q = "price >=",price,"or quantity >=",quantity;
As far as char* type can be used either for specifying
part of query (such as "price >=") either for parameter of string type,
GigaBASE uses special rule to resolve this ambiguity. This rule is based on the
assumption that there is no reason for splitting query text in two strings
like ("price ",">=") or specifying more than one parameter sequentially
("color=",color,color). So GigaBASE assumes first string to be
part of the query text and switches to operand mode
after it. In operand mode GigaBASE treats char* argument
as query parameter and switches back to query text mode, and so on...
It is also possible not to use this "syntax sugar" and construct
query elements explicitly by dbQuery::append(dbQueryElement::ElementType
type, void const* ptr) method. Before appending elements to the query,
it is necessary to reset query by dbQuery::reset() method
(operator = do it automatically).It is not to possible use C++ numeric constants as query parameters, because parameters are accessed by reference. But it is possible to use string constants, because strings are passed by value. There two possible ways of specifying string parameters in query: using string buffer or pointer to pointer to string:
dbQuery q;
char* type;
char name[256];
q = "name=",name,"and type=",&type;
scanf("%s", name);
type = "A";
cursor.select(q);
...
scanf("%s", name);
type = "B";
cursor.select(q);
...
Query variable can not be passed to a function as parameter or be assigned to other variable. When GigaBASE compiles the query, it saves compiled tree in this object. Next time the query will be used, no compilation is need and ready compiled tree can be used. It saves some time needed for query compilation.
GigaBASE provides two approaches of integration user-defined types in database.
First - definition of class methods - was already mentioned.
Another approach deals only with query construction. Programmer should
define methods, which will not do actual calculations, but instead
of this returns expression in terms of predefine database types, which
performs necessary calculation. It is better to describe it by example.
GigaBASE has no builtin datetime type. Instead of this normal C++
class dbDateTime can be used by programmer. This class defines
methods allowing to compare two dates using normal relational operators and
specify datetime field in order list:
class dbDateTime {
int4 stamp;
public:
...
dbQueryExpression operator == (char const* field) {
dbQueryExpression expr;
expr = dbComponent(field,"stamp"),"=",stamp;
return expr;
}
dbQueryExpression operator != (char const* field) {
dbQueryExpression expr;
expr = dbComponent(field,"stamp"),"<>",stamp;
return expr;
}
dbQueryExpression operator < (char const* field) {
dbQueryExpression expr;
expr = dbComponent(field,"stamp"),">",stamp;
return expr;
}
dbQueryExpression operator <= (char const* field) {
dbQueryExpression expr;
expr = dbComponent(field,"stamp"),">=",stamp;
return expr;
}
dbQueryExpression operator > (char const* field) {
dbQueryExpression expr;
expr = dbComponent(field,"stamp"),"<",stamp;
return expr;
}
dbQueryExpression operator >= (char const* field) {
dbQueryExpression expr;
expr = dbComponent(field,"stamp"),"<=",stamp;
return expr;
}
friend dbQueryExpression between(char const* field, dbDateTime& from,
dbDateTime& till)
{
dbQueryExpression expr;
expr=dbComponent(field,"stamp"),"between",from.stamp,"and",till.stamp;
return expr;
}
static dbQueryExpression ascent(char const* field) {
dbQueryExpression expr;
expr=dbComponent(field,"stamp");
return expr;
}
static dbQueryExpression descent(char const* field) {
dbQueryExpression expr;
expr=dbComponent(field,"stamp"),"desc";
return expr;
}
};
All these method receives as their parameter name of the field in the record.
This name is used to contract full name of the records components.
It can be done by class dbComponent, which constructor
takes name the the structure field and name of the component of the structure
and returns compound name separated by '.' symbol.
Class dbQueryExpression is used to collect expression items.
Expression is automatically enclosed in parentheses, eliminating conflicts
with operators precedence.
So, assuming record contains field delivery
of dbDateTime type it is possible
to construct queries like this:
dbDateTime from, till;
q1 = between("delivery", from, till),
"order by",dbDateTime::ascent("delivery");
q2 = till >= "delivery";
Except these methods, some class specific method can be also defined
in such way, for example method overlaps for region type.
The benefit of this approach is that database engine will work
with predefined types and is able to apply indices and other optimizations
to proceed such query. And from the other side, encapsulation of class
implementation is preserved, so programmer should not rewrite all queries
when class representation is changed.Variables of following C++ types can be used as query parameters:
| int1 | bool |
| int2 | char const* |
| int4 | char ** |
| int8 | char const** |
| real4 | dbReference<T> |
| real8 | dbArray< dbReference<T> > |
dbCursor<T>, where T is name of C++ classes
associated with database table. Cursor type should be specified in constructor
of the cursor. By default read-only cursor is created.
To create cursor for update, you should pass parameter
dbCursorForUpdate to the constructor.
Query is executed by cursor select(dbQuery& q)
or select() methods. Last method can be used to iterate through
all records in the table. It is also possible to specify cursor type in select statement:
dbCursorForUpdate or dbCursorViewOnly.
Select methods return number of selected records
and set current position to the first record (if available).
Cursors can be scrolled in forward or backward directions.
Methods next(), prev(), first(), last() can be used to
change current position of the cursor. If operation can not be performed
(no more records available), these methods return NULL
and cursor position is not changed. Method skip(int n) moves cursor
n positions forward if n is greater than zero,
or -n positions backward if n is less than zero.
Cursor for class T contains instance of class T, used for fetching current record. That is why table classes should have default constructor (constructor without parameters), which has no side effects. GigaBASE optimizes fetching records from database, copying only data from fixed part of the object. String bodies are not copied, instead of this correspondent field points directly in database. The same is true for arrays, which components has the same representation in database as in application (arrays of scalar types or arrays of nested structures of scalar components).
Application should not change
elements of strings and arrays in database directly. When array method need
to update array body, it create in-memory copy of the array and updates this
copy. If programmer wants to update string field, it should assign
to the pointer new value, but don't change string directly in database.
It is recommended to use char const* type instead of
char* for string components, to make it possible to compiler to
detect illegal usage of strings.
Cursor class provides get() method for obtaining pointer to
the current record (stored inside cursor). Also overloaded
operator->
can be used to access components of current record. If cursor is opened
for update, current record can be changed and stored in database
by update() method or can be removed. If current record is
removed, next record becomes current. If there is no next record, then previous
record becomes current (if exists). Method removeAll()
removes all records in the table and method removeAllSelected -
all records selected by the cursor.
When records are updated, database
size can be increased and extension of database section in virtual memory
is needed. As a result of such remapping, base address of the section can be
changed and all pointers to database fields kept by application will become
invalid. GigaBASE automatically updates current records in all opened
cursors when database section is remapped. So, when database is updated,
programmer should access record fields only through the cursors
-> method and do not use pointer variables.
Memory used for the current selection can be released by reset()
method. This method is automatically called by select(),
dbDatabase::commit(), dbDatabase::rollback() methods
and cursor destructor, so in most cases there is no need to
call reset() method explicitly.
Cursors can be also used to access records by reference. Method
at(dbReference<T> const& ref) set cursor to the record
pointed by the reference. In this case selection consists exactly of
one record and next(), prev() methods will always return
NULL. As far as cursors and references in GigaBASE are strictly
typed, all necessary checking can be done statically by compiler and
no dynamic type checking is needed. The only kind of checking,
which is done at runtime, is checking for null reference.
Object identifier of current record in the cursor can be obtained by
currentId() method.
It is possible to restrict number of records returned by select statement.
Cursor has two methods setSelectionLimit(size_t lim) and
unsetSelectionLimit(), which can be used to set/unset limitation
on number of records returned by query. In some situations programmer
wants to receive only one record or only few first records, so query execution
time and size of consumed memory can be reduced by limiting size of
selection. But if you specify order for selected records, query with
restriction for k records will no return first k records
with the smallest value of the key. Instead of this arbitrary k
records will be taken and then sorted.
So all operations with database data are performed by means of cursors. The only exception is insert operation. GigaBASE provides overloaded insert function:
template<class T>
dbReference<T> insert(T const& record);
This function will insert record at the end of the table and return reference
of the created object. Order of insertion is strictly specified in
GigaBASE and applications can use this assumption about records order in the
table. For applications widely using references for navigation between
objects it is necessary to have some root object, from which
traversal by references can be made. Good candidate for such root object
is first record in the table (it is also the oldest record in the
table). This record can be accessed by execution select()
method without parameter. The current record in the cursor will
be the first record in the table.
Starting from version 2.72 GigaBASE provides patch inserts.
Batch insert dramatically increase speed of database initialization,
for example without batch inserts, inserting first million of records with 2 keys
takes about 460 seconds, inserting second million of records - 18000 seconds (5 hours) !!!
With batch inserts, inserting first million of records takes 95 seconds, and second million - 97 seconds!
Such effect is achieved by delaying reconstruction of indices for inserted records until the transaction
commit or explicit call of dbDatabase::executeBatch() method. When batch is executed,
all previously inserted records are sorted by index field and than in this order added to the index.
Why it allows to significantly increase performance? When database is large (doesn't fit
in page pool) and inserted data contains pseudo-random values, then any insert of record
with indexed fields requires at least one read of B-Tree page from the disk. As
far as average disk access time is about 10ms, we could not insert more than 100 records per second.
But if we first sort inserted data, then most likely the same pages of B-Tree will
be used for several subsequent records and number of disk reads is significantly reduced.
Batch inserts are performed by dbDatabase::batchInsert template method
which has the same parameter is insert method. Inserted records will be indexed either when
transaction is committed, either by explicit execution of dbDatabase::executeBatch() method.
GigaBASE C++ API defines special null variable of reference type.
It is possible to compare null variable with references
or assign it to the reference:
void update(dbReference<Contract> c) {
if (c != null) {
dbCursor<Contract> contract(dbCursorForUpdate);
contract.at(c);
contract->supplier = null;
}
}
dbDatabase controls interaction of application
with database. It performs synchronization of concurrent accesses to the
database, transaction management, memory allocation, error handling,...
Constructor of dbDatabase objects allows programmer to specify
some database parameters:
dbDatabase(dbAccessType type = dbAllAccess,
size_t poolSize = 0, // autodetect size of available memory
size_t dbExtensionQuantum = dbDefaultExtensionQuantum,
size_t dbInitIndexSize = dbDefaultInitIndexSize,
int nThreads = 1);
Database can be opened in readonly mode (dbDatabase::dbReadOnly
access type) or in normal mode allowing modification of database
(dbDatabase::dbAllAccess). When database is opened in readonly
mode, no new class definitions can be added to database and also definition
of existed class and indices can not be altered.
Parameter poolSize specifies number of pages in page pool used
to optimize file IO. GigaBASE is using 8Kb pages. Size of pool should
not be larger than amount of physical memory at the computer and moreover some
amount of memory should be reserved for operating system and other application
data structures. When default value 0 of this parameter is used, GigaBASE will
automatically select page pool size using information about available physical
memory in the system. Current algorithm of page pool size calculation
is the following (it is the subject for change in future):
GigaBASE uses maximal number which is power of two and less than amount of
physical memory in the system unless difference of total amount of
available physical memory and this number is greater than some unused memory
threshold (currently 64Mb). If the difference is greater than threshold value,
then size of pool is taken as size of available physical memory minus
threshold.
Parameter dbExtensionQuantum specifies quantum of extension of
memory allocation bitmap.
Briefly speaking, value of this parameters specifies how much memory
will be allocated sequentially without attempt to reuse space of
deallocated objects. Default value of this parameter is 16 Mb.
See section Memory allocation for more details.
Parameter dbInitIndexSize specifies initial index size.
All objects in GigaBASE are accessed through object index.
There are two copies
of object index: current and committed. Object indices are reallocated on
demand and setting initial index size can only reduce (or increase)
number of reallocations. Default value of this parameter is 64K object
identifiers.
And the last parameter nThreads controls level of query
parallelization. If it is greater than 1, then GigaBASE can start parallel
execution of some queries (including sorting of result). Specified number of
parallel threads will be spawned by GigaBASE engine in this case. Usually
there is no sense to specify the value of this parameter greater than
number of online CPUs in the system. It is also possible to pass
zero as value of the parameter, in this case GigaBASE will automatically detect
number of online CPUs in the system. Number of threads can be also set
by dbDatabase::setConcurrency method at any moment of time.
Class dbDatabase contains static field
dbParallelScanThreshold, which specifies threshold for
number of records records in the table after which query parallelization
is used. Default value of this parameter is 1000.
Database can be opened by open(char const* fileName = NULL)
method. Unlike FastDB, GigaBASE is not needed in database name and only file
name should be specified. No any suffixes are implicitly appended to database
file name. This is the only difference in interfaces to FastDB and GigaBASE.
As far as some operating systems have limitations for maximal file size,
GigaBASE provides way to split one logical data file into several physical
segments (operating systems files). Segments can be located at different
partitions and file systems. Such file is called in GigaBASE
multifile. To create multifile you should specify @
symbol before database file name. In this case GigaBASE will treat this name
as name of the file with multifile segments description. Each line of this file
(except last) should contain name of the operating system file
(or raw partition)
corresponds to the multifile segment and size (in 8Kb pages)
of the segment.
Only the last segment of the multifile can by dynamically extended when
database is grown. That is why it is not necessary to specify size
of the last segment, so last line should contain only the name of the file.
It is possible to specify offset from the beginning of the file (it can be
useful for raw partitions). Offset should be specified as the suffix of the file
name in [] brackets without any spaces between, for example: "/dev/hda0[512]". Unlike size of segment, offset is specified in bytes, not in pages.
Below is the example of multifile description file consisting of two segments represented by physical disk partitions, first of which has size 4Gb:
/dev/hdb1 524288 /dev/hdc1
It is also possible to increase performance by balancing disk load. The idea is the same as with multifile - split the database file into several parts, placed at different disk controllers. But unlike multifile approach, blocks are cyclically distributed among disks - first block at first disk, second block at second disk, ... n-th block at n-disk, n+1 block at 1 disk,... The size of block is currently set to one megabyte (it can be changed with
.RaidBlockSize BLOCK_SIZE
line in description file. The partitions of raid file are specified in the same way as for
multifile - one partition name per line, but no size of segment should be specified (so if no
segment size was specified, then GigaBASE will create RAID file, otherwise - multifile).
Offset within partition can be specified in the same way as for multifile
Method open returns true if database was
successfully opened or false if open operation failed.
In last case database handleError method is called with
DatabaseOpenError error code. Database session can be terminated
by close method, which implicitly commits current transaction.
In multithreaded application each thread, which wants to access database,
should first be attached to it. Method dbDatabase::attach()
allocates thread specific data and attaches thread to the database.
This method is automatically called by open() method, so
there is no reason to call attach() method for the thread
opening database. When thread finishes work with database, it should
call dbDatabase::detach() method. Method
close automatically invokes detach() method.
Method detach() implicitly commits current transaction.
Attempt to access database by detached thread causes assertion failure.
GigaBASE is able to perform compilation and execution of queries in parallel, providing significant increase of performance in multiprocessor systems. But concurrent updates of database are not possible (this is a price for efficient log-less transaction mechanism and zero time recovery). When application wants to modify database (open cursor for update or insert new record in the table), it first locks database in exclusive mode, prohibiting accesses to database by other applications, even for read-only queries. So to avoid blocking of database application for a long time, modification transactions should be done as short as possible. No blocking operations (like waiting input from the user) should be done within transaction.
Using only shared and exclusive locks on database level, allows GigaBASE to almost eliminate overhead of locking and optimize speed of execution of non-conflicting operations. But if many applications simultaneously updates different parts of database, then approach used in GigaBASE will be very inefficient. That is why GigaBASE is most suitable for single-application database access model or for multiple applications with read-dominated access pattern model.
Cursor object should be used only by one thread in a multithreaded application. If there are more than one threads in your applications, use local variables for cursors in each thread. It is possible to share query variables between threads, but take care about query parameters. The query should either has no parameters, or relative form of parameters binding should be used.
The dbDatabase object is shared between all
threads and uses thread specific data to perform query
compilation and execution in parallel with minimal synchronization overhead.
There are few global things, which require synchronization: symbol table,
pool of tree node,... But scanning, parsing and execution of query can
be done without any synchronization, providing high level of concurrency
at multiprocessor systems.
Database transaction is started by first select or insert operation.
If cursor for update is used, then database is locked in exclusive
mode, prohibiting access to the database by other applications and threads.
If read-only cursor is used, then database is locked in shared mode preventing
other application and threads from modifying database, but allowing concurrent
read requests execution. Transaction should be explicitly terminated
either by dbDatabase::commit() method, which fixes all
changes done by transaction in database, or by
dbDatabase::rollback() method which undo all modifications
done by transaction. Method dbDatabase::close() automatically
commits current transaction.
If several threads are concurrently updating database, it will be possible
to increase total performance by using partial transaction commit.
Method dbDatabase::precommit() doesn't flush any changes to the
disk and switch object index. Instead of this it only release locks hold by
transaction allowing other threads to proceed. All cursors opened by the thread
are closed by dbDatabase::precommit() method. When the thread will
access the database next time, it will have to obtain database locks once
again. Using precommit method instead of commit
eliminates disk operations and so dramatically increases performance. But it
is necessary to remember that if application or system fault will take place
after precommit method execution, all changes made by transaction
will be lost.
If you start transaction by performing selection using read-only cursor and
then use cursor for update to perform some modifications of database,
database will be first locked in shared mode and then lock will be upgraded
to exclusive. This can cause deadlock problem if database is simultaneously
accessed by several applications. Imagine that application A starts
read transaction and application B also starts read transaction. Both
of them hold shared locks on the database. If both of them wants to
upgrade their locks to exclusive, they will forever block each other
(exclusive lock can not be granted until shared lock of other process exists).
To avoid such situation try to use cursor for update at the beginning of
transaction or explicitly use dbdatabase::lock() method.
More information about implementation of transactions in GigaBASE can be found
in section Transactions.
It is possible to explicitly lock database by lock() method.
Locking is usually done automatically and there are few cases when
you will want to use this method. It will lock database in exclusive
mode until the end of current transaction.
Backup of database can be done by the following method:
bool dbDatabase::backup(char const* backupFileName);
Backup locks database in shared mode and flush image of database
in main memory to specified file. Because of using of shadow object index,
database file is always in consistent state, so recovery from the backup can
be performed just by renaming backup file (if backup was performed on tape, it
should be first restored to the disk). If multifile was used as database
storage, then simple renaming or copying of backup file is not possible.
GigaBASE provides restore method:
bool dbDatabase::restore(char const* backupFileName,
const* databaseFileName);
This method should be called before opening database, restore of
online database is not possible. If databaseFileName
contains @ in first position, the rest of the name is treated
as the name of the file with multifile segments description (the same as
used by dbDatabase::open method). Database can be also restored
using restore code of subsql utility.
Class dbDatabase is also responsible for handling various
application errors, such as syntax errors in query compilation,
out of range index or null reference access during query execution.
There is virtual method dbDatabase::handleError, which handles
these errors:
virtual void handleError(dbErrorClass error,
char const* msg = NULL,
int arg = 0);
Programmer can derive his own subclass from dbDatabase
class and redefine default reaction on errors.
| Class | Description | Argument | Default reaction |
|---|---|---|---|
| QueryError | query compilation error | position in query string | abort compilation |
| ArithmeticError | arithmetic error during division or power operations | - | terminate application |
| IndexOutOfRangeError | index is out if array bounds | value of index | terminate application |
| DatabaseOpenError | error while database opening | - | open method will return false |
| FileError | failure of file IO operation | error code | terminate application |
| OutOfMemoryError | not enough memory for object allocation | requested allocation size | terminate application |
| Deadlock | upgrading lock cause deadlock | - | terminate application |
| NullReferenceError | null reference is accessed during query execution | - | terminate application |
GigaBASE provides multithreaded server for handling client CLI sessions. This server can be
started from SubSQL utility by
start server 'HOST:PORT' <number-of-threads> command.
This server will accept local (within one system) and global clients connections and
attach one thread from the threads pool to each connection. The size of thread's pool
is controlled by number-of-threads parameters. But the server can spawn more
than specified number of threads if there are many active connections. A thread is attached
to the client until the end of the session. If session is abnormally terminated, all
changes made by the client are rollbacked. The server can be stopped by correspondent
stop server 'HOST:PORT' command.
Gigabase CLI API also optionally supports authentication. To enable it, you should recompile
GigaBASE with SECURE_SERVER macro defined. When this macro is defined,
the table UserInfo is created at server. cli_open method takes
user_name and password parameters which are passed to the server.
When connection is established, server searches in UserInfo table for the user with
specified name and password. If the user is not found or password doesn't match, the server returns
cli_login_failed code to the client. To enter new user or remove old one, database
administrator can use SubSQL insert and remove commands. As far as update command is not currently
implemented, you will have to perform remove and insert if you what to change users password.
Do not forget to commit your changes, otherwise all clients will be blocked.
| Error code | Description |
|---|---|
| cli_ok | Successful completion |
| cli_bad_address | Invalid format of server URL |
| cli_connection_refused | Connection with server could not be established |
| cli_bad_statement | Text of SQL statement is not correct |
| cli_parameter_not_found | Parameter was not found in statement |
| cli_unbound_parameter | Parameter was not specified |
| cli_column_not_found | No such column in the table |
| cli_incompatible_type | Conversion between application and database type is not possible |
| cli_network_error | Connection with server is broken |
| cli_runtime_error | Error during query execution |
| cli_bad_descriptor | Invalid statement/session description |
| cli_unsupported_type | Unsupported type for parameter or column |
| cli_not_found | Record was not found |
| cli_not_update_mode | Attempt to update records selected by view only cursor |
| cli_table_not_found | There is no table with specified name in the database |
| cli_not_all_columns_specified | Insert statement doesn't specify values for all table columns |
| cli_not_fetched | cli_fetch method was not called |
| cli_already_updated | cli_update method was invoked more than once for the same record |
| cli_login_failed | Login to the server is failed |
| cli_empty_parameter | Parameter is not assigned a value |
| cli_closed_connection | Access to the closed connection |
| cli_table_already_exists | Attempt to create existed table |
| cli_not_implemented | Function is not implemented |
| Type | Description | Size |
|---|---|---|
| cli_oid | Object identifier | 4 |
| cli_bool | Boolean type | 1 |
| cli_int1 | Tiny integer type | 1 |
| cli_int2 | Small integer type | 2 |
| cli_int4 | Integer type | 4 |
| cli_int8 | Big integer type | 8 |
| cli_real4 | Single precision floating point type | 4 |
| cli_real8 | Double precision floating point type | 8 |
| cli_decimal | Decimal numeric type (number is represented as zero terminated ASCII string) | 1*N |
| cli_asciiz | Zero terminated string of bytes | sizeof(char_t)*N |
| cli_pasciiz | Pointer to zero terminated string (pointer should be initialized by application) | sizeof(char_t)*N |
| cli_cstring | String with counter (see declaration of cli_ctring_t in cli.h). The terminated null character is not stored. When cli_cstring is used as fetched column type, the pointer is set to the internal buffer, so no memory should be allocated and deallocated to hold value. | 8 |
| cli_array_of_oid | Array of references | 4*N |
| cli_array_of_bool | Array of booleans | 1*N |
| cli_array_of_int1 | Array of tiny integers | 1*N |
| cli_array_of_int2 | Array of small integers | 2*N |
| cli_array_of_int4 | Array of integers | 4*N |
| cli_array_of_int8 | Array of big integers | 8*N |
| cli_array_of_real4 | Array of reals | 4*N |
| cli_array_of_real8 | Array of long reals | 8*N |
| cli_any | can be used only for binding columns, server will use the same type for column as stored in the database | ? |
| cli_datetime | time in seconds since 00:00:00 UTC, January 1, 1970. | 4 |
| cli_autoincrement | column automatically assigned value during record insert | 4 |
| cli_rectangle | rectangle (by default R2 rectangle with int4 coordinates | dimension*2*sizeof(cli_coord_t) |
int cli_open(char const* server_url, int max_connect_attempts, int reconnect_timeout_sec, char_t const* user_name, char_t const* password, int pooled_connection);
server_url - zero terminated string with server address and port,
for example "localhost:5101" or "195.239.208.240:6100". It can contain multiple addresses, in this case
replication socket will be created. Data written to such socket will be broadcasted to all specified servers.
And read from this socket cause receiving response from each server and checking that them are equal.
If connection with one of the server is broken or response received from this server is not equal to other
responses, then this server is excluded from list of available servers. Client can continue work with database until
at least one server is available and it is possible to choose correct response (quorum is achieved).
max_connect_attempts - number of attempts to establish connection
reconnect_timeout_sec - timeout in seconds between connection attempts
reconnect_timeout_sec - timeout in seconds between connection attempts
user_name - user name for login
password - password for login
pooled_connection - if not 0, then connection will be allocated from the connection pool
>= 0 - connection descriptor to be used in all other cli calls
< 0 - error code as described in cli_result_code enum
int cli_close(int session);
session - session descriptor returned by cli_open
cli_result_code enum
void cli_clear_connection_pool();
int cli_statement(int session, char const* stmt);
session - session descriptor returned by cli_open
stmt - zero terminated string with SubSQL statement
>= 0 - statement descriptor
< 0 - error code as described in cli_result_code enum
int cli_parameter(int statement, char const* param_name, int var_type, void* var_ptr);
statement - statement descriptor returned by cli_statement
param_name - zero terminated string with parameter name.
Parameter name should start with '%'
var_type - type of variable as described in cli_var_type enum.
Only scalar and zero terminated string types are supported.
var_ptr - pointer to the variable
cli_result_code enum
int cli_column(int statement, char const* column_name, int var_type, int* var_len, void* var_ptr);
statement - statement descriptor returned by cli_statement
column_name - zero terminated string with column name
var_type - type of variable as described in cli_var_type enum
var_len - pointer to the variable to hold length of array variable.
This variable should be assigned the maximal length of the array/string buffer,
pointed by var_ptr. After the execution of the statement it is assigned the
real length of the fetched array/string. If it is large than length of the buffer,
then only part of the array will be placed in the buffer, but var_len
still will contain the actual array length.
var_ptr - pointer to the variable
cli_result_code enum
typedef void* (*cli_column_set)(int var_type, void* var_ptr, int len); typedef void* (*cli_column_get)(int var_type, void* var_ptr, int* len); int cli_array_column(int statement, char const* column_name, int var_type, void* var_ptr, cli_column_set set, cli_column_get get);
statement - statement descriptor returned by cli_statement
column_name - zero terminated string with column name
var_type - type of variable as described in cli_var_type enum
var_ptr - pointer to the variable
set - function which will be called to construct fetched field.
It receives pointer to the variable, length of the fetched array and returns pointer
to the array's elements.
get - function which will be called to update the field in the
database. Given pointer to the variable, it should return pointer to the array elements
and store length of the array to the variable pointer by len parameter
cli_result_code enum
typedef void* (*cli_column_set_ex)(int var_type, void* var_ptr, int len, char const* column_name, int statement, void const* data_ptr); typedef void* (*cli_column_get_ex)(int var_type, void* var_ptr, int* len, char const* column_name, int statemen); int cli_array_column(int statement, char const* column_name, int var_type, void* var_ptr, cli_column_set_ex set, cli_column_get_ex get);
statement - statememt descriptor returned by cli_statement
column_name - zero terminated string with column name
var_type - type of variable as described in cli_var_type enum
var_ptr - pointer to the variable
set - function which will be called to construct fetched field.
It receives type of the vartiable, pointer to the variable, length of the fetched array, name of the fetched column,
statement descriptor and pointer to the array data. If this method returns not NULL pointer,
database will copy unpacked array to the returned location. Otherwise it is assumed that
function handle data itself.
get - function which will be called to update the field in the
database. Given type of the vartiable, pointer to the variable, column name and statment descriptor,
it should return pointer to the array elements and store length of the array to the variable pointer by len parameter
cli_result_code enum
enum cli_cursor_type_t {
cli_cursor_view_only,
cli_cursor_for_update,
/**
* Execute query incrementally
*/
cli_cursor_incremental,
/** Detached cursor commits transaction immediately after execution of the query,
* and then temporarily start new transaction for fetching each element.
* Such cursor should be used in conjunction with DO_NOT_REUSE_OID_WITHIN_SESSION to detect
* removed objects.
* Please notice that this cursor is static - it only executed from the selection removed objects,
* but doesn't try to add the selection new records which match search criteria or remove
* record which do not match this criteria any more.
*/
cli_cursor_detached
};
int cli_fetch(int statement, int cursor_type);
statement - statement descriptor returned by cli_statement
cursor_type - combination of cursor type described by cli_cursor_type_t enum
>= 0 - success, for select statements number of fetched rows is returned
< 0 - error code as described in cli_result_code enum
int cli_insert(int statement, cli_oid_t* oid);
statement - statement descriptor returned by cli_statement
oid - object identifier of created record.
cli_result_code enum
int cli_get_first(int statement);
statement - statement descriptor returned by cli_statement
cli_result_code enum
int cli_get_last(int statement);
statement - statement descriptor returned by cli_statement
cli_result_code enum
int cli_get_next(int statement);
cli_fetch function call, is will fetch the first record in selection.
statement - statement descriptor returned by cli_statement
cli_result_code enum
int cli_get_prev(int statement);
cli_fetch function call, is will fetch the last record in selection.
statement - statement descriptor returned by cli_statement
cli_result_code enum
cli_oid_t cli_get_oid(int statement);
statement - statement descriptor returned by cli_statement
int cli_update(int statement);
cursor_type parameter of cli_fetch to
cli_cursor_for_update in order to be able to perform updates.
Updated value of row fields will be taken from bound column variables.
statement - statement descriptor returned by cli_statement
cli_result_code enum
int cli_remove(int statement);
cursor_type parameter of cli_fetch to
cli_cursor_for_update in order to be able to remove records.
statement - statement descriptor returned by cli_statement
cli_result_code enum
int cli_close_cursor(int statement);
statement - statement descriptor returned by cli_statement
cli_result_code enum
int cli_free(int statement);
statement - statement descriptor returned by cli_statement
cli_result_code enum
int cli_commit(int session);
session - session descriptor as returned by cli_open
cli_result_code enum
int cli_abort(int session);
session - session descriptor as returned by cli_open
cli_result_code enum
int cli_show_tables(int session, cli_table_descriptor** tables);
typedef struct cli_table_descriptor {
char const* name;
} cli_table_descriptor;
session - session descriptor as returned by cli_open
tables - address of the pointer to the array with table descriptors.
cli_show_tables uses malloc to allocate this array, and it should be deallocated by application using free() function.
>= 0 - number of tables in the database (Metatable is not returned/counted)
< 0 - error code as described in cli_result_code enum
int cli_describe(int session, char const* table, cli_field_descriptor** fields);
typedef struct cli_field_descriptor {
enum cli_var_type type;
int flags;
char_t const* name;
char_t const* refTableName;
char_t const* inverseRefFieldName;
} cli_field_descriptor;
session - session descriptor as returned by cli_open
table - name of the table
fields - address of the pointer to the array with field descriptors. cli_describe uses malloc to allocate this array, and it should be deallocated by application using free() function.
>= 0 - number of fields in the table
< 0 - error code as described in cli_result_code enum
int cli_describe_layout(int session, char_t const* table, cli_field_layout** fields, int* rec_size);
typedef struct cli_field_layout {
cli_field_descriptor desc;
int offs;
int size;
} cli_field_layout;
session - session descriptor as returned by cli_open
table - name of the table
fields - address of the pointer to the array with field layout descriptors. cli_describe_layout uses malloc to allocate this array, and it should be deallocated by application using free() function.
rec_size - extracted record size (application can use this size to allocate buffer for cli_execute_query function)
>= 0 - number of fields in the table
< 0 - error code as described in cli_result_code enum
int cli_create_table(int session,
char_t const* tableName,
int nFields,
cli_field_descriptor* fields);
session - session descriptor as returned by cli_open
tableName - name of the created table
nFields - number of columns in the table
fields - array with table columns descriptors. Descriptor is has the following structure:
enum cli_field_flags {
cli_hashed = 1, /* field should be indexed usnig hash table */
cli_indexed = 2, /* field should be indexed using B-Tree */
cli_case_insensitive = 4 /* index is case insensitive */
};
typedef struct cli_field_descriptor {
enum cli_var_type type;
int flags;
char_t const* name;
char_t const* refTableName;
char_t const* inverseRefFieldName;
} cli_field_descriptor;
cli_result_code enum
int cli_alter_table(int session,
char_t const* tableName,
int nFields,
cli_field_descriptor* fields);
session - session descriptor as returned by cli_open
tableName - name of the altered table
nFields - number of columns in the table
fields - array with table columns descriptors. Descriptor is has the following structure:
enum cli_field_flags {
cli_hashed = 1, /* field should be indexed usnig hash table */
cli_indexed = 2, /* field should be indexed using B-Tree */
cli_case_insensitive = 4 /* index is case insensitive */
};
typedef struct cli_field_descriptor {
enum cli_var_type type;
int flags;
char_t const* name;
char_t const* refTableName;
char_t const* inverseRefFieldName;
} cli_field_descriptor;
cli_result_code enum
int cli_drop_table(int session, char_t const* tableName);
session - session descriptor as returned by cli_open
tableName - name of the created table
cli_result_code enum
int cli_alter_index(int session, char_t const* tableName char_t const* fieldName, int newFlags);
session - session descriptor as returned by cli_open
tableName - name of the created table
fieldName - name of the field
newFlags - new flags of the field, if index exists for this field, but is not specified in
newFlags mask, then it will be removed; if index not exists, but is
specified in newFlags mask, then it will be created.
cli_result_code enum
int cli_freeze(int statement);
statement - statememt descriptor returned by cli_statement
cli_result_code enum
int cli_unfreeze(int statement);
statement - statememt descriptor returned by cli_statement
cli_result_code enum
int cli_seek(int statement, cli_oid_t oid);
statement - statememt descriptor returned by cli_statement
oid - object identifier of the record to which cursor should be positioned
>= 0 - success, position of the record in the selection
< 0 - error code as described in cli_result_code enum
int cli_skip(int statement, int n);
statement - statememt descriptor returned by cli_statement
n - number of objects to be skippedn is positive, then this function has the same effect as executing cli_get_next() function n times.
n is negative, then this function has the same effect as executing cli_get_prev() function -n times.
n is zero, this method just reloads current record
cli_result_code enum
cli.lib and if you want to
access database locally - link it with gigabase.lib.
To create local session you should use cli_create function instead of cli_open.
Calling cli_create when your application is linked with cli.lib or
cli_open when it is linked with gigabase.lib cause cli_bad_address
error.
int cli_create(char_t const* databasePath,
unsigned transactionCommitDelay,
int openAttr,
size_t poolSize);
databasePath - path to the database file
transactionCommitDelay - transaction commit delay (specify 0 to disable)
openAttr - ask of cli_open_attributes. You can specify combination of the following
attributes:cli_open_default
cli_open_readonly open database in readonly mode
cli_open_truncate truncate database file on open
cli_open_no_buffering do not use file system cache
cli_open_multiclient use OS file locks to provide multiclient access to the database
poolSize - size of page pool (in pages), specify 0 to let GigaBASE automatically detect pool size
>= 0 - connection descriptor to be used in all other cli calls
< 0 - error code as described in cli_result_code enum
int cli_attach(int session);
session - session descriptor as returned by cli_open
cli_result_code enum
int cli_detach(int session, int detach_mode);
session - session descriptor as returned by cli_open
detach_mode - bit mask representing detach mode
enum cli_detach_mode {
cli_commit_on_detach = 1,
cli_destroy_context_on_detach = 2
};
cli_result_code enum
typedef void* cli_transaction_context_t; cli_transaction_context_t cli_create_transaction_context();
int cli_join_transaction(int session, cli_transaction_context_t ctx);
session - session descriptor as returned by cli_open
ctx - transaction context created by cli_create_transaction_context
cli_result_code enum
void cli_remove_transaction_context(cli_transaction_context_t ctx);
ctx - transaction context created by cli_create_transaction_context
int cli_prepare_query(int session, char_t const* query);
session - session descriptor as returned by cli_open
query - query string with optional parameters. Parameters are specified
as '%T' where T is one or two character code of parameter type using the same notation as in printf:
| %d or %i | cli_int4_t |
| %f | cli_int8_t |
| %Li, %li, %ld or %Ld | cli_int8_t |
| %p | cli_oid_t |
| %s | char_t* |
>= 0 - statement descriptor
< 0 - error code as described in cli_result_code enum
int cli_execute_query(int statement, int cursor_type, void* record_struct, ...)
cli_ types
in declaring structure and table fields. For array types, you should use cli_array_t
structure. Strings should be represented as char* and programmer should not try to
deallocate them or copy this pointer and access it outside context of the current record.
statement - statement descriptor returned by cli_prepare_query
cursor_type - combination of cursor type described by cli_cursor_type_t enum
record_struct - structure to receive selected record fields.
... - varying list of query parameters
cli_result_code enum
int cli_execute_query_ex(int statement, int cursor_type, void* record_struct, int n_params, int* param_types, void** param_values)
cli_ types
in declaring structure and table fields. For array types, you should use cli_array_t
structure. Strings should be represented as char* and programmer should not try to
deallocate them or copy this pointer and access it outside context of the current record.
statement - statement descriptor returned by cli_prepare_query
cursor_type - combination of cursor type described by cli_cursor_type_t enum
record_struct - structure to receive selected record fields.
n_params - number of parameters
param_types - types of parameters (cli_var_type)
param_values - array of pointers to parameter values
cli_result_code enum
int cli_insert_struct(int session, char_t const* table_name, void* record_struct, cli_oid_t* oid);
cli_ types
in declaring structure and table fields. For array types, you should use cli_array_t
structure. Strings should be represented as char_t*.
session - session descriptor as returned by cli_open
table_name - name of the destination table
record_struct - structure specifying value of record fields
oid - pointer to the location to receive OID of created record (may be NULL)
cli_result_code enum
typedef enum cli_export_method {
cli_export_all, /* just export all tables */
cli_export_all_expect, /* '-' Export all tables except the named tables */
cli_export_named_only /* '+' Only export explicitly named tables */
} cli_export_method;
int cli_xml_export(int session, FILE* out, char_t const* const* tables, int n_tables, cli_export_method method);
session - session descriptor as returned by cli_open
out - output stream
tables - list of table names
n_tables - number of tables
method - export method
cli_result_code enum
int cli_xml_import(int session, FILE* in);
session - session descriptor as returned by cli_open
in - input stream
cli_result_code enum
| Error code | Description |
|---|---|
| cli_ok | Successful completion |
| cli_bad_address | Invalid format of server URL |
| cli_connection_refused | Connection with server could not be established |
| cli_bad_statement | Text of SQL statement is not correct |
| cli_parameter_not_found | Parameter was not found in statement |
| cli_unbound_parameter | Parameter was not specified |
| cli_column_not_found | No such column in the table |
| cli_incompatible_type | Conversion between application and database type is not possible |
| cli_network_error | Connection with server is broken |
| cli_runtime_error | Error during query execution |
| cli_close_statement | Access to the closed statement object |
| cli_unsupported_type | Unsupported type for parameter or column |
| cli_not_found | Record was not found |
| cli_not_update_mode | Attempt to update records selected by view only cursor |
| cli_table_not_found | There is no table with specified name in the database |
| cli_not_all_columns_specified | Insert statement doesn't specify values for all table columns |
| cli_not_fetched | cli_fetch method was not called |
| cli_already_updated | cli_update method was invoked more than once for the same record |
| cli_empty_parameter | Attempt to bind parameters which was not defined (assigned value or set type) |
| cli_closed_connection | Access to the closed connection object |
| cli_table_already_exists | Attempt to create table when table with such name already exists |
| cli_not_implemented | This function is not implemented by CLI API |
function new_connection($host_address, $host_port, $user_name = "guest", $password = "");
host_address - string with server host name
host_port - integer number with server port
user_name - user name to login
password - password to login
function release_connection($conxn);
conxn - connection been placed in the pool.
function close();
function open($host_address, $host_port, $user_name = "guest", $password = "");
host_address - string with server host name
host_port - integer number with server port
user_name - user name to login
password - password to login
function close();
function create_statement($stmt);
stmt - string with SubSQL statement. Parameter names should
start with '%' character.
gb_statement object or null if
specified statement is invalidfunction insert($obj, $oid);
obj - object to be stored in the database. Name of the
object class should match with the name of some of database tables.
oid - reference to the variable to receive OID of created object.
function commit();
function rollback();
function show_tables($tables)
table - reference of the variable to receive
array with table names
function describe_table($name, $table)
name - name of the table
table - reference of the variable to receive associative
array bind_parameter function bind_parameter($name, $binding);
name - string with name of parameter. Parameter name should
start with '%'.
binding - reference to the parameter variable.
function bind_column($name, $binding);
name - string with name of the column.
binding - reference to the column variable.
function bind_object($name, $obj);
binding - reference to the object variable.
function bind_array($name, $arr);
binding - reference to the array variable.
function fetch($for_update = false);
for_update - true if fetched rows will be updated
function fetch_objects($arr);
arr - reference to the variable to receive
array with all selected objects
function fetch_tuples($arr);
arr - reference to the variable to receive
array with all selected tuples
function insert($oid);
oid - reference to variable to receive OID of created object
function get_first();
function get_last();
function get_next();
fetch method call,
is will fetch the first record in the selection.
function get_prev();
fetch method call, is will fetch the last record in selection.
function get_oid();
gb_reference object with OID or null if no object was selected
function update();
for_update parameter of fetch to 1 in order to be able
to perform updates. Updated value of row fields will be taken
from bound column variables.
function remove();
for_update parameter of fetch to 1 in order to be able to remove records.
function free();
gb_reference is used to represent database references.
It contains the single fields - integer oid (object identifier).
But in many cases it is acceptable to loose changes for few last seconds (but preserving consistency of the database). With this assumption, database performance can be significantly increased. GigaBASE provides "delayed transaction commit model" for such applications. When commit transaction delay is non zero, database doesn't perform commit immediately, instead of it delay it for specified timeout. After expiration of this timeout, transaction is normally committed, so it ensures that only changes done within specified timeout can be lost in case of system crash.
If thread, which has initiated delayed transaction, starts new transactions before delayed commit of transaction is performed, then delayed commit operation is skipped. So GigaBASE is able to group several subsequent transactions performed by on client into the large single transaction. And it will greatly increase performance, because it reduces number of synchronous writes and number created shadow pages (see section Transactions).
If some other client tries to start transaction before expiration of delayed commit timeout, then GigaBASE force delayed commit to proceed and release resource for another thread. So concurrency is not suffered from delayed commit.
By default delayed commits are disabled (timeout is zero). You can sepcify commit delay
parameter as second optional argument of dbDatabase::open method.
In SubSQL utility, it is possible to specify value of transaction commit
delay by setting "GIGABASE_COMMIT_DELAY" environment variable (seconds).
Transaction commit scheme used in GigaBASE guaranty recovery after software and hardware fault if image of the database at the disk was not corrupted (all information which was written to the disk can be correctly read). If for some reasons, database file is corrupted, then the only way to recover database is use backup (hoping that it was performed not so long time ago).
Backup can be done by just copying database file when database is offline.
Class dbDatabase provides backup method which is able to perform online backup,
which doesn't require stopping of the database. It can be called at any time by programmer.
But going further, GigaBASE provides backup scheduler, which is able to perform backup automatically.
The only things needed - name of the backup file and interval of time between backups.
The method dbDatabase::scheduleBackup(char_t const* fileName, time_t period)
spawns separate thread which performs backups to the specified location with specified period (in seconds).
If fileName ends with "?" character, then data of backup initiation is appended to the file
name, producing the unique file name. In this case all backup files are kept on the disk (it is
responsibility of administrator to remove too old backup files or transfer them to another media).
Otherwise backup is performed to the file with fileName + ".new" name, and after completion
of backup, old backup file is removed and new file is renamed to fileName.
Also in last case, GigaBASE will check the creation date of the old backup file (if exists) and adjust
wait timeout in such way, that delta of time between backups will be equal to specified period
(so if database server is started only for 8 hours per day and backup period is 24 hours, then
backup will be performed each day, unlike scheme with uniquely generated backup file names).
It is possible to schedule backup processing in SubSQL utility by setting
GIGABASE_BACKUP_NAME environment variable.
Period value is taken from GIGABASE_BACKUP_PERIOD environment variable if specified, otherwise it
is set to one day. To recover from backup it is enough to copy some of the backup files instead of
corrupted database file.
= < > <= >= between like)
= > >= < <= between
or operation is like and pattern string contains no wildcard
symbol % or _ in first position.
If index is used to search prefix of like expression, and
suffix is not just '%' character, then index search operation can return
more records than really match the pattern. In this case we should filter
index search output by applying pattern match operation.
When search condition is disjunction of several subexpressions
(expression contains several alternatives combined by or
operator), then several indices can be used for query execution.
To avoid record duplicates in this case, bitmap is used in cursor
to mark records already included in the selection.
If search condition requires sequential table scan, B-tree index
still can be used if order by clause contains the single
record field for which B-tree index is defined. As far as sorting is very
expensive operation, using of index instead of sorting significantly
reduce time of query execution.
It is possible to check which indices are used for query execution
and number of probes done during index search be compiling GigaBASE
with option -DDEBUG=DEBUG_TRACE. In this case GigaBASE will
dump trace information about database functionality including information
about indices.
When record with declared relations is inserted in the table, inverse references in all tables been in relation with this records are updated to point to this record. When record is updated and field specifying records relationships are changed, then inverse references are also reconstructed automatically, by removing references to updated record from that records which are no more in relation with this record and setting inverse references to updated record for new records included in relation. And when record is deleted from the table, references to it are removed from all inverse reference fields.
Due to efficiency reasons, GigaBASE is not able to guaranty consistency of all references. If you remove record from the table, there are still can be references to removed record in database. Accessing these references can cause unpredictable behavior of application and even database corruption. Using inverse references allows to eliminate this problem, because all references will be updated automatically and consistency of references is preserved.
Lets use the following table definitions as example:
class Contract;
class Detail {
public:
char const* name;
char const* material;
char const* color;
real4 weight;
dbArray< dbReference<Contract> > contracts;
TYPE_DESCRIPTOR((KEY(name, INDEXED),
KEY(material, INDEXED),
KEY(color, INDEXED),
KEY(weight, INDEXED),
RELATION(contracts, detail)));
};
class Supplier {
public:
char const* company;
char const* location;
bool foreign;
dbArray< dbReference<Contract> > contracts;
TYPE_DESCRIPTOR((KEY(company, INDEXED),
KEY(location, INDEXED),
FIELD(foreign),
RELATION(contracts, supplier)));
};
class Contract {
public:
dbDateTime delivery;
int4 quantity;
int8 price;
dbReference<Detail> detail;
dbReference<Supplier> supplier;
TYPE_DESCRIPTOR((KEY(delivery, INDEXED),
KEY(quantity, INDEXED),
KEY(price, INDEXED),
RELATION(detail, contracts),
RELATION(supplier, contracts)));
};
In this example there are one-to-many relations between tables
Detail-Contract and Supplier-Contract. When Contract
record is inserted in database, it is necessary only to set references
detail and supplier to correspondent
records of Detail and Supplier tables.
Inverse references contracts in these records will be updated
automatically. The same is happened when Contract record is
removed, references to removed record will be automatically excluded
from contracts field of referenced Detail and
Supplier records.
Moreover using inverse reference allows to chose more effective plan of query execution. Consider the following query selecting all details shipped by some company:
q = "exists i:(contracts[i].supplier.company=",company,")";
The straightforward approach to execution of this query is scanning
Detail table and testing each record for this condition.
But using inverse reference we can choose another approach: perform
index search in Supplier table for records with specified
company name and then use inverse references to locate records from
Detail table been in transitive relation with
selected supplier records. Certainly we should eliminate duplicates of
records, which can appear because company can ship a number of different
details. This is done by bitmap in cursor object.
As far as index search is significantly faster than sequential search
and accessing record by reference is very fast operation, total
time of such query execution is much shorter comparing with
straightforward approach.Starting from 1.21 version GigaBASE supports cascade deletes. If field is declared using OWNER macro, the record is treated as owner of the hierarchical relation. When the owner records is removed all members of this relation (records referenced from the owner) will be automatically removed. If member record of the relation should contain reference to the owner record, this field should be declared using RELATION macro.
To split table scan, GigaBASE starts N threads each of them tests N-s record of the table (i.e. thread number 0 test records 0,N,2*N,... thread number 1 test records 1,1+N,1+2*N,... and so on). Each thread builds its own list of selected records. After termination of all threads, these lists are concatenated to construct the single result list.
If result should be sorted, then each thread, after finishing the table scan, sorts the records it selected. After termination of all threads, their lists are merged (as it is done with external sort).
Parallel query execution is controlled by two parameters: number of spawned
threads and parallel search threshold. First is specified in
dbDatabase class constructor or set by
dbDatabase::setConcurrency method. Zero value of this parameter
asks GigaBASE to automatically detect number of online CPUs in the system and
spawn exactly this number of threads. By default number of threads is set to 1,
so no parallel query execution takes place.
Parallel search threshold parameter specifies minimal number of records in the
table for which parallelization of query can improve query performance
(starting of threads has its own overhead). This parameter is static
component of dbDatabase class and can be changed by application at
any moment of time.
Parallel query execution is not possible when:
dbDatabase::dbParallelScanThreshold;
start from part;
Replication is implemented in dbReplicatedDatabase class
which is derived from dbDatabase class. In addition to
parameters passed to dbDatabase constructor, the constructor
of dbReplicatedDatabase class allows to specify
replication manager class. Also it always opens the database in full-access
mode, so this parameter is skipped.
Replication at master and slaves nodes is start by open method.
It accepts to additional parameters: master host address
(included port separated from name by ':') and number of replicas.
If number of replicas is zeros, then this database is assumed to be slave
and tries to establish connection with master.
Otherwise node is assumed to be master and it tries to accept specified number
of connections with slaves. Control is not returned from open method
until all connections are established.
Replication at master is implicitly stopped by close method
or can be explicitly terminated by stopMasterReplication()
method. Replication at slave is terminated when it receives termination
request from master or when connection with master is broken.
If slave receives termination request from master, it means that currently
it has completely the same database as master and can continue work
with the database (even update it). But if slave detects disconnection with
client, then its database is inconsistent and recovery should be performed
(slave should exit without closing and the try to open database once again).
GigaBASE replication mechanism works in the assumption that at startup content of database files of master and slaves is identical. GigaBASE doesn't provide mechanism for synchronizing state of repositories if some node was unavailable for some time and then tries to become new slave or master. In this case database file should be replicated at the node using standard network file copy mechanism (certainly all databases should be offline at this moment).
Slave can inspect and control replication process by implementing
dbReplicationManager abstract class and passing it to
dbReplicatedDatabase constructor. This class defines the following
abstract method:
| Method | Description |
|---|---|
bool connectionBroken(char* hostName) | This method is invoked when connection with master/slave is broken |
void transactionCommitted() | This method is invoked when transaction is completely transferred to the client. This method can be used for example to perform refresh in GUI component |
void replicationEnd() | This method is called when slave receives replication end request from master |
bool preserveSlaveConsistency() | This method is used to specify transaction policy at slave. If this method returns true, then GigaBASE will preserve consistency of slave replica of the database. So in case of master or slave crash, it will be possible to recover and continue work with slave database. If this method returns false, the slave performance is greatly increased (because of avoiding flushing file buffers to the disk), but in case of fault, slave database may be stayed in inconsistent state |
Example of replication can be found in testreplic.cpp example.
At master you should launch it with the following command line:
testreplic masterN [master-host-address] [number-of-iterations]
where N - is positive integer specifying number of replicas
At slave(s) node it can be run with the following command:
testreplic slave [master-host-address]
GiSTstore interface which is
responsible for storing/retrieving GiST pages in database. GigaBASE includes
release 0.9 beta 1 of the GiST C++ library with additional GiSTdb.cpp and
GiSTdb.h files and changed BTree, RTree and RSTree examples, patched to work
with GigaBASE instead of plain fails.GiST documentation is available here
First of all you should create database. So open database in standard
way and perform it's initialization. Then you should use compressdb
utility to compress database file. It is command line utility, first mandatory parameter
specifies path to the database file, second optional parameter specifies size of compressed segment
and third optional parameter - compression level. This utility creates compressed file in the same
directory where original file is located but with ".dbz" extension.
This utility split database files into segments
and compress independently each of them. At the beginning of the file it writes table
of segment offsets in the compressed file. This table is used to translate offsets
in the original (uncompressed) file into offsets in the compressed file.
When compressed file is ready, you can open it using dbZipFile.
Please look at testblob example. Been built with USE_ZLIB option, it uses dbZipFile
and works with compressed database. To build it execute "make testblobzip".
First you should fill database, by running testblob and specifying list of files to
be placed in the database. Then you should convert database using convertdb
utility. After it you can run testblobzip which will extract data from the compressed
database. In case of placing in the database five files with the following command:
testblob configure database.cpp GigaBASE.htm aclocal.m4 ltmain.sh
...size of database file is 9871360 (9Mb). After compression its size is 301506 (0.3Mb).
GigaBASE performs cyclic scanning of bitmap pages. It keeps identifier
of current bitmap page and current position within the page. Each time
when allocation request arrives, scanning of the bitmap starts from the
current position.
When last allocated bitmap page is scanned, scanning continues from the
beginning (from the first bitmap page) and until current position.
When no free space is found after full cycle through all bitmap pages,
new bulk of memory is allocated. Size of extension is maximum of
size of allocated object and extension quantum. Extension quantum is parameter
of database, specified in constructor. Bitmap is extended to be able to map
additional space. If virtual space is exhausted and no more
bitmap pages can be allocated, then OutOfMemory error
is reported.
Allocation memory using bitmap provides high locality of references (objects are mostly allocated sequentially) and also minimizes number of modified pages. Minimization of number of modified pages is significant when commit operation is performed and all dirty pages should be flushed on the disk. When all cloned objects are placed sequentially, number of modified pages is minimal and so transaction commit time is also reduced. Using extension quantum also helps to preserve sequential allocation. Once bitmap is extended, objects will be allocated sequentially until extension quantum will be completely used. Only after reaching the end of the bitmap, scanning restarts from the beginning searching for holes in previously allocated memory.
To reduce number of bitmap pages scans, GigaBASE associates descriptor with each page, which is used to remember maximal size of the hole on the page. Calculation of maximal hole size is performed in the following way: if object of size M can not be allocated from this bitmap pages, then maximal hole size is less than M, so M is stored in the page descriptor if previous value of descriptor is large than M. For next allocation of object of size greater or equal than M, we will skip this bitmap page. Page descriptor is reset when some object is deallocated within this bitmap page.
Some database objects (like hash table pages) should be aligned on page boundary to provide more efficient access. GigaBASE memory allocator checks requested size and if it is aligned on page boundary, then address of allocated memory segment is also aligned on page boundary. Search of free hole will be done faster in this case, because GigaBASE increases step of current position increment according to the value of alignment.
To be able to deallocate memory used by object, GigaBASE needs to keep somewhere information about object size. GigaBASE memory allocator deals with two types of objects - normal table records and page objects. All table records are prepended by record header, which contains record size and pointer of L2-list linking all records in the table. So size of the table record object can be extracted from record header. Page objects always occupies the whole database page are are allocated at the positions aligned on page boundary. Page objects has no headers. GigaBASE distinguish page objects with normal object by using special marker in object index.
By default maximal database size supported by GigaBASE is limited to 4Gb.
It is possible to increase (or reduce) this value by specifying
values of dbDatabaseOffsetBits parameter. Default value of this
parameter is 32. When value of this parameter is not greater than 32,
GigaBASE will use 4 byte integers to represent offsets with database
This two times reduce size of object index and also number of bitmap page
handles reserved in index. By default GigaBASE is not able to handle more than
1Gb objects. It is possible to overcome this limitation by specifying value of
dbDatabaseOidBits parameter greater than 32.
When object is modified first time, it is cloned (copy of the object is created) and object handle in current index is changed to point to newly created object copy. And shadow index still contains handle which points to the original version of the object. All changes are done with the object copy, leaving original object unchanged. GigaBASE marks in special bitmap page of the object index, which contains modified object handle.
When transaction is committed, GigaBASE first checks if size of object index was increased during current transaction. If so, it also reallocates shadow copy of object index. Then GigaBASE frees memory for all "old objects", i.e. objects which was cloned within transaction. Memory can not be deallocated before commit, because we wants to preserve consistent state of the database by keeping cloned object unchanged. If we deallocate memory immediately after cloning, new object can be allocated at the place of cloned object and we loose consistency. As far as memory deallocation is done in GigaBASE by bitmap using the same transaction mechanism as for normal database objects, deallocation of object space will require clearing some bits in bitmap page, which also should be cloned before modification. Cloning bitmap page will require new space for allocation the page copy, and we can reuse space of deallocated objects. And it is not acceptable due to the reason explained above - we will loose database consistency. That is why deallocation of object is done in two steps. When object is cloned, all bitmap pages used for marking objects space, are also cloned (if not not cloned before). So when transaction is committed, we only clear bits in bitmap pages and no more requests for allocation memory can be generated at this moment.
After deallocation of old copies, GigaBASE flushes all modified pages on disk to synchronize content of the memory and disk file. After that GigaBASE changes current object index indicator in database header to switch roles of the object indices. Now object index, which was current becomes shadow, and shadow index becomes current. Then GigaBASE again flushes modified page (i.e. page with database header) on disk, transferring database to new consistent state. After that GigaBASE copies all modified handles from new object index to object index which was previously shadow and now becomes current. At this moment contents of both indices is synchronized and GigaBASE is ready to start new transaction.
Bitmap of modified object index pages is used to minimize time of committing transaction. Not the whole object index, but only its modified pages should be copied. After committing of transaction bitmap is cleared.
When transaction is explicitly aborted by dbDatabase::rollback
method, shadow object index is copied back to the current index, eliminating
all changes done by aborted transaction. After the end of copying,
both indices are identical again and database state corresponds to the moment
before the start of current transaction.
Allocation of object handles is done by free handles list. Header of the list is also shadowed and two instances of list headers are stored in database header. Switch between them is done in the same way as switch of object indices. When there are no more free elements in the list, GigaBASE allocates handles from the unused part of new index. When there is no more space in the index, it is reallocated. Object index is the only entity in database whose is not cloned on modification. Instead of this two copies of object index are always used.
There are some predefined OID values in GigaBASE. OID 0 is reserved as invalid object identifier. OID 1 is used as identifier of metatable object - table containing descriptors of all other tables in database. This table is automatically constructed while database initialization and descriptors of all registered application classes are stored in this metatable. OID starting from 2 are reserved for bitmap pages. Number of bitmap pages depends on database maximum virtual space. For one terabyte virtual space, 8 Kb page size and 64 byte allocation quantum, 32K bitmap pages are required. So 32K handles are reserved in object index for bitmap. Bitmap pages are allocated on demand, when database size is extended. So OID of first users object will be 0x8002.
dirty flag is set in database header), then GigaBASE performs
database recovery. Recovery is very similar to rollback of transaction.
Indicator of current index in database object header is used to
determine index corresponding to consistent database state and object handles
from this index are copied to another object index, eliminating
all changes done by uncommitted transaction. As far as the only action
performed by recovery procedure is copying of objects index (really only
handles having different values in current and shadow indices are copied to
reduce number of modified pages) and size of object index is small,
recovery can be done very fast.
Fast recovery procedure reduces "out-of-service" time of application.
There is one hack which used in GigaBASE to increase database performance.
All records in the table are linked in L2-list, allowing efficient traversal
through the list and insertion/removing of records.
Header of the list is stored in table object (which is record of
Metatable table). L2-list pointers are
stored at the beginning of the object together with object size.
New records are always appended in GigaBASE to the end of the list.
To provide consistent inclusion in database list we should clone last record
in the table and table object itself. But record size can be big enough, so
cloning of last record for each inserted record can cause significant space
and time overhead.
To eliminate this overhead GigaBASE do not clone last record allowing temporary inconsistency of the list. In which state will be list if system fault happens before commit of the transaction ? Consistent version of table object will point to the record which was last record in previous consistent state of database. But as far as this record was not cloned, it can contain pointer to next record, which doesn't exist in this consistent database state. To fix this inconsistency, GigaBASE checks all tables in database during recovery procedure and if last record in the table contains not null next reference, it is changed to null to restore consistency.
If database file was corrupted on disk, the only way of database recovery
is to use backup file (certainly if you do not forget to make it).
Backup file can be made by interactive SQL utility using backup
command or from application by dbDatabase::backup() method.
It creates snapshot of database in specified file (it can be name of a
device, tape for example). As far as database file is always in consistent
state, the only think needed to perform recovery from the backup file
is to replace original database file with backup file. If backup was stored
at tape or some other external device, it should be first extract to the
disk.
If some of application starts transaction, locks database and then crashes, then database is left in locked state and no other application can access it. To restore from this situation you should stop all applications working with database. First application opening the database after this will initialize database monitor and perform recovery after crash.
The pages of B-tree contain key values and references to the objects.
For scalar values, key values and object references are stored in two arrays
grown towards each other. Array of key values grows from the beginning of
the page to the page end. Object reference corresponding to the key value
in i-th position is stored in the position
*((oid_t*)(page + page_size) - i - 1). For string keys,
leaf page contains array of elements with object reference, string size and
string body offset within page. Strings bodies are allocated starting from the
end of the page.
To keep minimal tree height, B-tree makes a restriction for minimal number of nodes at the page. All internal pages except root, should have not less than half of the pages used (this criteria can be changed). As far as length of string keys stored in leave pages are different, it is not possible to set limitation on the number of nodes for pages with string keys. Instead of this limitation for size used at leaf page is specified. If more than half of the page is free, then reconstruction of the tree is needed.
All operations with B-tree (insert/find/remove) have log(N) complexity, where N is number if nodes in the tree (size of the table). Elements are stored at B-Tree pages in increasing order, so it is possible to use binary search to locate element at the page.
When new item is inserted in the tree, adding new element to the page can cause page overflow. In this case, new page is created and half of the nodes from overflown page are moved to the newly created page. As far as creating of new page cause insertion of new element in the parent page, propagation of inserts can continue to the root of the tree. If root page is splitted, then new root page is created and tree height is increased.
When item is removed from the page, page underflow can happen - more than half of the page is not used. In this case neighbor page is investigated and either merge of neighbor page with underflown page takes place, either reallocation of nodes between underflown page and neighbor page, restoring tree invariants, is performed. As well as with insert operations, propagation of removes can reach the root page and when the single element is left at the root page, this root page is deallocated and tree height is decreased.
At the end of B-tree description we want to illustrate efficiency of
B-tree search operations. Lets say we have 150 million records in the
table with string primary key having average length 8 bytes. Then
average number of elements at leave pages is
page_size / (string_key_element + average_string_length) * 3 / 4 = 8192
/ 16 * 3 / 4 = 384.
So there are about 150000000 / 384 = 390625 leave pages.
Average number of items at the internal page is
page_size / 8 * 3 / 4 = 768
and 512 for the root page. Because
512*768 = 393216 > 390625, then height of the B-tree will be 3.
As far as root has very goods chances to be always present in page pool,
index search of record within 150 millions object will require only two
page reads.
The following rules in BNF-like notation specifies grammar of SUBSQL directives:
directive ::=
select (*) from table-name select-condition ;
| insert into table-name values values-list ;
| create index on table-name.field-name ;
| create table table-name (field-descriptor {, field-descriptor}) ;
| alter table table-name (field-descriptor {, field-descriptor}) ;
| update table-name set field-name = expression {, field-name = expression} where condition ;
| drop index table-name.field-name ;
| drop table table-name
| open (@)database-file-name ;
| delete from table-name
| backup backup-file-name
| start server server-URL number-of-threads
| stop server server-URL
| start http server server-URL
| stop http server server-URL
| restore backup-file-name database-file-name
| export xml-file-name
| import xml-file-name
| commit
| rollback
| autocommit (on | off)
| show
| exit
| help
table-name ::= identifier
values-list ::= tuple { , tuple }
tuple ::= ( value { , value } )
value ::= number | string | true | false
| tuple
index ::= index | hash
field-descriptor ::= field-name field-type (inverse field-name)
field-name ::= identifier { . identifier }
database-file-name ::= 'file-name'
| '@file-name' |
backup-file-name ::= 'file-name'
xml-file-name ::= 'file-name'
server-URL ::= 'HOST:PORT'
SUBSQL automatically commits read-only transaction after each
select statement in order to release shared database lock as soon as possible.
But all database modification operations should be explicitly committed
by commit statement or undone by rollback
statement. Directives open and exit first closes
opened database (if it was opened) and so implicitly commits last transaction.
If database file name is prepended by @ symbol, then
file with such name (without @) is treated as descriptor
of multifile, containing specification of multifile segments.
Select statement always print all record fields. GigaBASE doesn't support
tuples, and result of the selection is always set of objects (records).
Format of select statement output is similar with one accepted by insert
statement (with exception of reference fields). So it is possible to
export/import database table without references by means of
select/insert directives of SUBSQL.
Select statement prints references in format
"#hexadecimal-number". But it is not possible to use this format
in insert statement. As far as object references are represented
in GigaBASE by internal object identifiers, reference field can not be set in
insert statement (when objects are inserted in database they will
be assigned new OID, so there is not sense in specifying reference field
in insert statement). To ensure database reference consistency,
GigaBASE just ignores reference fields when new records are inserted in the table
with references. You should specify value 0 at the place of reference fields.
If you omit '*' symbol in select statement, GigaBASE will output object
identifiers of each selected record.
It is necessary to provide values for all records fields in insert
statement, no default values are not supported. Components of structures and
arrays should be enclosed in parentheses.
It is not possible to create or drop indices and tables while other
applications are working with database. Such operations change
database scheme and after such modification other applications state
will become incorrect. But delete operation
doesn't change database scheme, so it can be performed as normal transaction,
when database is concurrently used by several applications.
If SubSQL hangs trying to execute some statement, then some other application
holds the lock on database, preventing SUBSQL from accessing it.
SubSQL can be used to start CLI and HTTP servers. CLI server accept connections of clients using CLI protocol to access database. Each client is server by separate thread. SubSQL maintains pool of thread, taking thread from the pool when client is attached and place thread back in the pool when client is disconnected.
HTTP server is used to provide view-only access to the database from Web browsers.
Unlike CLI servers, only one instance of HTTP server can be started.
To view main database page, just type in your Web browser URL which you have
specified in start http server command.
Database can be exported as XML file using SubSQL export command.
This command cause dump of the whole database in XML format to the specified file according
to the following rules:
id attribute which specified OID of the object.
"". All characters greater than 'z' or less then ' ' or
equal to '%' or '"' are printed as two hex digits prepended by '%': space character = "%20".
<ref id=OID/> element, where OID is object identifier of
referenced object.
<array-element> elements
<rectangle> element with two <vertex>
subelements containing list of coordinate attributes.
Produced XML file can be used to export data to other programs or for flexible restoring of database.
To be able to import data from XML file you should have all table created. In C++ application it is enough
to open and close database, then all described tables will be created. SubSQL import
command will import data from XML file, mapping OIDs of records specified in XML file to OIDs of created records.
Such export/import sequence can be used to convert database to the new format. For example, if you
increase value of dbDatabaseOffsetBits constant to extend database file size limit, format of database will be incompatible with one created with previous value of dbDatabaseOffsetBits. In this
case export/import can be used to prevent loosing of all data stored in the database.
` ets say that you have database "YourOldDatabase.dbs" created by your application YourOldApplication and you want to convert it to new format so it can be used by new version of your application: YourNewApplication. First Of all you need to initialize new database. To do it you should run YourNewApplication and open/close database "YourNewDatabase.dbs" in it. Then prepare two SubSQL command files:
export.sql:
open 'YourOldDatabase.dbs'; export '-' exit
import.sql:
open 'YourNewDatabase.dbs'; import '-' exit
subsql export.xml | subsql import.xml
To be able to print dbDateTime in SubSQL in readable form, you should
define SUBSQL_DATE_FORMAT environment variablee (for example '%c'). To get more information about
date time formats see documentation of strftime function. If SUBSQL_DATE_FORMAT is not defined,
dbDateTime structure will be printed as normal structure
with integer component. SubSQL doesn't currently allow convertion of date from string during insert or update operations.
Parameters of the database in SubSQL can be set using the following environment variables:
Interaction with Web server is based on three-tier model:
Web Server -> CGI stub -> GigaBASE application
CGI call local socket connection
Using GigaBASE built-in HTTP server provides maximum performance, because in
this no communication and process creation overhead takes place.
In both cases the same API for receiving and unpacking requests
and constructing responses is used. So the same application
can be used for interaction with external Web server as well as
stand-alone HTTP server.
GigaBASE application is request-driven program, receiving data from
HTML forms and dynamically generating result HTML page. Classes
WWWapi and WWWconnection provide simple and
convenient interface for getting HTTP requests, constructing HTML page and
sending reply back to WWW browser. Abstract class WWWapi
has two implementations: CGIapi and HTTPapi,
first of which implements protocol of interaction with Web server by means of
CGI mechanism, and the second - protocol of direct serving HTTP requests.
Built-in HTTP server is able to handle two types of requests - transfer HTML file find in place relative to the current working directory in response to GET HTTP request and perform action specified by GET or POST requests with parameters. Built-in HTTP server provides persistent connections - server will not close connection with client immediately after sending response, instead of this connection will be kept during some specified interval of time. Also this built-in server supports concurrent requests processing by several threads of control. But starting of threads should be performed by client application.
Virtual method WWWapi::connect(WWWconnection& con)
accept clients connection (either from CGISTUB program of from WWW browser).
This method returns true if connection is established.
In this case programmer should call
CGIapi::serve(WWWconnection& con) to receive and handle client's
requests. This method return false if and only if handler
of request returns false. Even if request was not correctly
received or could not be handled, true is returned by
serve method. The connection is always closed after return from
serve method. It is possible to start separate thread for
exceution of each server method.
To construct responce to the request special overloaded >>
operators are provided in WWWconnection class. First line of
response should specify type of response body, for example:
Content-type: text/html\r\n\r\nTwo CR-LF character after this line separate HTTP header from the body. Three encoding schemes can be used for constructing response body:
WWWconnection
class performs automatic switching between encodings. Initially TAG
encoding is always used. Then encodings are implicitly changed using the
following rules:
TAG -> HTML
HTML -> TAG
URL -> TAG
It certainly possible to explicitly specify encoding for the next output
operation by means of special << operator, which accepts
one of the following constants: TAG, HTML, URL.
Information about HTML form slots values or request parameters can be obtained
using WWWconnection::get(char const* name, int n = 0) method.
Optional second parameter is used only for getting value of selectors with
multiple selection allows option. If parameter with such name is not found,
NULL is returned. There are some mandatory parameters
which always should be present in all forms handled by GigaBASE:
| Parameter name | Parameter Description |
|---|---|
| socket | address of the server, used for constructing new links |
| page | symbolic name of the page, used for request dispatching |
| stub | name of CGI stub program always defined by API |
This is example of "navigation-only" application - no queries are used in this application at all. All navigation between records (object) is done by means of references. Really this application is more suitable for object oriented databases, but I include it in GigaBASE
testperf program by number of iterations.
| System | Insertion*) | Index search | Sequential search | Sequential search with sorting |
|---|---|---|---|---|
| Pentium-II 250, Windows NT 4.0 | 0.207 | 0.155 | 88 100 | 520 000 |
| Pentium-II 250, Linux | 0.070 | 0.128 | 90 200 | 458 000 |
| Ultra-5, Sparc 270 Mhz, Solaric 2.6 | 0.137 | 0.263 | - | - |
*) doesn't include commit time
USE_EXTERNAL_HTTP_SERVER.
Database can be
accessed from any computer running some WWW browser. To build
bugdb application in Unix you should specify www
target to make utility.
To run this BUGDB with external WWW server you should first customize your
WWW server.
It should be able to access buglogin.htm file and run
CGI script cgistub. Also user, under which CGI scripts will
be executed, should have enough permissions to establish connection with
GigaBASE application (by sockets). It is better to run GigaBASE application and
GigaBASE CGI scripts under the same user. For example, I have changed the
following variables in Apache configuration file:
httpd.conf:
User konst
Group users
access.conf:
<Directory /usr/konst/gigabase>
Options All
AllowOverride All
allow from all
</Directory>
DocumentRoot /usr/konst/gigabase
srm.conf:
ScriptAlias /cgi-bin/ /usr/konst/gigabase/
It is also possible not to change configuration of WWW server, but place
cgistub and bugdb programs in standard CGI
script directory and change in the file buglogin.htm path to
the cgistub program.
After preparing configuration files you should start WWW server.No configuration is needed when you are using built-in HTTP server. Just make sure that user has enough permission to access port number 80 (default port for HTTP server). If some HTTP server is already started at your computer, you should either stop it or specify another port for built-in HTTP server. In last case you also need to specify the same port in the settings of WWW browser to make it possible to establish connection with right HTTP server. Also do not forget to specify real name of your computer in ACTION field of buglogin.htm file.
After starting bugdb application itself you can visit
buglogin.htm page in WWW browser and start to work with
BUGDB database. When database is initialized, "administrator" user is
created in the database. First time you should login as administrator using
empty password. Than you can create some other users/engineers and
change the password. BUGDB doesn't use secure protocol of passing passwords and
doesn't worry much about restricting access of users to the database.
So if you are going to use BUGDB in real life, you should first
think about protecting database from unauthorized access.
As well as BUGDB, this Web database example can work either through CGI interface with some external HTTP server or use built-in HTTP server (last one is significantly more efficient than interaction through CGI scripts). Look previous section for more information about configuration of HTTP server. Do not forget to specify real name of your computer in ACTION field of clilogin.htm file.
After starting clidb application itself you can visit
clilogin.htm page in WWW browser and start to work with
CLIDB database. When database is initialized, "administrator" user is
created in the database. This database allows to specify IP address for the manager from
which this manager can login to the system. So user authentication is based on
the name and host computer IP address. Value '*' allows user to login from any host.
If you specify wrong IP address for the administrator and so are not able to login to
the database, you can invoke clidb as
clidb login_from_any_host
CLIDB also can be considered as example of multithreaded Web database server.
There is special class QueueManager which can be used for maintaining pool of threads
and distributing user HTTP requests between these threads. If you recompile CLIDB
application with USE_QUEUE_MANAGER option set,
then CLIDB will spawn 8 threads for handling HTTP requests. In this case persistent
connection with client's WWW browsers are established (so there is no
extra overhead for establishing connecting for each HTTP request).
REGISTER macro
(it should be done in some implementation module). If you are going
to redefine default GigaBASE error handler (for example, if you want to use
message window for reporting instead of stderr), you should
define your own database class and derive it from dbDatabase.
You should create instance of database class and make it accessible to
all application modules.
Before you can do something with database, you should open it.
By checking of dbDatabase::open() return code you can
understand if database was successfully opened. Errors during database
opening doesn't cause application termination (but they are reported)
even with default error handler.
Once you are certain that database is normally opened, you can start
to work with database. If your application is multithreaded and several threads
will work with the same database, you should attach each thread to the
database by dbDatabase::attach method. Before thread termination,
it should detached itself from database by invoking
dbDatabase::detach() method. If your application uses navigation
through database objects by references, you need some kind of root objects
which can be located without any references. Best candidate for the root
objects is the first record of the table. GigaBASE guarantee that new
records are always inserted at the end of the table. So first table record
is also the oldest record in the table.
To access database data you should create a number of dbQuery
and dbCursor objects. If several threads are working with
database, each thread should have its own instances of query and
cursor objects. Usually it is enough to have one cursor for each table
(or two if your application also can update table records). But in case
of nested queries, using of several cursors may be needed.
Query objects are usually created for each type of queries. Query objects are
used also for caching compiled queries, so it will be good idea to
extend live area of query variables (may be make them static).
There are four main operations with database: insert, select, update, remove.
First is done without using cursors, by means of global overloaded
template function insert. Selection, updating and deleting of
records is performed using cursors. To be able to modify table you should
use cursor for update. Cursor in GigaBASE is typed and contains instance
of object of table class. Overloaded -> operator
of the cursor can be used to access components of current record
and also to update these components. Method update
copies data from cursor's object to the current table record.
Cursor's method remove will remove current cursor record,
method removeAllSelected will remove all selected records and
method removeAll will remove all records in the table.
Each transaction should be either committed by
dbDatabase::commit() or aborted by
dbDatabase::rollback() method. Transaction is started
automatically when first select, insert or remove operation is executed.
Before exiting from your application do not forget to close database.
Also remember, that method dbDatabase::close() will automatically
commit last transaction, so if it is not what you want, then explicitly perform
dbDatabase::rollback before exit.
So template of GigaBASE application can look something like this:
//
// Header file
//
#include "gigabase.h"
extern dbDatabase db; // create database object
class MyTable {
char const* someField;
...
public:
TYPE_DESCRIPTOR((FIELD(someField)));
};
//
// Implementation
//
REGISTER(MyTable);
int main()
{
if (db.open("mydatabase.dbs")) {
dbCursor<MyTable> cursor;
dbQuery q;
char value[bufSize];
q = "someField=",value;
gets(value);
if (cursor.select(q) > 0) {
do {
printf("%s\n", cursor->someField);
} while (cursor.next());
}
db.close();
return EXIT_SUCCESS;
} else {
return EXIT_FAILURE;
}
}
To compile GigaBASE application you need to include header file
"gigabase.h".
This header file includes other GigaBASE header files,
so make sure that GigaBASE directory is in compiler include directories list.
To link GigaBASE application you need GigaBASE library
("gigabase.lib"
for Windows or "libgigabase.a" for Unix). You can either
specify full path to this library or place it in some default
library catalog (for example /usr/lib for Unix).
GigaBASE will be able to handle non-ANSI character code sets if
library is compiled with USE_LOCALE_SETTINGS name defined.
In this case strcoll() functions will be used for
string comparison instead of strcmp() (strcmp()
still will be used for comparison of strings for equality).
To make this string comparisons and conversions (performed by
toupper() and tolower() functions) work perfectly,
you should set correct value of locale by setlocale()
function. GigaBASE also supports Unicode strings. To switch on Unicode support
set UNICODE_SUPPORT=1 in makfile.mvc (at Windows) or uncomment definition of UNICODE in config.h
To build GigaBASE library just type make in GigaBASE directory.
There is no autoconfiguration utility included in GigaBASE distribution.
Most system dependent parts of code are compiled using
conditional compilation. There are two makefiles in GigaBASE distribution.
One for MS Windows with MS Visual C++ (makefile.mvc)
and another one for generic Unix with gcc compiler(makefile).
If you want to use Posix threads or some other compiler, you
should edit this makefile.
There is also make.bat, which just spawns
nmake -f makefile.mvc command.
Target install target in
Unix makefile will copy GigaBASE header files,
GigaBASE library and subsql utility
to directories specified by INCSPATH, LIBSPATH and
BINSPATH makefile variables correspondingly.
Default values of this variables are the following:
INCSPATH=/usr/include
LIBSPATH=/usr/lib
BINSPATH=/usr/bin
Once your application starts to work, you will be busy with
support and extension of you application. GigaBASE is able to perform
automatic scheme evaluation for such cases as adding new field to the table and
changing type of the field. Programmer can also add new indices or remove
rarely used indices. Database trace can be switched on (by recompilation
GigaBASE library with -DDEBUG=DEBUG_TRACE compiler options) to
perform analysis of database functionality and efficiency of using indices.
SUBSQL utility can be used for database browsing and inspection, performing online backups, database recovery, importing data to and exporting data from database. GigaBASE will perform automatic recovery after system or application crash, you should not worry about it. The only thing you can have to do manually is stopping all database application if one of them is crashed leaving database blocked.
GigaBASE allows to declare queries as static variables.
But in this case destructors of these queries can be called after destructors of the static
objects from the GigaBASE library itself (because order of invoking destructors for different modules is
not specified. To solve this problem, in GigaBASE some global objects are replaced with references.
As a result destructors are never called for these objects. But it leads to memory leaks
in application. These memory leaks can not actually cause some problems (like memory exhaustion),
but it case warning in some memory-checking utilities. And many C++ programmers like to use such utilities
to check if all emory is correctly deallocated. To avoid these warning for GigaBASE objects you can
register with atexit system function static method dbDatabase::clenup,
which will delete all objects allocated by GigaBASE.
First of all, you can reduce value of dbDefaultInitIndexSize in
database.h file. Default value of this parameter is 1024*1024. As far as size of
each entry in of the object index is 4 bytes long and there are two copies of index (primary
and shadow), it results 8 megabyte file. If typical number of object in your application is
small then one million, you can choose the smaller value of this parameter. When number
of objects exceeds size of object index, it will be reallocated. So reducing value of this
doesn't set any limitations on size of database and number of objects - it just increase number
of possible index reallocation. Index reallocation is requires copying of old instance of the
index to the new location, so it is time consuming operation (but when the size of object index
is smaller than million, it will take fractions of second).
You can even further reduce size of the database by reducing dbDatabaseOffsetBits
parameter in class.h file. But this parameter limits the maximal size of the
database file and benefit in space is not so significant. So I do not recommend you to change
this parameter. But if you want to support databases large than 4 gigabytes, you need to
increase value of this parameters (default value is 32). When value of this parameters is
larger than 32, each entry in the object index is 8 bytes long instead of 4 bytes. So size
of the object index (and so initial size of the database) is doubled.
REGISTER_UNASSIGNED macro.
dbCursor constructor.
insert is now member of dbDatabase class (if
C++ compiler doesn't support member templates, you should define
NO_MEMBER_TEMPLATES macro and path pointer to database as first parameter).
For examle:
class MyClass {
...
};
REGISTER_UNASSIGNED(MyClass);
dbDatabase* db;
dbCursor<MyClass> cursor(db);
MyClass rec;
if (cursor.select(q) == 0) {
db->insert(rec);
}
I will provide e-mail support and help you with development of GigaBASE applications.
Look for new version at my homepage | E-Mail me about bugs and problems