ibis Namespace Reference

The current implementation of FastBit is code named IBIS; most data structures and functions are in the name space ibis. More...

Namespaces
	math
	A namespace for arithmetic expressions.
	util
	Organize the miscellaneous functions under the name util.

Classes
class	ambit
	The multi-level range based (cumulative) index. More...
class	array_t
	Template array_t is a replacement of std::vector. More...
class	bad_alloc
	A specialization of std::bad_alloc. More...
class	bak
	Maps each value to a lower prevision (decimal) values and use the the low precision value as center of the bin. More...
class	bak2
	A variation on ibis::bak, it splits each bin of ibis::bak in two, one for entries less than the mapped value and one for the entries that greater and equal to the mapped value. More...
class	bin
	The equality encoded bitmap index with binning. More...
class	bitvector
	A data structure to represent a sequence of bits. More...
class	bitvector64
	A data structure to represent a sequence of bits. More...
class	blob
	A class to provide a minimal support for byte arrays. More...
class	bord
	Class ibis::bord stores all its data in memory. More...
class	bundle
	The public interface of bundles. More...
class	bundle0
	The null bundle. It contains only a list of RIDs. More...
class	bundle1
	The bundle with only one component. More...
class	bundles
	The bundle with multiple components. More...
class	bylt
	The precise version of the two-level range-equality index. More...
class	category
	A specialized low-cardinality text field. More...
class	colBlobs
	A class to store raw binary values. More...
class	colBytes
	A class to store signed 8-bit integer values. More...
class	colDoubles
	A class to store double precision floating-point values. More...
class	colFloats
	A class to store single precision float-point values. More...
class	colInts
	A class to store integer values. More...
class	colLongs
	A class to store signed 64-bit integer values. More...
class	colShorts
	A class to store short integer values. More...
class	colStrings
	A class to store string values. More...
class	colUBytes
	A class to store unsigned 64-bit integer values. More...
class	colUInts
	A class to store unsigned integer values. More...
class	colULongs
	A class to store unsigned 64-bit integer values. More...
class	column
	The class to represent a column of a data partition. More...
class	colUShorts
	A class to store unsigned short integer values. More...
class	colValues
	A pure virtual base class for storing selected values in memory. More...
class	compRange
	The class compRange stores computed ranges. More...
class	countQuery
	A simple count query. More...
class	deprecatedJoin
	A join is defined by two names and a numerical expression. More...
class	dictionary
	Provide a dual-directional mapping between strings and integers. More...
class	direkte
	A version of precise index that directly uses the integer values. More...
class	discretePoisson
	Discrete random number with Poisson distribution exp(-x/lambda). More...
class	discretePoisson1
	Specialized version of the Poisson distribution exp(-x). More...
class	discreteZipf
	Discrete Zipf distribution. More...
class	discreteZipf1
	A specialized case of the Zipf distribution f(x) = 1/(1+x). More...
class	discreteZipf2
	A specialized version of the Zipf distribution f(x) = 1/(1+x)^2. More...
class	egale
	The multicomponent equality code on bins. More...
class	entre
	The multicomponent interval code on bins. More...
class	fade
	The multicomponent range-encoded index. More...
class	fileManager
	This fileManager is intended to allow different objects to share the same open file. More...
class	filter
	A simple filtering query. More...
class	fromClause
	A class to represent the from clause. More...
class	fromLexer
	Defines a new class with the desired lex function for C++ output of bison. More...
class	fuge
	A two-level index. More...
class	fuzz
	The precise version of two-level interval-equality index. More...
class	horometer
	Horometer – a primitive timing instrument. More...
class	hyperslab
	Class ibis::hyperslab for recording a HDF5 style hyperslab. More...
class	index
	The base index class. More...
class	jNatural
	In-memory Natual Join. More...
class	join
	An abstract join interface. More...
class	joinIN
	FastBit In-memory Natual Join. More...
class	jRange
	In-memory Range Join. More...
class	keywords
	Class ibis::keywords defines a boolean term-document matrix. More...
class	liga
	A specialization of class mensa. More...
class	mensa
	Class ibis::mensa contains multiple (horizontal) data partitions (ibis::part) to form a logical data table. More...
class	MersenneTwister
	Mersenne Twister. More...
class	mesa
	This class implements the interval encoding of Chan and Ioannidis. More...
class	meshQuery
	The class adds more functionality to ibis::query to handle data from regular meshes. More...
class	moins
	The multicomponent range code on bins. More...
class	nameList
	A data structure to store a small set of names. More...
class	pack
	A two-level index. More...
class	pale
	A two-level index. More...
class	part
	The class ibis::part represents a partition of a relational table. More...
class	qAllWords
	The class qAllWords encapsulates a search for many keywords. More...
class	qAnyAny
	A user specifies this type of query expression with the following syntax,. More...
class	qAnyString
	The column contains one of the values in a list. More...
class	qContinuousRange
	Simple range condition. More...
class	qDiscreteRange
	A discrete range expression. More...
class	qExists
	This data structure holds a single name. More...
class	qExpr
	The top level query expression object. More...
class	qIntHod
	This query expression has similar meaning as ibis::qDiscreteRange, however, it stores the values as signed 64-bit integers. More...
class	qKeyword
	The class qKeyword encapsulates a search for a single keyword in a text field. More...
class	qLike
	Representing the operator 'LIKE'. More...
class	qRange
	A class to represent simple range conditions. More...
class	qString
	The class qString encapsulates information for comparing string values. More...
class	quaere
	An abstract query interface. More...
class	query
	A data structure for representing user queries. More...
class	qUIntHod
	This query expression has similar meaning as ibis::qDiscreteRange, however, it stores the values as unsigned 64-bit integers. More...
class	randomGaussian
	Continuous Gaussian distribution. More...
class	randomPoisson
	Continuous Poisson distribution. More...
class	randomZipf
	Continuous Zipf distribution. More...
class	range
	The range encoded bitmap index based. More...
class	relic
	The basic bitmap index. More...
class	resource
	A container for name-value pairs. More...
class	ridHandler
	A class for handling file IO for ibis::rid_t. More...
class	roster
	A roster is a list of values in ascending order plus their original positions. More...
class	sapid
	The multicomponent equality encoded index. More...
class	sbiad
	The multicomponent interval encoded index. More...
class	selectClause
	A class to represent the select clause. More...
class	selectLexer
	Defines a new class with the desired lex function for C++ output of bison. More...
class	skive
	The binary encoded index with recoding of keyvalues. More...
class	slice
	The bit-sliced index. More...
class	tabele
	A trivial class for a table with exactly one row, one column and one integer value. More...
class	table
	The abstract table class. More...
class	tableList
	A list of tables. More...
class	tablex
	The class for expandable tables. More...
class	tabula
	A trivial class for a table with no columns. More...
class	tafel
	An expandable table. More...
class	text
	A data structure for storing null-terminated text. More...
class	uniformRandomNumber
	A functor to generate uniform random number in the range [0, 1). More...
class	whereClause
	A representation of the where clause. More...
class	whereLexer
	Defines a new class with the desired lex function for C++ output of bison. More...
class	zona
	The precise version of the two-level equality-equality index. More...
class	zone
	A two-level index. More...

Typedefs
typedef std::vector< colValues * >	colList
typedef std::vector< joinspec * >	joinlist
typedef std::map< const char *, part *, lessi >	partAssoc
	An associative array for data partitions. More...
typedef array_t< rid_t >	RIDSet

Enumerations
enum	TYPE_T {   UNKNOWN_TYPE =0, OID, BYTE, UBYTE,   SHORT, USHORT, INT, UINT,   LONG, ULONG, FLOAT, DOUBLE,   BIT, CATEGORY, TEXT, BLOB,   UDT }
	Supported data types. More...

Functions
ibis::part *	findDataset (const char *)
	Find a dataset with the given name. More...
ibis::part *	findDataset (const char *, const ibis::partList &)
	Find a dataset with the given name among the given list. More...
ibis::resource &	gParameters ()
	List of in-memory data. More...
void	init (const char *rcfile=0, const char *mesgfile=0)
	Initializes the memory manager of FastBit. More...

Variables
partList	datasets
	!< Select clause. More...
const char **	TYPESTRING = _ibis_TYPESTRING_local
	Human readable version of the enumeration types.

Detailed Description

The current implementation of FastBit is code named IBIS; most data structures and functions are in the name space ibis.

The name IBIS could be considered as a short-hand for an Implementation of Bitmap Indexing System or Ibis Bitmap Indexing System.

Typedef Documentation

typedef std::map< const char*, part*, lessi > ibis::partAssoc

An associative array for data partitions.

Only used internally for sorting data partitions.

Enumeration Type Documentation

enum ibis::TYPE_T

Supported data types.

Enumerator
UNKNOWN_TYPE	Unknown type, a place holder. Can not process data of this type!
OID	A special eight-byte ID type for internal use.
UBYTE	!< One-byte signed integers, internally char.
SHORT	!< One-byte unsigned integers, internally unsigned char.
USHORT	!< Two-byte signed integers, internally int16_t.
INT	!< Two-byte unsigned integers, internally uint16_t.
UINT	!< Four-byte signed integers, internally int32_t.
LONG	!< Four-byte unsigned integers, internally uint32_t.
ULONG	!< Eight-byte signed integers, internally int64_t.
FLOAT	!< Eight-byte unsigned integers, internally uint64_t.
DOUBLE	!< Four-byte IEEE floating-point numbers, internally float.
BIT	!< Eight-byte IEEE floating-point numbers, internally double.
CATEGORY	!< One bit per record, represented by a bit vector. Low cardinality null-terminated strings. Strings are internally stored with the null terminators. Each string value is intended to be treated as a single atomic item.
TEXT	Arbitrary null-terminated strings. Strings are internally stored with the null terminators. Each string could be further broken into tokens for a full-text index known as keyword index. Could search for presence of some keywords through expression "contains" such as "contains(textcolumn, 'Berkeley', 'California')".
BLOB	Byte array. Also known as Binary Large Objects (blob) or opaque objects. A column of this type requires special handling for input and output. It can not be used as a part of any searching criteria.
UDT	User-defined type. FastBit does not know much about it.

Function Documentation

ibis::part * ibis::findDataset

(

const char *

pn

)

Find a dataset with the given name.

If the named data partition is found, a point to the data partition is returned, otherwise, a nil pointer is returned. If the name is nil, a nil pointer will be returned.

References datasets, and ibis::util::envLock.

Referenced by ibis::quaere::create(), and findDataset().

ibis::part * ibis::findDataset	(	const char *	pn,
		const ibis::partList &	prts
	)

Find a dataset with the given name among the given list.

It performs a linear search.

References datasets, and findDataset().

ibis::resource & ibis::gParameters

(

)

List of in-memory data.

This function returns a reference to a set of global parameters.

The reference to the global configuration parameters.

These parameters can affect the execution of the FastBit, such as the maximum number of byte the memory manager may use.

Note

This function returns an empty object when called the first time. The caller is expected to either use ibis::init or ibis::resource::read to input a user-specified configuration files.

Some of the parameters are consulted once. For example, the maximum bytes used by the memory manager is only used once at the construction of the memory manager; modifying this parameter after the initialization of the memory manager will not affect the memory manager any more. Therefore, we recommend the caller to perform all necessary operations with ibis::gParameters before performing other operations.

Referenced by ibis::mensa::addPartition(), ibis::part::append1(), ibis::part::append2(), ibis::bord::backup(), ibis::bin::bin(), ibis::query::clear(), ibis::bin::construct(), ibis::index::create(), fastbit_init(), ibis::fileManager::fileManager(), ibis::util::getLogFile(), ibis::text::IDColumnForKeywordIndex(), init(), ibis::query::keepQueryRecords(), ibis::column::loadIndex(), ibis::part::loadIndexes(), ibis::mensa::mensa(), ibis::index::optionalUnpack(), ibis::bin::parseNbins(), ibis::bin::parsePrec(), ibis::bin::parseScale(), ibis::part::part(), ibis::query::query(), ibis::part::queryTest(), ibis::part::quickTest(), ibis::query::removeQueryRecords(), ibis::part::selfTest(), ibis::relic::speedTest(), ibis::bin::speedTest(), and ibis::tafel::writeMetaData().

void ibis::init ( const char *  rcfile = 0, const char *  mesgfile = 0  )

inline

Initializes the memory manager of FastBit.

It reads the RC file (rcfile) first before initializes the memory manager. If the caller wishes to read multiple RC files or add parameters to ibis::gParameters, these operations need to take place before calling this function or any function that creates, initializes or uses ibis::array_t, ibis::bitvector, ibis::part or ibis::table. If the user neglects to call ibis::init, the memory manager will be initialized when the first time it is needed.

Parameters

rcfile	A file containing name-value pairs that specifies parameters for controlling the behavior of ibis.
mesgfile	Name of the file to contain messages printed by FastBit functions.

If an RC file is not specified or the file name is null, this function will attempt to read one of the following file (in the given order).

1. a file named in environment variable IBISRC,
2. a file named ibis.rc in the current working directory,
3. a file named .ibisrc in the user's home directory.

In an RC file, one parameter occupies a line and the equal sign "=" is required to delimit the name and the value, for example,

*    dataDir = /data/dns
*    cacheDir = /tmp/ibiscache
* 

The minimal recommended parameters of an RC file are

• dataDir, which can also be written as dataDir1 or indexDir. It tells ibis where to find the data to be queried. Multiple data directories may be specified by adding prefix to the parameter name, for example, dns.dataDir and random.dataDir.
• cacheDir, which can also be written as cacheDirectory. This directory is used by ibis to write internal data for recovery and other purposes.

The message file (also called the log file) name may also be specified in the RC file under the key logfile, e.g.,

*    logfile = /tmp/ibis.log
* 

One may call ibis::util::closeLogFile to close the log file, but this is not mandatory. The runtime system will close all open files upon the termination of the user program.

References ibis::util::closeLogFile(), datasets, ibis::util::envLock, ibis::util::gatherParts(), gParameters(), ibis::fileManager::instance(), ibis::resource::read(), and ibis::util::setLogFileName().

Variable Documentation

ibis::partList ibis::datasets

!< Select clause.

The global variable holding all the data partitions.

A global list of data partitions.

Referenced by ibis::filter::count(), ibis::quaere::create(), ibis::util::emptyCache(), findDataset(), init(), ibis::filter::roughCount(), ibis::filter::select(), ibis::util::updateDatasets(), and ibis::fileManager::~fileManager().