ibis Namespace Reference

The current implementation of FastBit is code named IBIS and most data structures and functions are in the name space ibis. More...

Classes
class	ambit
	The multi-level range based (cumulative) index. More...
class	bad_alloc
	A specialization of std::bad_alloc. More...
class	bak
	Maps each value to a lower prevision (decimal) values and use the the low precision value as center of the bin. More...
class	bak2
	A variation on ibis::bak, it splits each bin of ibis::bak in two, one for entries less than the mapped value and one for the entries that greater and equal to the mapped value. More...
class	bin
	The equality encoded bitmap index with binning. More...
class	bitvector
	A data structure to represent a sequence of bits. More...
class	bitvector64
	A data structure to represent a sequence of bits. More...
class	bord
	Class ibis::bord stores all its data in memory. More...
class	bundle
	The public interface of bundles. More...
class	bundle0
	The null bundle. It contains only a list of RIDs. More...
class	bundle1
	The bundle with only one component. More...
class	bundles
	The bundle with multiple components. More...
class	bylt
	The two-level range-equality code. More...
class	category
	A specialized low-cardinality text field. More...
class	colDoubles
	A class to store double precision floating-point values. More...
class	colFloats
	A class to store single precision float-point values. More...
class	colInts
	A class to store integer values. More...
class	colLongs
	A class to store integer values. More...
class	colUInts
	A class to store unsigned integer values. More...
class	colULongs
	A class to store unsigned integer values. More...
class	column
	The class to represent a column of a data partition. More...
class	colValues
	A pure virtual base class. More...
class	compRange
	The class compRange stores computed ranges. More...
class	dictionary
	Provide a mapping between strings and integers. More...
class	direkte
	Directly use the integer values as bin number to avoid some intemdiate steps. More...
class	discretePoisson
	Discrete random number with Poisson distribution exp(-x/lambda). More...
class	discretePoisson1
	Specialized version of the Poisson distribution exp(-x). More...
class	discreteZipf
	Discrete Zipf distribution: p(k) is proportional to (v+k)^(-a) where a > 1, k >= 0. More...
class	discreteZipf1
	A specialized case of the Zipf distribution f(x) = 1/(1+x). More...
class	discreteZipf2
	A specialized version of the Zipf distribution f(x) = 1/(1+x)^2. More...
class	egale
	The multicomponent equality code on bins. More...
class	entre
	The multicomponent interval code on bins. More...
class	fade
	The multicomponent range-encoded index. More...
class	fileManager
	This fileManager is intended to allow different objects to share the same open file. More...
class	fuge
	A two-level index. More...
class	fuzz
	The two-level interval-equality code. More...
class	horometer
	Horometer -- a primitive timing instrument. More...
class	index
	The base index class. More...
class	join
	An abstract join interface. More...
class	joinIN
	FastBit In-memory Natual Join. More...
class	keywords
	Class ibis::keywords defines a boolean term-document matrix. More...
struct	lessi
	A case-insensitive version of less for comparing names of tables, columns, and resources. More...
class	mensa
	Class ibis::mensa contains multiple (horizontal) data partitions (ibis::part) to form a logical data table. More...
class	MersenneTwister
	Mersenne Twister generates uniform random numbers efficiently. More...
class	mesa
	This class implements the two-side range encoding from Chan and Ioannidis. More...
class	meshQuery
	The class adds more functionality to ibis::query to handle data from meshes. More...
class	moins
	The multicomponent range code on bins. More...
class	nameList
	A data structure to store a small set of names. More...
class	pack
	A two-level index. More...
class	pale
	A two-level index. More...
class	part
	The class ibis::part represents a partition of a relational table. More...
class	qAnyAny
	A user specifies this type of query expression with the following syntax,. More...
class	qContinuousRange
	Simple range condition. More...
class	qDiscreteRange
	A discrete range expression. More...
class	qExpr
	The top level query expression object. More...
class	qMultiString
class	qRange
	A class to represent simple range conditions. More...
class	qString
	The class qString encapsulates information for comparing string values. More...
class	query
	A data structure for representing user queries. More...
class	range
	The range encoded bitmap index based. More...
class	rangeJoin
	A join is defined by two names and a numerical expression. More...
class	relic
	The basic bitmap index. More...
class	resource
	A container for name-value pairs. More...
union	rid_t
	To store configuration parameters. More...
class	ridHandler
	A class for handling file IO for ibis::rid_t. More...
class	roster
	A roster list is a list of indices for ordering the values in the ascending order. More...
class	sapid
	The multicomponent equality encoded index. More...
class	sbiad
	The multicomponent interval encoded index. More...
class	selected
	A data structure to store the select clause of a query. More...
class	slice
	The bit-sliced index (O'Neil). It used the binary encoding. More...
class	tabele
	A trivial class for table with one row and one column. More...
class	table
	The abstract table class. More...
class	tableList
	A list of tables. More...
class	tablex
	The class for expandable tables. More...
class	tabula
	A trivial class for table with no columns. More...
class	tafel
	An expandable table. More...
class	text
	A minimalistic structure for storing arbitrary text fields. More...
class	uniformRandomNumber
	A functor to generate uniform random number in the range [0, 1). More...
class	zona
	The two-level equality-equality code. More...
class	zone
	A two-level index. More...
Typedefs
typedef std::vector< colValues * >	colList
typedef std::map< const char *, part *, lessi >	partList
typedef array_t< rid_t >	RIDSet
Enumerations
enum	TYPE_T {   UNKNOWN_TYPE = 0, OID, BYTE, UBYTE,   SHORT, USHORT, INT, UINT,   LONG, ULONG, FLOAT, DOUBLE,   CATEGORY, TEXT }
	Supported data types. More...
Functions
ibis::resource &	gParameters ()
	This implementation uses a local static variable to hold the globally unique resource object.
void	init (const int verbose=0, const char *rcfile=0, const char *mesgfile=0)
	Initializes internal resources required by ibis.
const ibis::bitvector64 &	outerProduct (const ibis::bitvector &a, const ibis::bitvector &b, ibis::bitvector64 &c)
	This implementation only uses public functions of ibis::bitvector and ibis::bitvector64.
const ibis::bitvector64 &	outerProductUpper (const ibis::bitvector &a, const ibis::bitvector &b, ibis::bitvector64 &c)
	The result contains only the strict upper triangular portion of the full outer product.
qExpr *	parseQuery (const char *str)
	Parse a query string.
Variables
int	accessIndexInWhole = 0
int	gVerbose = 0
	Verbosity level.
const char *	TYPECODE = "?OBAHGIULVRDCS"
	One-character code for the enumeration types.
const char **	TYPESTRING = _ibis_TYPESTRING_local
	Human readable version of the enumeration types.

---

Detailed Description

The current implementation of FastBit is code named IBIS and most data structures and functions are in the name space ibis.

The name IBIS could be considered as a short-hand for an implementation of Bitmap Index Searching system or Ibis Bitmap Index System.

---

Enumeration Type Documentation

enum ibis::TYPE_T

Supported data types.

Enumerator:

UNKNOWN_TYPE	Unknown type, can't really do anything with it.
OID	A special eight-byte ID type for internal use.
BYTE	One-byte signed integers.
UBYTE	One-byte unsigned integers.
SHORT	Two-byte signed integers.
USHORT	Two-byte unsigned integers.
INT	Four-byte signed integers.
UINT	Four-byte unsigned integers.
LONG	Eight-byte signed integers.
ULONG	Eight-byte unsigned integers.
FLOAT	Four-byte IEEE floating-point numbers.
DOUBLE	Eight-byte IEEE floating-point numbers.
CATEGORY	Low cardinality null-terminated strings.
TEXT	Arbitrary null-terminated strings.

---

Function Documentation

ibis::resource & ibis::gParameters

(

)

This implementation uses a local static variable to hold the globally unique resource object.

The reference to the global configuration parameters.

Referenced by ibis::part::append1(), ibis::part::append2(), ibis::bin::bin(), ibis::query::clear(), ibis::index::create(), fastbit_init(), ibis::fileManager::fileManager(), init(), ibis::query::keepQueryRecords(), ibis::part::loadIndex(), ibis::column::loadIndex(), ibis::index::optionalUnpack(), ibis::part::part(), ibis::query::query(), ibis::query::removeQueryRecords(), ibis::part::selfTest(), ibis::relic::speedTest(), ibis::bin::speedTest(), and ibis::tafel::write().

void ibis::init	(	const int	verbose = 0,
		const char *	rcfile = 0,
		const char *	mesgfile = 0
	)			[inline]

Initializes internal resources required by ibis.

It must be called by user code before any other functions.

Parameters:

	verbose	An integer indicating the level of verboseness. A negative number make ibis silent, otherwise the larger it is the more ibis will print out.
	rcfile	A file containing name-value pairs that specifies parameters for controlling the behavior of ibis. If a file name is not specified, it will attempt to read one of the following file (in the given order). a file named in environment variable IBISRC, a file named ibis.rc in the current working directory, a file named .ibisrc in the user's home directory. In an RC file, one parameter occupies a line and the equal sign "=" is required to delimit the name and the value, for example, dataDir = /data/dns cacheDir = /tmp/ibiscache The minimal recommended parameters of an RC file are dataDir, which can also be written as dataDir1 or indexDir. It tells ibis where to find the data to be queried. Multiple data directories may be specified by adding prefix to the parameter name, for example, dns.dataDir and random.dataDir. cacheDir, which can also be written as cacheDirectory. This directory is used by ibis to write internal data for recovery and other purposes.
	mesgfile	Name of the file to contain messages printed by FastBit functions. The message file (also called log file) name may also be specified in the RF file under the key logfile, e.g., logfile = /tmp/ibis.log One should call ibis::util::closeLogFile to close the log file to ensure the content of the message properly preserved.

References gParameters(), gVerbose, and ibis::resource::read().

const ibis::bitvector64 & ibis::outerProduct	(	const ibis::bitvector &	a,
		const ibis::bitvector &	b,
		ibis::bitvector64 &	c
	)

This implementation only uses public functions of ibis::bitvector and ibis::bitvector64.

Compute the outer product of a and b, add the result to c.

This should make it possible to use both WAH and BBC compress bitvector classes. It is not likely that we can gain much performance by directly using member variables of ibit::bitvector because the unit of compressed data from ibit::vector do not fit neatly into ibis::bitvector64::word_t.

Return a const reference of c. If the input does not have the correct size, it will be replaced by the outer product.

References ibis::bitvector64::adjustSize(), ibis::bitvector64::appendFill(), ibis::util::logger::buffer(), ibis::bitvector64::bytes(), ibis::bitvector64::cnt(), ibis::bitvector::cnt(), ibis::bitvector::firstIndexSet(), gVerbose, ibis::bitvector64::setBit(), ibis::bitvector64::size(), ibis::bitvector::size(), ibis::horometer::start(), and ibis::bitvector64::swap().

Referenced by ibis::index::estimate(), ibis::query::processJoin(), ibis::relic::speedTest(), and ibis::bin::speedTest().

const ibis::bitvector64 & ibis::outerProductUpper	(	const ibis::bitvector &	a,
		const ibis::bitvector &	b,
		ibis::bitvector64 &	c
	)

The result contains only the strict upper triangular portion of the full outer product.

Add the strict upper triangular portion of the outer production between a and b to c.

References ibis::bitvector64::adjustSize(), ibis::bitvector64::appendFill(), ibis::util::logger::buffer(), ibis::bitvector64::bytes(), ibis::bitvector64::cnt(), ibis::bitvector::cnt(), ibis::bitvector::firstIndexSet(), gVerbose, ibis::bitvector64::setBit(), ibis::bitvector64::size(), ibis::bitvector::size(), ibis::horometer::start(), and ibis::bitvector64::swap().

ibis::qExpr * ibis::parseQuery

(

const char *

str

)

Parse a query string.

Parse a string into a query expression.

The query expression is stored in a binary tree (ibis::qExpr). There is a mutual exclusion lock to serialize accesses to this function. The implication is that only one query can be parsed at any given time.

References gVerbose.

Referenced by ibis::query::setWhereClause().

---

Variable Documentation

int ibis::gVerbose = 0

Verbosity level.

The larger the value, the more is printed. Default value is 0. A negative value will disable all printing.

Referenced by ibis::part::accessHint(), ibis::index::activate(), ibis::column::actualMinMax(), ibis::index::addBins(), ibis::index::addBits(), ibis::query::addJoinConstraints(), ibis::mensa::addPartition(), ibis::ridHandler::append(), ibis::zone::append(), ibis::pale::append(), ibis::pack::append(), ibis::ambit::append(), ibis::relic::append(), ibis::range::append(), ibis::mesa::append(), ibis::bak2::append(), ibis::bak::append(), ibis::bin::append(), ibis::column::append(), ibis::text::append(), ibis::category::append(), ibis::part::append1(), ibis::part::append2(), ibis::part::appendToBackup(), array_t< T >::array_t(), ibis::bin::bin(), ibis::bin::binning(), ibis::bin::binningT(), ibis::bin::binOrderT(), ibis::meshQuery::bitvectorToCoordinates(), ibis::part::buildIndex(), ibis::mensa::buildIndex(), ibis::bylt::bylt(), ibis::bin::checkBin(), ibis::query::clear(), ibis::mensa::clear(), ibis::fileManager::clear(), ibis::bitvector64::clusteringFactor(), ibis::bitvector::clusteringFactor(), ibis::part::coarsenBins(), ibis::column::column(), ibis::part::commit(), ibis::bord::column::computeMinMax(), ibis::relic::construct(), ibis::query::countEqualPairs(), ibis::part::countHits(), ibis::part::countPages(), ibis::index::create(), ibis::colValues::create(), ibis::bundle::create(), ibis::bitvector::decompress(), ibis::part::digestMeshShape(), ibis::direkte::direkte(), ibis::index::divideCounts(), ibis::part::doBackup(), ibis::part::doCompare(), ibis::part::doCompare0(), ibis::part::doScan(), ibis::fileManager::storage::enlarge(), ibis::bitvector::erase(), ibis::query::estimate(), ibis::zone::estimate(), ibis::pale::estimate(), ibis::pack::estimate(), ibis::fuge::estimate(), ibis::ambit::estimate(), ibis::relic::estimate(), ibis::index::estimate(), ibis::bin::estimate(), ibis::part::estimateRange(), ibis::column::estimateRange(), ibis::query::evaluate(), ibis::joinIN::evaluate(), ibis::zona::evaluate(), ibis::fuzz::evaluate(), ibis::bylt::evaluate(), ibis::bin::evaluate(), ibis::part::evaluateJoin(), ibis::column::evaluateRange(), ibis::bord::column::evaluateRange(), fastbit_build_index(), fastbit_build_indexes(), fastbit_build_query(), fastbit_build_result_set(), fastbit_cleanup(), fastbit_destroy_query(), fastbit_get_verbose_level(), fastbit_purge_index(), fastbit_set_verbose_level(), ibis::fileManager::fileManager(), ibis::text::findString(), ibis::bitvector::flip(), ibis::fileManager::flushDir(), ibis::fileManager::flushFile(), ibis::fuge::fuge(), ibis::fuzz::fuzz(), ibis::fileManager::gainReadAccess(), ibis::part::get1DDistribution(), ibis::part::get2DDistribution(), ibis::part::get2DDistributionD(), ibis::part::get2DDistributionI(), ibis::part::get3DDistribution(), ibis::mensa::getColumnAsBytes(), ibis::mensa::getColumnAsDoubles(), ibis::mensa::getColumnAsFloats(), ibis::mensa::getColumnAsInts(), ibis::mensa::getColumnAsLongs(), ibis::mensa::getColumnAsShorts(), ibis::mensa::getColumnAsStrings(), ibis::mensa::getColumnAsUBytes(), ibis::mensa::getColumnAsUInts(), ibis::mensa::getColumnAsULongs(), ibis::mensa::getColumnAsUShorts(), ibis::part::getCumulativeDistribution(), ibis::part::getDistribution(), ibis::fileManager::getFile(), ibis::fileManager::getFileSegment(), ibis::meshQuery::getHitsAsBlocks(), ibis::part::getJointDistribution(), ibis::query::getMaxNumHits(), ibis::query::getMinNumHits(), ibis::query::getNumHits(), ibis::meshQuery::getPointsOnBoundary(), ibis::query::getQualifiedInts(), ibis::query::getRIDs(), ibis::query::getRIDsInBundle(), ibis::query::getState(), ibis::part::getUndecidable(), init(), ibis::query::isValidToken(), ibis::keywords::keywords(), ibis::query::limit(), ibis::part::loadIndex(), ibis::column::loadIndex(), ibis::column::logError(), ibis::part::makeBackupCopy(), ibis::part::mapValues(), ibis::index::mapValues(), ibis::bak2::mapValues(), ibis::part::matchMetaTags(), ibis::roster::mergeBlock2(), ibis::mesa::mesa(), ibis::part::negativeScan(), ibis::part::vault::open(), ibis::part::barrel::open(), ibis::bitvector::operator &=(), ibis::bitvector::operator-=(), ibis::bitvector::operator^=(), ibis::bitvector::operator|=(), ibis::query::orderby(), ibis::mensa::orderby(), ibis::query::orderPairs(), outerProduct(), outerProductUpper(), ibis::part::packCumulativeDistribution(), ibis::part::packDistribution(), ibis::table::parseNames(), ibis::resource::parseNameValuePairs(), parseQuery(), ibis::part::part(), ibis::qExpr::print(), ibis::zone::print(), ibis::zona::print(), ibis::pale::print(), ibis::pack::print(), ibis::fuzz::print(), ibis::fuge::print(), ibis::bylt::print(), ibis::ambit::print(), ibis::slice::print(), ibis::sbiad::print(), ibis::sapid::print(), ibis::relic::print(), ibis::range::print(), ibis::mesa::print(), ibis::keywords::print(), ibis::fade::print(), ibis::direkte::print(), ibis::bak2::print(), ibis::bak::print(), ibis::moins::print(), ibis::entre::print(), ibis::egale::print(), ibis::bin::print(), ibis::bundles::print(), ibis::bundle1::print(), ibis::bitvector64::print(), ibis::bitvector::print(), ibis::bundles::printAll(), ibis::bundle1::printAll(), ibis::bundle0::printAll(), ibis::query::printRIDs(), array_t< T >::printStatus(), ibis::query::processJoin(), ibis::part::purgeInactive(), array_t< T >::push_back(), ibis::qContinuousRange::qContinuousRange(), ibis::qDiscreteRange::qDiscreteRange(), ibis::query::query(), ibis::ridHandler::read(), ibis::resource::read(), ibis::zone::read(), ibis::zona::read(), ibis::pale::read(), ibis::pack::read(), ibis::fuzz::read(), ibis::fuge::read(), ibis::bylt::read(), ibis::ambit::read(), ibis::slice::read(), ibis::relic::read(), ibis::range::read(), ibis::keywords::read(), ibis::fade::read(), ibis::direkte::read(), ibis::egale::read(), ibis::bin::read(), ibis::fileManager::storage::read(), ibis::bitvector64::read(), ibis::bitvector::read(), array_t< T >::read(), ibis::tafel::readCSV(), ibis::keywords::readLine(), ibis::part::readMeshShape(), ibis::part::readMetaTags(), ibis::part::readRIDs(), ibis::bundle::readRIDs(), ibis::text::readString(), ibis::part::readTDC(), ibis::keywords::readTermDocFile(), ibis::fileManager::releaseAccess(), ibis::relic::relic(), ibis::selected::remove(), ibis::query::removeComplexConditions(), ibis::qExpr::reorder(), ibis::part::reorder(), ibis::part::reorderValues(), array_t< T >::resize(), ibis::part::rollback(), ibis::roster::roster(), ibis::column::saveSelected(), ibis::text::search(), ibis::category::search(), ibis::selected::select(), ibis::mensa::select(), ibis::mensa::select2(), ibis::column::selectBytes(), ibis::column::selectDoubles(), ibis::bord::column::selectDoubles(), ibis::column::selectFloats(), ibis::bord::column::selectFloats(), ibis::column::selectLongs(), ibis::bord::column::selectLongs(), ibis::column::selectShorts(), ibis::column::selectUInts(), ibis::bord::column::selectUInts(), ibis::column::selectULongs(), ibis::column::selectValues(), ibis::part::selfTest(), ibis::qExpr::separateSimple(), ibis::query::sequentialScan(), ibis::bitvector64::setBit(), ibis::bin::setBoundaries(), ibis::query::setRIDs(), ibis::query::setSelectClause(), ibis::bitvector::setSize(), ibis::query::setTable(), ibis::query::setWhereClause(), ibis::fileManager::signalMemoryAvailable(), ibis::qExpr::simplify(), ibis::query::sortEquiJoin(), ibis::query::sortRangeJoin(), ibis::text::startPositions(), ibis::fileManager::storage::storage(), ibis::column::string2int(), ibis::index::sumBins(), ibis::index::sumBits(), ibis::column::truncateData(), ibis::fileManager::tryGetFile(), ibis::bitvector::turnOnRawBit(), ibis::part::unloadIndex(), ibis::column::unloadIndex(), ibis::tafel::write(), ibis::ridHandler::write(), ibis::resource::write(), ibis::zone::write(), ibis::zona::write(), ibis::pale::write(), ibis::pack::write(), ibis::fuzz::write(), ibis::fuge::write(), ibis::bylt::write(), ibis::ambit::write(), ibis::slice::write(), ibis::sbiad::write(), ibis::sapid::write(), ibis::relic::write(), ibis::range::write(), ibis::mesa::write(), ibis::keywords::write(), ibis::fade::write(), ibis::direkte::write(), ibis::bak2::write(), ibis::bak::write(), ibis::bin::write(), array_t< T >::write(), ibis::column::writeData(), ibis::part::writeTDC(), ibis::part::writeValues(), ibis::zona::zona(), and ibis::part::~part().