A simple filtering query. More...

#include <filter.h>

Inheritance diagram for ibis::filter:

Public Member Functions
virtual int64_t	count () const
	Produce the exact number of hits.

	filter (const ibis::whereClause *)
	Constructor. More...

	filter (const ibis::selectClause , const ibis::constPartList , const ibis::whereClause *)
	Constructor. More...

	filter (const ibis::bitvector &, const ibis::part &)
	Constructor. More...

virtual void	roughCount (uint64_t &nmin, uint64_t &nmax) const
	Produce a rough count of the number of hits.

virtual table *	select () const
	Produce a projection of the joint table. More...

virtual table *	select (const char *) const
	Produce a project based on the given select clause. More...

virtual table *	select (const ibis::table::stringArray &colnames) const
	Produce a projection of all known data partitions. More...

virtual	~filter ()
	Destructor.

Static Public Member Functions
static table *	sift (const ibis::selectClause &sel, const ibis::constPartList &pl, const ibis::whereClause &wc)
	Select the rows satisfying the where clause and store the results in a table object. More...

static table *	sift0 (const ibis::selectClause &, const ibis::constPartList &)
	Select all rows from each data partition and place them in a table object. More...

static table *	sift0S (const ibis::selectClause &, const ibis::constPartList &)
	Select all rows from each data partition and place them in a table object. More...

static table *	sift1 (const ibis::selectClause &, const ibis::constPartList &, const ibis::whereClause &)
	Select the rows satisfying the where clause and store the results in a table object. More...

static table *	sift1S (const ibis::selectClause &, const ibis::constPartList &, const ibis::whereClause &)
	Perform the filter operation involving one column only. More...

static table *	sift2 (const ibis::selectClause &, const ibis::constPartList &, const ibis::whereClause &)
	Select the rows satisfying the where clause and store the results in a table object. More...

static table *	sift2 (const ibis::selectClause &, const ibis::constPartList &, const ibis::array_t< ibis::bitvector * > &)
	Select the rows satisfying the where clause and store the results in a table object. More...

static table *	sift2 (const ibis::selectClause &, const ibis::constPartList &, const ibis::whereClause &, ibis::array_t< ibis::bitvector * > &)
	Select the rows satisfying the where clause and store the results in a table object. More...

static table *	sift2S (const ibis::selectClause &, const ibis::constPartList &, const ibis::whereClause &)
	Select the rows satisfying the where clause and store the results in a table object. More...

static table *	sift2S (const ibis::selectClause &, const ibis::constPartList &, const ibis::array_t< ibis::bitvector * > &)
	Select the rows satisfying the where clause and store the results in a table object. More...

static table *	sift2S (const ibis::selectClause &, const ibis::constPartList &, const ibis::whereClause &, ibis::array_t< ibis::bitvector * > &)
	Select the rows satisfying the where clause and store the results in a table object. More...

Static Public Member Functions inherited from ibis::quaere
static quaere *	create (const char sel, const char from, const char *where)
	Create a query object using the global datasets.

static quaere *	create (const char sel, const char from, const char *where, const ibis::partList &prts)
	Generate a query expression. More...

static quaere *	create (const ibis::part partr, const ibis::part parts, const char colname, const char condr=0, const char conds=0, const char sel=0)
	Specify a natural join operation. More...

Protected Member Functions
	filter ()
	Default constructor. More...

Protected Member Functions inherited from ibis::quaere
	quaere ()
	Default constructor. Only used by derived classes.

Protected Attributes
array_t< ibis::bitvector * >	cand_
	Candidate query results.

array_t< ibis::bitvector * >	hits_
	Solution in bitvector form. More...

const ibis::constPartList *	parts_
	A list of data partitions to query.

const ibis::selectClause *	sel_
	The select clause. More...

const ibis::whereClause *	wc_
	The where clause.

Detailed Description

A simple filtering query.

The where clause does not contain any table names. Following the convention used in older version of the query class, the same where clause is applied to all known data partitions.

Constructor & Destructor Documentation

ibis::filter::filter ( const ibis::whereClause * w )

explicit

Constructor.

The incoming where clause is applied to all known data partitions in ibis::datasets.

ibis::filter::filter	(	const ibis::selectClause *	s,
		const ibis::constPartList *	p,
		const ibis::whereClause *	w
	)

Constructor.

The caller supplies all three clauses of a SQL select statement. The arguments are copied if they are not empty.

Note: This constructor makes a copy of the container for the data partitions, but not the data partitions themselves. In the destructor, only the container is freed, not the data partitions.

ibis::filter::filter	(	const ibis::bitvector &	s,
		const ibis::part &	p
	)

Constructor.

This constructor takes a bit vector and a single data partition. It is intended to regenerate a query result set saved as a hit vector. The caller can use various versions of the function select to reprocess the result from another query.

References hits_, ibis::part::name(), ibis::part::nRows(), and ibis::bitvector::size().

ibis::filter::filter ( )

inlineprotected

Default constructor.

Nothing can be done without explicitly initializing the member variables.

Member Function Documentation

ibis::table * ibis::filter::select ( ) const

virtual

Produce a projection of the joint table.

The select clause associated with the query object is evaluated. If no select clause is provided, it returns a table with no columns. This is different from having a 'count(*)' as the select clause, which produce a table with one row and one column.

Note: We assume that this query object might be reused later and therefore store partial results associated with the query object.

Implements ibis::quaere.

References ibis::datasets, ibis::util::emptyCache(), ibis::fileManager::instance(), ibis::fileManager::printStatus(), sift0(), sift0S(), sift2(), and sift2S().

ibis::table * ibis::filter::select ( const char * ) const

virtual

Produce a project based on the given select clause.

The joint data table is defined by the where clause and the from clause given to the constructor of this object.

Implements ibis::quaere.

References ibis::datasets, ibis::util::emptyCache(), ibis::fileManager::instance(), ibis::selectClause::isSeparable(), ibis::fileManager::printStatus(), sift0(), sift0S(), sift2(), and sift2S().

ibis::table * ibis::filter::select ( const ibis::table::stringArray & colnames ) const

virtual

Produce a projection of all known data partitions.

This function selects all values of the named columns that are not NULL.

Note: This function assumes that this query object is used only once and therefore does not cache the results.

Implements ibis::quaere.

References ibis::datasets, ibis::selectClause::empty(), ibis::util::emptyCache(), ibis::fileManager::instance(), ibis::selectClause::isSeparable(), ibis::fileManager::printStatus(), sift(), sift0(), sift0S(), sift2(), and sift2S().

ibis::table * ibis::filter::sift	(	const ibis::selectClause &	tms,
		const ibis::constPartList &	plist,
		const ibis::whereClause &	cond
	)

static

Select the rows satisfying the where clause and store the results in a table object.

This function determine which of the various variations to call based on the number of columns involved in the query and whether the aggregation functions are separable or not.

References ibis::table::computeHits(), ibis::selectClause::empty(), ibis::whereClause::empty(), ibis::whereClause::getExpr(), ibis::qExpr::getType(), ibis::selectClause::isSeparable(), ibis::selectClause::isUnivariate(), sift1(), sift1S(), sift2(), and sift2S().

Referenced by select(), and ibis::table::select().

ibis::table * ibis::filter::sift0	(	const ibis::selectClause &	tms,
		const ibis::constPartList &	plist
	)

static

Select all rows from each data partition and place them in a table object.

It concatenates the results from different data partitions in the order of the data partitions given in mylist.

It expects both incoming arguments to be valid and non-trivial. It will return a nil pointer if those arguments are nil pointers or empty.

References ibis::selectClause::aggSize(), ibis::bord::append(), ibis::bitvector::cnt(), ibis::bord::describe(), ibis::selectClause::empty(), ibis::bord::groupby(), ibis::bord::nColumns(), ibis::bord::nRows(), ibis::selectClause::numGroupbyKeys(), ibis::util::shortName(), ibis::selectClause::termName(), and ibis::selectClause::verify().

Referenced by select(), and ibis::table::select().

ibis::table * ibis::filter::sift0S	(	const ibis::selectClause &	tms,
		const ibis::constPartList &	plist
	)

static

Select all rows from each data partition and place them in a table object.

It works with select clause containing separable aggregation operations.

This function does check whether aggregations are separable! The caller need to make sure the aggregations are separable befor calling this function.

Note: As of 12/21/2011, Tomas Rybka introduced a logarithmic list of accumulators in this function to reduce the number of times the intermediate results are copied. In this approach, all partitions are merged to a similar-sized accumulator. As their sizes grow, they are merged into larger ones. Finally, all of them are merged together into the final partition. This approach copies each row at most log(n) times instead of n times as in the earlier implementation, where n is the number of data partitions in plist. However, because this appraoch holds the partial results in memory for longer period of time, therefore, it may require more memory than the previous version. Overall this function should still takes less memory than the function sift0.

References ibis::selectClause::aggSize(), ibis::bord::append(), ibis::bitvector::cnt(), ibis::bord::describe(), ibis::table::describe(), ibis::selectClause::empty(), ibis::bord::groupbya(), ibis::bord::groupbyc(), ibis::bord::limit(), ibis::util::log2(), ibis::bord::nColumns(), ibis::table::nColumns(), ibis::bord::nRows(), ibis::table::nRows(), ibis::selectClause::numGroupbyKeys(), ibis::util::shortName(), ibis::selectClause::termName(), and ibis::selectClause::verify().

Referenced by select().

ibis::table * ibis::filter::sift1	(	const ibis::selectClause &	tms,
		const ibis::constPartList &	plist,
		const ibis::whereClause &	cond
	)

static

Select the rows satisfying the where clause and store the results in a table object.

It concatenates the results from different data partitions in the order of the data partitions given in mylist.

This version is intended to work with only one column of raw data in the select clause and one term in the where clause, the where clause must be a simple range expression, and the column involved in these clauses are expected to be the same. If any of these conditions is not satisfied, it returns a nil pointer.

References ibis::selectClause::aggSize(), ibis::bord::append(), ibis::bord::describe(), ibis::whereClause::getExpr(), ibis::qExpr::getType(), ibis::bord::groupby(), ibis::selectClause::isUnivariate(), ibis::bord::nColumns(), ibis::bord::nRows(), ibis::selectClause::numGroupbyKeys(), ibis::util::shortName(), ibis::selectClause::termName(), and ibis::selectClause::verify().

Referenced by sift().

ibis::table * ibis::filter::sift1S	(	const ibis::selectClause &	tms,
		const ibis::constPartList &	plist,
		const ibis::whereClause &	cond
	)

static

Perform the filter operation involving one column only.

The operations in the select clause are all separable. The caller is to make sure the where clause and the select clause involve one column and the aggregations operations in the select clause are separable. If any of these conditions are violated, this function will return a nil pointer.

References ibis::selectClause::aggSize(), ibis::bord::append(), ibis::bord::describe(), ibis::whereClause::getExpr(), ibis::qExpr::getType(), ibis::bord::groupbya(), ibis::bord::groupbyc(), ibis::selectClause::isUnivariate(), ibis::bord::limit(), ibis::util::log2(), ibis::bord::nColumns(), ibis::bord::nRows(), ibis::selectClause::numGroupbyKeys(), ibis::util::shortName(), ibis::selectClause::termName(), and ibis::selectClause::verify().

Referenced by sift().

ibis::table * ibis::filter::sift2	(	const ibis::selectClause &	tms,
		const ibis::constPartList &	plist,
		const ibis::whereClause &	cond
	)

static

Select the rows satisfying the where clause and store the results in a table object.

There can be arbitrary number of columns involved in the where clause and the select clause. It concatenates the results from different data partitions in the order of the data partitions given in mylist and therefore requires more memory than sift2S.

It expects all three arguments to be valid and non-trivial. It will return a nil pointer if those arguments are nil pointers or empty.

References ibis::selectClause::aggSize(), ibis::bord::append(), ibis::bitvector::cnt(), ibis::table::computeHits(), ibis::bord::describe(), ibis::bord::dump(), ibis::selectClause::empty(), ibis::whereClause::empty(), ibis::whereClause::getExpr(), ibis::bord::groupby(), ibis::bord::nColumns(), ibis::bord::nRows(), ibis::selectClause::numGroupbyKeys(), ibis::countQuery::setWhereClause(), ibis::util::shortName(), ibis::selectClause::termName(), and ibis::selectClause::verify().

Referenced by select(), and sift().

ibis::table * ibis::filter::sift2	(	const ibis::selectClause &	tms,
		const ibis::constPartList &	plist,
		const ibis::array_t< ibis::bitvector * > &	hits
	)

static

Select the rows satisfying the where clause and store the results in a table object.

It concatenates the results from different data partitions in the order of the data partitions given in mylist.

This verison takes the existing solutions as the 3rd argument instead of a set of query conditions.

References ibis::selectClause::aggSize(), ibis::bord::append(), ibis::bitvector::cnt(), ibis::bord::describe(), ibis::selectClause::empty(), ibis::bord::groupby(), ibis::bord::nColumns(), ibis::bord::nRows(), ibis::selectClause::numGroupbyKeys(), ibis::util::shortName(), ibis::array_t< T >::size(), ibis::selectClause::termName(), and ibis::selectClause::verify().

ibis::table * ibis::filter::sift2	(	const ibis::selectClause &	tms,
		const ibis::constPartList &	plist,
		const ibis::whereClause &	cond,
		ibis::array_t< ibis::bitvector * > &	hits
	)

static

Select the rows satisfying the where clause and store the results in a table object.

It concatenates the results from different data partitions in the order of the data partitions given in mylist.

This version records the bitvectors generated as the intermediate solutions.

References ibis::selectClause::aggSize(), ibis::bord::append(), ibis::util::clear(), ibis::bitvector::cnt(), ibis::table::computeHits(), ibis::array_t< T >::copy(), ibis::bord::describe(), ibis::selectClause::empty(), ibis::whereClause::empty(), ibis::whereClause::getExpr(), ibis::bord::groupby(), ibis::bord::nColumns(), ibis::bord::nRows(), ibis::selectClause::numGroupbyKeys(), ibis::array_t< T >::push_back(), ibis::array_t< T >::reserve(), ibis::countQuery::setWhereClause(), ibis::util::shortName(), ibis::array_t< T >::size(), ibis::selectClause::termName(), and ibis::selectClause::verify().

ibis::table * ibis::filter::sift2S	(	const ibis::selectClause &	tms,
		const ibis::constPartList &	plist,
		const ibis::whereClause &	cond
	)

static

Select the rows satisfying the where clause and store the results in a table object.

This version can accept an arbitrary where clause. It assumes the select clause can be evaluated one partition at a time, in other word, the aggregations are separable.

References ibis::selectClause::aggSize(), ibis::bord::append(), ibis::bitvector::cnt(), ibis::table::computeHits(), ibis::bord::describe(), ibis::selectClause::empty(), ibis::whereClause::empty(), ibis::whereClause::getExpr(), ibis::bord::groupbya(), ibis::bord::groupbyc(), ibis::bord::limit(), ibis::util::log2(), ibis::bord::nColumns(), ibis::bord::nRows(), ibis::selectClause::numGroupbyKeys(), ibis::countQuery::setWhereClause(), ibis::util::shortName(), ibis::selectClause::termName(), and ibis::selectClause::verify().

Referenced by select(), and sift().

ibis::table * ibis::filter::sift2S	(	const ibis::selectClause &	tms,
		const ibis::constPartList &	plist,
		const ibis::array_t< ibis::bitvector * > &	hits
	)

static

Select the rows satisfying the where clause and store the results in a table object.

It performs the aggregations on each data partition separately. This is only valid if the aggregations in the select clause is indeed separable. The caller is responsible for making sure the select clause is separable.

References ibis::selectClause::aggSize(), ibis::bord::append(), ibis::bitvector::cnt(), ibis::bord::describe(), ibis::selectClause::empty(), ibis::bord::groupbya(), ibis::bord::groupbyc(), ibis::bord::limit(), ibis::util::log2(), ibis::bord::nColumns(), ibis::bord::nRows(), ibis::selectClause::numGroupbyKeys(), ibis::util::shortName(), ibis::array_t< T >::size(), ibis::selectClause::termName(), and ibis::selectClause::verify().

ibis::table * ibis::filter::sift2S	(	const ibis::selectClause &	tms,
		const ibis::constPartList &	plist,
		const ibis::whereClause &	cond,
		ibis::array_t< ibis::bitvector * > &	hits
	)

static

Select the rows satisfying the where clause and store the results in a table object.

It performs the aggregation operations one data partition at a time. This is only valid if the aggregations are indeed separable.

References ibis::selectClause::aggSize(), ibis::bord::append(), ibis::util::clear(), ibis::bitvector::cnt(), ibis::table::computeHits(), ibis::array_t< T >::copy(), ibis::bord::describe(), ibis::selectClause::empty(), ibis::whereClause::empty(), ibis::whereClause::getExpr(), ibis::bord::groupbya(), ibis::bord::groupbyc(), ibis::bord::limit(), ibis::util::log2(), ibis::bord::nColumns(), ibis::bord::nRows(), ibis::selectClause::numGroupbyKeys(), ibis::array_t< T >::push_back(), ibis::array_t< T >::reserve(), ibis::countQuery::setWhereClause(), ibis::util::shortName(), ibis::array_t< T >::size(), ibis::selectClause::termName(), and ibis::selectClause::verify().

Member Data Documentation

array_t<ibis::bitvector*> ibis::filter::hits_

mutableprotected

Solution in bitvector form.

If cand is no nil, then this bit vector is a lower bound.

Referenced by filter().

const ibis::selectClause* ibis::filter::sel_

protected

The select clause.

Also used to spply aliases. If the function select is called with an empty select clause, then this variable will be used as the substitute.

The documentation for this class was generated from the following files:

filter.h
filter.cpp



	Contact us Disclaimers FastBit source code FastBit mailing list archive

Public Member Functions

Static Public Member Functions

Protected Member Functions

Protected Attributes

Detailed Description

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation