Public Member Functions | Static Public Member Functions | Protected Member Functions | Protected Attributes | List of all members
ibis::bord::column Class Reference

An in-memory version of ibis::column. More...

#include <bord.h>

Inheritance diagram for ibis::bord::column:
ibis::column

Public Member Functions

void addCounts (uint32_t)
 Extend the buffer to have nr elements. More...
 
virtual long append (const char *dt, const char *df, const uint32_t nold, const uint32_t nnew, uint32_t nbuf, char *buf)
 Append new data in directory df to the end of existing data in dt. More...
 
virtual long append (const void *vals, const ibis::bitvector &msk)
 Append user supplied data to the current column. More...
 
virtual long append (const ibis::column &scol, const ibis::bitvector &msk)
 Append selected values from the given column to the current column. More...
 
virtual long append (const ibis::column &scol, const ibis::qContinuousRange &cnd)
 Append selected values from the given column to the current column. More...
 
void append (const void *, uint32_t)
 Append a value. More...
 
void append (const void *, uint32_t, const void *, uint32_t, ibis::selectClause::AGREGADO)
 Append the value genenerated from the the operation on the incoming columns. More...
 
 column (const ibis::bord *tbl, ibis::TYPE_T t, const char *name, void *buf=0, const char *desc="", double low=DBL_MAX, double high=-DBL_MAX)
 Constructor.
 
 column (ibis::TYPE_T t, const char *nm, void *st, uint64_t *dim, uint64_t nd)
 Constructor.
 
 column (FastBitReadExtArray rd, void *ctx, uint64_t *dims, uint64_t nd, ibis::TYPE_T t, const char *name, const char *desc="", double lo=DBL_MAX, double hi=-DBL_MAX)
 Constructor. More...
 
 column (const ibis::bord *, const ibis::column &, void *buf)
 Constructor. More...
 
 column (const column &rhs)
 Copy constructor. Performs a shallow copy of the storage buffer.
 
virtual void computeMinMax ()
 Compute the actual min/max values. More...
 
virtual void computeMinMax (const char *)
 Compute the actual min/max values. More...
 
virtual void computeMinMax (const char *, double &, double &, bool &) const
 Compute the actual min/max of the data in directory dir. More...
 
int dump (std::ostream &out, uint32_t i) const
 
bool equal_to (const column &) const
 Does this column have the same values as the other.
 
bool equal_to (const column &, uint32_t, uint32_t) const
 Does the ith value of this column equal to the jth value of other?
 
virtual long evaluateRange (const ibis::qContinuousRange &cmp, const ibis::bitvector &mask, ibis::bitvector &res) const
 Compute the exact answer. More...
 
virtual long evaluateRange (const ibis::qDiscreteRange &cmp, const ibis::bitvector &mask, ibis::bitvector &res) const
 Compute the exact answer to a discrete range expression.
 
void *& getArray ()
 
void * getArray () const
 
virtual const ibis::dictionarygetDictionary () const
 Return the dictionary associated with the column. More...
 
const ibis::array_t< uint64_t > & getMeshShape () const
 
virtual ibis::fileManager::storagegetRawData () const
 Retrieve the raw data buffer as an ibis::fileManager::storage. More...
 
virtual int getString (uint32_t, std::string &) const
 Return the string at the ith row. More...
 
virtual int getValuesArray (void *) const
 Makes a copy of the in-memory data. More...
 
virtual bool hasRawData () const
 Does the raw data file exist?
 
virtual long keywordSearch (const char *, ibis::bitvector &) const
 Find the given keyword and return the rows.
 
virtual long keywordSearch (const char *) const
 Return an upper bound on the number of matches.
 
virtual long keywordSearch (const std::vector< std::string > &, ibis::bitvector &) const
 Find the given keyword and return the rows.
 
virtual long keywordSearch (const std::vector< std::string > &) const
 Return an upper bound on the number of matches.
 
bool less_than (const column &, uint32_t, uint32_t) const
 Is the ith value of this column less than the jth value of other?
 
int limit (uint32_t nr)
 Reduce the number of rows stored in this column object to nr. More...
 
virtual long patternSearch (const char *) const
 Compute an estimate of the maximum number of possible matches. More...
 
virtual long patternSearch (const char *, ibis::bitvector &) const
 
int restoreCategoriesAsStrings (const ibis::category &)
 Convert the integer representation back to the string representation. More...
 
void reverseRows ()
 
virtual array_t< signed char > * selectBytes (const ibis::bitvector &) const
 Retrieve selected 1-byte integer values. More...
 
virtual array_t< double > * selectDoubles (const ibis::bitvector &) const
 Put the selected values into an array as doubles. More...
 
virtual array_t< float > * selectFloats (const ibis::bitvector &) const
 Put selected values of a float column into an array.
 
virtual array_t< int32_t > * selectInts (const ibis::bitvector &) const
 Return selected rows of the column in an array_t object. More...
 
virtual array_t< int64_t > * selectLongs (const ibis::bitvector &) const
 Can be called on all integral types. More...
 
virtual std::vector< ibis::opaque > * selectOpaques (const bitvector &mask) const
 
virtual array_t< int16_t > * selectShorts (const ibis::bitvector &) const
 Return selected rows of the column in an array_t object. More...
 
virtual std::vector< std::string > * selectStrings (const bitvector &mask) const
 Output the selected values as strings. More...
 
virtual array_t< unsigned char > * selectUBytes (const ibis::bitvector &) const
 Return selected rows of the column in an array_t object. More...
 
virtual array_t< uint32_t > * selectUInts (const ibis::bitvector &) const
 Can be called on columns of unsigned integral types, UINT, CATEGORY, USHORT, and UBYTE. More...
 
virtual array_t< uint64_t > * selectULongs (const ibis::bitvector &) const
 Return selected rows of the column in an array_t object. More...
 
virtual array_t< uint16_t > * selectUShorts (const ibis::bitvector &) const
 Return selected rows of the column in an array_t object. More...
 
void setDictionary (const ibis::dictionary *d)
 Assign the dictionary to use.
 
int setMeshShape (uint64_t *, uint64_t)
 Specify the shape of the array. More...
 
virtual long stringSearch (const char *, ibis::bitvector &) const
 Locate the strings that match the given string. More...
 
virtual long stringSearch (const std::vector< std::string > &, ibis::bitvector &) const
 
virtual long stringSearch (const char *) const
 Compute an estimate of the maximum number of possible matches. More...
 
virtual long stringSearch (const std::vector< std::string > &) const
 Compute an estimate of the maximum number of possible matches. More...
 
virtual ~column ()
 Destructor. More...
 
- Public Member Functions inherited from ibis::column
virtual int attachIndex (double *, uint64_t, int64_t *, uint64_t, void *, FastBitReadBitmaps) const
 
virtual int attachIndex (double *, uint64_t, int64_t *, uint64_t, uint32_t *, uint64_t) const
 
void binWeights (std::vector< uint32_t > &) const
 Retrive the number of rows in each bin.
 
template<typename T >
long castAndWrite (const array_t< double > &vals, ibis::bitvector &mask, const T special)
 Cast the incoming array into the specified type T before writing the values to the file for this column. More...
 
 column (const column &rhs)
 The copy constructor. More...
 
 column (const part *tbl, FILE *file)
 Reconstitute a column from the content of a file. More...
 
 column (const part *tbl, ibis::TYPE_T t, const char *name, const char *desc="", double low=DBL_MAX, double high=-DBL_MAX)
 Construct a new column object based on type and name.
 
int contractRange (ibis::qContinuousRange &rng) const
 Contract the range expression so that the new range falls exactly on the bin boundaries. More...
 
const char * dataFileName (std::string &fname, const char *dir=0) const
 Name of the data file in the given data directory. More...
 
const char * description () const
 Description of the column. Can be an arbitrary string.
 
void description (const char *d)
 
int elementSize () const
 Size of a data element in bytes.
 
virtual double estimateCost (const ibis::qContinuousRange &cmp) const
 Estimate the cost of evaluating the query expression.
 
virtual double estimateCost (const ibis::qDiscreteRange &cmp) const
 Estimate the cost of evaluating a dicreate range expression.
 
virtual double estimateCost (const ibis::qIntHod &cmp) const
 Estimate the cost of evaluating a dicreate range expression.
 
virtual double estimateCost (const ibis::qUIntHod &cmp) const
 Estimate the cost of evaluating a dicreate range expression.
 
virtual double estimateCost (const ibis::qString &) const
 Estimate the cost of evaluating a string lookup.
 
virtual double estimateCost (const ibis::qAnyString &) const
 Estimate the cost of looking up a group of strings.
 
virtual long estimateRange (const ibis::qContinuousRange &cmp, ibis::bitvector &low, ibis::bitvector &high) const
 Compute a lower bound and an upper bound on the number of hits using the bitmap index. More...
 
virtual long estimateRange (const ibis::qDiscreteRange &cmp, ibis::bitvector &low, ibis::bitvector &high) const
 Compute a lower bound and an upper bound for hits. More...
 
virtual long estimateRange (const ibis::qIntHod &cmp, ibis::bitvector &low, ibis::bitvector &high) const
 Compute a lower bound and an upper bound for hits. More...
 
virtual long estimateRange (const ibis::qUIntHod &cmp, ibis::bitvector &low, ibis::bitvector &high) const
 Compute a lower bound and an upper bound for hits. More...
 
virtual long estimateRange (const ibis::qContinuousRange &cmp) const
 Use the index of the column to compute an upper bound on the number of hits. More...
 
virtual long estimateRange (const ibis::qDiscreteRange &cmp) const
 
virtual long estimateRange (const ibis::qIntHod &cmp) const
 Compute an upper bound on the number of hits. More...
 
virtual long estimateRange (const ibis::qUIntHod &cmp) const
 Compute an upper bound on the number of hits. More...
 
virtual long evaluateAndSelect (const ibis::qContinuousRange &, const ibis::bitvector &, void *, ibis::bitvector &) const
 Evaluate a range condition and retrieve the selected values. More...
 
virtual long evaluateRange (const ibis::qIntHod &cmp, const ibis::bitvector &mask, ibis::bitvector &res) const
 Compute the exact answer to a discrete range expression.
 
virtual long evaluateRange (const ibis::qUIntHod &cmp, const ibis::bitvector &mask, ibis::bitvector &res) const
 Compute the exact answer to a discrete range expression.
 
int expandRange (ibis::qContinuousRange &rng) const
 Expand the range expression so that the new range falls exactly on the bin boundaries. More...
 
virtual const char * findString (const char *) const
 Determine if the input string has appeared in this data partition. More...
 
std::string fullname () const
 Fully qualified name. More...
 
int getDataflag () const
 
array_t< double > * getDoubleArray () const
 Return all rows of the column as an array_t object.
 
array_t< float > * getFloatArray () const
 Return all rows of the column as an array_t object.
 
array_t< int32_t > * getIntArray () const
 Return all rows of the column as an array_t object. More...
 
void getNullMask (bitvector &mask) const
 If there is a null mask stored already, return a shallow copy of it in mask. More...
 
virtual int getOpaque (uint32_t, ibis::opaque &) const
 Return the raw binary value for the ith row. More...
 
const unixTimeScribegetTimeFormat () const
 
virtual float getUndecidable (const ibis::qContinuousRange &cmp, ibis::bitvector &iffy) const
 Compute the locations of the rows can not be decided by the index. More...
 
virtual float getUndecidable (const ibis::qDiscreteRange &cmp, ibis::bitvector &iffy) const
 Find rows that can not be decided with the existing index.
 
virtual float getUndecidable (const ibis::qIntHod &cmp, ibis::bitvector &iffy) const
 Find rows that can not be decided with the existing index. More...
 
virtual float getUndecidable (const ibis::qUIntHod &cmp, ibis::bitvector &iffy) const
 Find rows that can not be decided with the existing index. More...
 
bool hasIndex () const
 !< Are the values sorted? More...
 
bool hasRoster () const
 Is there a roster list built for this column? Returns true for yes, false for no. More...
 
uint32_t indexedRows () const
 Compute the number of rows captured by the index of this column. More...
 
virtual void indexSerialSizes (uint64_t &, uint64_t &, uint64_t &) const
 Compute the sizes (in number of elements) of three arrays that would be produced by writeIndex. More...
 
virtual long indexSize () const
 Compute the index size (in bytes). More...
 
const char * indexSpec () const
 
void indexSpec (const char *spec)
 !< Retrieve the number of bins used. More...
 
void indexSpeedTest () const
 Perform a set of built-in tests to determine the speed of common operations. More...
 
virtual int indexWrite (ibis::array_t< double > &, ibis::array_t< int64_t > &, ibis::array_t< uint32_t > &) const
 Write the index into three arrays.
 
bool isFloat () const
 Are they floating-point values?
 
bool isInteger () const
 Are they integer values?
 
bool isNumeric () const
 Are they numberical values?
 
bool isSignedInteger () const
 Are they signed integer values?
 
bool isSorted () const
 
void isSorted (bool)
 Change the flag m_sorted. More...
 
bool isUnsignedInteger () const
 Are they unsigned integer values?
 
virtual void loadIndex (const char *iopt=0, int ropt=0) const throw ()
 Load the index associated with the column. More...
 
void logMessage (const char *event, const char *fmt,...) const
 Log messages using printf syntax.
 
void logWarning (const char *event, const char *fmt,...) const
 Log warming message using printf syntax.
 
const double & lowerBound () const
 The lower bound of the values.
 
void lowerBound (double d)
 
const char * name () const
 Name of the column.
 
void name (const char *nm)
 Rename the column.
 
int nRows () const
 
const char * nullMaskName (std::string &fname) const
 Name of the NULL mask file. More...
 
uint32_t numBins () const
 !< Retrieve the index specification.
 
const partpartition () const
 
const part *& partition ()
 
void preferredBounds (std::vector< double > &) const
 Retrive the bin boundaries if the index currently in use.
 
virtual void print (std::ostream &out) const
 Print some basic infomation about this column.
 
void purgeIndexFile (const char *dir=0) const
 Purge the index files assocated with the current column.
 
virtual long saveSelected (const ibis::bitvector &sel, const char *dest, char *buf, uint32_t nbuf)
 Write the selected records to the specified directory. More...
 
long selectValues (const bitvector &, void *) const
 Return selected rows of the column in an array_t object. More...
 
long selectValues (const bitvector &, void *, array_t< uint32_t > &) const
 Return selected rows of the column in an array_t object along with their positions. More...
 
long selectValues (const ibis::qContinuousRange &, void *) const
 Select the values satisfying the specified range condition.
 
void setDataflag (int df)
 
int setNullMask (const bitvector &)
 Change the null mask to the user specified one. More...
 
void setTimeFormat (const char *)
 Add a custom format for the column to be interpretted as unix time stamps.
 
void setTimeFormat (const unixTimeScribe &)
 
virtual long truncateData (const char *dir, uint32_t nent, ibis::bitvector &mask) const
 Truncate the number of records in the named dir to nent. More...
 
ibis::TYPE_T type () const
 Type of the data. More...
 
virtual void unloadIndex () const
 Unload the index associated with the column. More...
 
const double & upperBound () const
 The upper bound of the values.
 
void upperBound (double d)
 
virtual void write (FILE *file) const
 Write the metadata entry. More...
 
virtual long writeData (const char *dir, uint32_t nold, uint32_t nnew, ibis::bitvector &mask, const void *va1, void *va2=0)
 Write the content in array va1 to directory dir. More...
 
virtual double getActualMin () const
 A group of functions to compute some basic statistics for the column values. More...
 
virtual double getActualMax () const
 Compute the actual maximum value by reading the data or examining the index. More...
 
virtual double getSum () const
 Compute the sum of all values by reading the data.
 
long getCumulativeDistribution (std::vector< double > &bounds, std::vector< uint32_t > &counts) const
 Compute the actual data distribution. More...
 
long getDistribution (std::vector< double > &bbs, std::vector< uint32_t > &counts) const
 Count the number of records in each bin. More...
 

Static Public Member Functions

static int addBlobs (std::vector< ibis::opaque > *&, uint32_t, const std::vector< ibis::opaque > &)
 
template<typename T >
static int addIncoreData (array_t< T > *&to, uint32_t nold, const array_t< T > &from, const T special)
 Append new data (in from) to a larger array (pointed to by to). More...
 
static int addStrings (std::vector< std::string > *&, uint32_t, const std::vector< std::string > &)
 
- Static Public Member Functions inherited from ibis::column
template<typename T >
static void actualMinMax (const array_t< T > &vals, const ibis::bitvector &mask, double &min, double &max, bool &asc)
 Compute the minimum and maximum of the values in the array.
 
template<typename T >
static T computeMax (const array_t< T > &vals, const ibis::bitvector &mask)
 Compute the maximum value in the array.
 
template<typename T >
static T computeMin (const array_t< T > &vals, const ibis::bitvector &mask)
 Compute the minimum value in the array.
 
template<typename T >
static double computeSum (const array_t< T > &vals, const ibis::bitvector &mask)
 Compute the sum of values in the array.
 

Protected Member Functions

columnoperator= (const column &)
 
- Protected Member Functions inherited from ibis::column
void actualMinMax (const char *fname, const ibis::bitvector &mask, double &min, double &max, bool &asc) const
 Given the name of the data file, compute the actual minimum and the maximum value. More...
 
long appendStrings (const std::vector< std::string > &, const ibis::bitvector &)
 Append the strings to the current data. More...
 
template<typename T >
long appendValues (const array_t< T > &, const ibis::bitvector &)
 Append the content of incoming array to the current data. More...
 
double computeMax () const
 Read the base data to compute the maximum value.
 
double computeMin () const
 Read the data values and compute the minimum value.
 
double computeSum () const
 Read the base data to compute the total sum.
 
template<typename T >
uint32_t findLower (int fdes, const uint32_t nr, const T tgt) const
 Find the smallest value >= tgt. More...
 
template<typename T >
uint32_t findUpper (int fdes, const uint32_t nr, const T tgt) const
 Find the smallest value > tgt. More...
 
void logError (const char *event, const char *fmt,...) const
 Print messages started with "Error" and throw a string exception.
 
virtual int searchSorted (const ibis::qContinuousRange &, ibis::bitvector &) const
 Resolve a continuous range condition on a sorted column.
 
virtual int searchSorted (const ibis::qDiscreteRange &, ibis::bitvector &) const
 Resolve a discrete range condition on a sorted column.
 
virtual int searchSorted (const ibis::qIntHod &, ibis::bitvector &) const
 Resolve a discrete range condition on a sorted column.
 
virtual int searchSorted (const ibis::qUIntHod &, ibis::bitvector &) const
 Resolve a discrete range condition on a sorted column.
 
template<typename T >
int searchSortedICC (const array_t< T > &vals, const ibis::qContinuousRange &rng, ibis::bitvector &hits) const
 Resolve a continuous range condition on an array of values.
 
template<typename T >
int searchSortedICD (const array_t< T > &vals, const ibis::qDiscreteRange &rng, ibis::bitvector &hits) const
 Resolve a discrete range condition on an array of values.
 
template<typename T >
int searchSortedICD (const array_t< T > &vals, const ibis::qIntHod &rng, ibis::bitvector &hits) const
 Resolve a discrete range condition on an array of values.
 
template<typename T >
int searchSortedICD (const array_t< T > &vals, const ibis::qUIntHod &rng, ibis::bitvector &hits) const
 Resolve a discrete range condition on an array of values.
 
template<typename T >
int searchSortedOOCC (const char *fname, const ibis::qContinuousRange &rng, ibis::bitvector &hits) const
 Resolve a continuous range condition using file operations. More...
 
template<typename T >
int searchSortedOOCD (const char *fname, const ibis::qDiscreteRange &rng, ibis::bitvector &hits) const
 Resolve a discrete range condition using file operations. More...
 
template<typename T >
int searchSortedOOCD (const char *fname, const ibis::qIntHod &rng, ibis::bitvector &hits) const
 Resolve a discrete range condition using file operations. More...
 
template<typename T >
int searchSortedOOCD (const char *fname, const ibis::qUIntHod &rng, ibis::bitvector &hits) const
 Resolve a discrete range condition using file operations. More...
 
template<typename T >
long selectToOpaques (const char *, const bitvector &, std::vector< ibis::opaque > &) const
 
template<typename T >
long selectToStrings (const char *, const bitvector &, std::vector< std::string > &) const
 Extract the values masked 1 and convert them to strings.
 
template<>
long selectToStrings (const char *, const bitvector &, std::vector< std::string > &) const
 
template<>
long selectToStrings (const char *, const bitvector &, std::vector< std::string > &) const
 
template<>
long selectToStrings (const char *dfn, const bitvector &mask, std::vector< std::string > &str) const
 
template<>
long selectToStrings (const char *dfn, const bitvector &mask, std::vector< std::string > &str) const
 
template<typename T >
long selectValuesT (const char *, const bitvector &, array_t< T > &) const
 Select values marked in the bitvector mask. More...
 
template<typename T >
long selectValuesT (const char *, const bitvector &mask, array_t< T > &vals, array_t< uint32_t > &inds) const
 Select the values marked in the bitvector mask. More...
 
long string2int (int fptr, dictionary &dic, uint32_t nbuf, char *buf, array_t< uint32_t > &out) const
 Convert strings in the opened file to a list of integers with the aid of a dictionary. More...
 

Protected Attributes

void * buffer
 The in-memory storage. More...
 
const ibis::dictionarydic
 A dictionary. More...
 
ibis::array_t< uint64_t > shape
 Shape of the mesh for the data. More...
 
void * xmeta
 Context to be passed back to reader.
 
FastBitReadExtArray xreader
 Reader for externally managed data.
 
- Protected Attributes inherited from ibis::column
int dataflag
 Presence of the data file. More...
 
ibis::indexidx
 The index for this column. It is not considered as a must-have member.
 
ibis::util::sharedInt32 idxcnt
 The number of functions using the index.
 
double lower
 !< Are the column values in ascending order?
 
std::string m_bins
 !< Free-form description of the column.
 
std::string m_desc
 !< Name of the column.
 
std::string m_name
 !< Data type.
 
bool m_sorted
 !< Index/binning specification.
 
ibis::TYPE_T m_type
 !< The entries marked 1 are valid.
 
unixTimeScribem_utscribe
 !< The maximum value.
 
ibis::bitvector mask_
 !< Data partition containing this column.
 
const partthePart
 
double upper
 !< The minimum value.
 

Detailed Description

An in-memory version of ibis::column.

For integers and floating-point values, the buffer (with type void*) points to an ibis::array_t<T> where the type T is designated by the column type. For a string-valued column, the buffer (with type void*) is std::vector<std::string>*.

Note
Since the in-memory data tables are typically created at run-time through select operations, the data types associated with a column is only known at run-time. Casting to void* is a ugly option; the developers welcome suggestions for a replacement.

Constructor & Destructor Documentation

ibis::bord::column::column ( FastBitReadExtArray  rd,
void *  ctx,
uint64_t *  dims,
uint64_t  nd,
ibis::TYPE_T  t,
const char *  name,
const char *  desc = "",
double  lo = DBL_MAX,
double  hi = -DBL_MAX 
)

Constructor.

Use the external reader. This is meant for fix-sized elements only, i.e., integers and floating-point numbers. More specifically, this is not for strings.

References ibis::part::setMeshShape().

ibis::bord::column::column ( const ibis::bord tbl,
const ibis::column old,
void *  st 
)

Constructor.

Note
Transfer the ownership of st to the new column object.

References ibis::column::dataflag, ibis::column::getNullMask(), and ibis::column::mask_.

ibis::bord::column::~column ( )
virtual

Destructor.

It acquires a write lock to make sure all other operations have completed.

Reimplemented from ibis::column.

References ibis::table::freeBuffer().

Member Function Documentation

void ibis::bord::column::addCounts ( uint32_t  nr)

Extend the buffer to have nr elements.

All new elements have the value 1U.

References ibis::array_t< T >::size(), and ibis::UINT.

Referenced by ibis::bord::append().

template<typename T >
int ibis::bord::column::addIncoreData ( array_t< T > *&  to,
uint32_t  nold,
const array_t< T > &  from,
const T  special 
)
static

Append new data (in from) to a larger array (pointed to by to).

References ibis::array_t< T >::copy(), ibis::array_t< T >::reserve(), and ibis::array_t< T >::size().

long ibis::bord::column::append ( const char *  dt,
const char *  df,
const uint32_t  nold,
const uint32_t  nnew,
uint32_t  nbuf,
char *  buf 
)
virtual

Append new data in directory df to the end of existing data in dt.

Append the content of file in df to end of file in dt.

It returns the number of rows appended or a negative number to indicate error.

Note
The directories dt and df can not be same.
This function does not update the mininimum and the maximum of the column.

Reimplemented from ibis::column.

References ibis::column::append().

Referenced by ibis::bord::append(), and fastbit_iapi_extend_array().

long ibis::bord::column::append ( const void *  vals,
const ibis::bitvector msk 
)
virtual

Append user supplied data to the current column.

The incoming values is carried by a void*, which is cast to the same type as the buffer used by the column. The mask is used to indicate which values in the incoming array are to be included.

Reimplemented from ibis::column.

References ibis::BIT, ibis::BLOB, ibis::CATEGORY, ibis::bitvector::cnt(), ibis::DOUBLE, ibis::FLOAT, ibis::INT, ibis::LONG, ibis::SHORT, ibis::bitvector::size(), ibis::TEXT, ibis::TYPESTRING, ibis::UBYTE, ibis::UINT, ibis::ULONG, and ibis::USHORT.

long ibis::bord::column::append ( const ibis::column scol,
const ibis::bitvector msk 
)
virtual

Append selected values from the given column to the current column.

This function extracts the values using the given mask from scol, and then append the values to the current column. The type of scol must be ligitimately converted to the type of this column. It returns the number of values added to the column on success, or a negative number to indicate errors.

References ibis::BLOB, ibis::CATEGORY, ibis::bitvector::cnt(), dic, ibis::DOUBLE, ibis::FLOAT, ibis::INT, ibis::LONG, ibis::column::selectBytes(), ibis::column::selectDoubles(), ibis::column::selectFloats(), ibis::column::selectInts(), ibis::column::selectLongs(), ibis::column::selectShorts(), ibis::column::selectStrings(), ibis::column::selectUBytes(), ibis::column::selectUInts(), ibis::column::selectULongs(), ibis::column::selectUShorts(), ibis::SHORT, ibis::bitvector::size(), ibis::TEXT, ibis::TYPESTRING, ibis::UBYTE, ibis::UINT, ibis::ULONG, and ibis::USHORT.

long ibis::bord::column::append ( const ibis::column scol,
const ibis::qContinuousRange cnd 
)
virtual

Append selected values from the given column to the current column.

This function extracts the values using the given range condition on scol, and then append the values to the current column. The type of scol must be ligitimately converted to the type of this column.

It returns 0 to indicate success, a negative number to indicate error.

References ibis::DOUBLE, ibis::FLOAT, ibis::INT, ibis::LONG, ibis::column::selectValues(), ibis::SHORT, ibis::TYPESTRING, ibis::UBYTE, ibis::UINT, ibis::ULONG, and ibis::USHORT.

void ibis::bord::column::append ( const void *  c1,
uint32_t  i1 
)
inline

Append a value.

Note
The first argument c1 is expected to be an array_t object with data type same as this column.

References ibis::BLOB, ibis::CATEGORY, ibis::DOUBLE, ibis::FLOAT, ibis::INT, ibis::LONG, ibis::array_t< T >::push_back(), ibis::SHORT, ibis::TEXT, ibis::TYPESTRING, ibis::UBYTE, ibis::UINT, ibis::ULONG, and ibis::USHORT.

void ibis::bord::column::append ( const void *  c1,
uint32_t  i1,
const void *  c2,
uint32_t  i2,
ibis::selectClause::AGREGADO  agg 
)
inline

Append the value genenerated from the the operation on the incoming columns.

Note
Both arguemnt c1 and c2 are expected to array_t objects with the same data type as this column.

References ibis::DOUBLE, ibis::FLOAT, ibis::INT, ibis::LONG, ibis::array_t< T >::push_back(), ibis::SHORT, ibis::TYPESTRING, ibis::UBYTE, ibis::UINT, ibis::ULONG, and ibis::USHORT.

void ibis::bord::column::computeMinMax ( )
virtual

Compute the actual min/max values.

It actually goes through all the values. This function reads the data in the active data directory and modifies the member variables to record the actual min/max.

Reimplemented from ibis::column.

References ibis::part::computeMinMax().

void ibis::bord::column::computeMinMax ( const char *  dir)
virtual

Compute the actual min/max values.

It actually goes through all the values. This function reads the data in the given directory and modifies the member variables to record the actual min/max.

Reimplemented from ibis::column.

References ibis::part::computeMinMax().

void ibis::bord::column::computeMinMax ( const char *  dir,
double &  min,
double &  max,
bool &  asc 
) const
virtual

Compute the actual min/max of the data in directory dir.

Report the actual min/max found back through output arguments min and max. This version does not modify the min/max recorded in this column object.

Reimplemented from ibis::column.

References ibis::column::actualMinMax(), ibis::BIT, ibis::DOUBLE, ibis::FLOAT, ibis::INT, ibis::LONG, ibis::OID, ibis::SHORT, ibis::TYPESTRING, ibis::UBYTE, ibis::UINT, ibis::ULONG, and ibis::USHORT.

long ibis::bord::column::evaluateRange ( const ibis::qContinuousRange cmp,
const ibis::bitvector mask,
ibis::bitvector low 
) const
virtual

Compute the exact answer.

Attempts to use the index if one is available, otherwise use the base data.

Return a negative value to indicate error, 0 to indicate no hit, and positive value to indicate there are zero or more hits.

Reimplemented from ibis::column.

References ibis::bitvector::adjustSize(), ibis::BIT, ibis::BLOB, ibis::bitvector::clear(), ibis::bitvector::cnt(), ibis::bitvector::copy(), ibis::part::doScan(), ibis::DOUBLE, ibis::FLOAT, ibis::part::getNullMask(), ibis::qContinuousRange::inRange(), ibis::INT, ibis::LONG, ibis::OID, ibis::bitvector::set(), ibis::SHORT, ibis::bitvector::size(), ibis::bitvector::sloppyCount(), ibis::TEXT, ibis::TYPESTRING, ibis::UBYTE, ibis::UDT, ibis::UINT, ibis::ULONG, ibis::UNKNOWN_TYPE, and ibis::USHORT.

virtual const ibis::dictionary* ibis::bord::column::getDictionary ( ) const
inlinevirtual

Return the dictionary associated with the column.

A dictionary is associated with the column originally stored as ibis::category, but has been converted to be an integer column of type ibis::UINT.

Reimplemented from ibis::column.

References dic.

Referenced by ibis::bord::backup(), ibis::bord::bord(), ibis::bord::copyColumn(), ibis::bord::cursor::cursor(), ibis::bord::evaluateTerms(), ibis::bord::getColumnAsOpaques(), ibis::bord::getColumnAsStrings(), ibis::jNatural::select(), ibis::jRange::select(), and ibis::bord::xgroupby().

ibis::fileManager::storage * ibis::bord::column::getRawData ( ) const
virtual

Retrieve the raw data buffer as an ibis::fileManager::storage.

Since this function exposes the internal storage representation, it should not be relied upon for general uses. This is mostly a convenience for FastBit internal development!

Note
Only fix-sized columns are stored using ibis::fileManager::storage objects. It will return a nil pointer for string-valued columns.

Reimplemented from ibis::column.

References ibis::DOUBLE, ibis::FLOAT, ibis::array_t< T >::getStorage(), ibis::INT, ibis::LONG, ibis::OID, ibis::SHORT, ibis::UBYTE, ibis::UINT, ibis::ULONG, and ibis::USHORT.

int ibis::bord::column::getString ( uint32_t  i,
std::string &  val 
) const
virtual

Return the string at the ith row.

If the raw data is not present, but a dictionary is present, then this function return the string value corresponding to the integer value i. Note that this fall-back option does not conform to the original intention of this function.

Reimplemented from ibis::column.

References ibis::CATEGORY, and ibis::TEXT.

int ibis::bord::column::getValuesArray ( void *  vals) const
virtual

Makes a copy of the in-memory data.

Uses a shallow copy for ibis::array_t objects, but a deap copy for the string values.

Reimplemented from ibis::column.

References ibis::CATEGORY, ibis::array_t< T >::copy(), ibis::DOUBLE, ibis::FLOAT, ibis::INT, ibis::LONG, ibis::array_t< T >::resize(), ibis::SHORT, ibis::TEXT, ibis::TYPESTRING, ibis::UBYTE, ibis::UINT, ibis::ULONG, and ibis::USHORT.

int ibis::bord::column::limit ( uint32_t  nr)

Reduce the number of rows stored in this column object to nr.

It does nothing if the current size is no more than nr.

It returns 0 upcon successful completion, -1 otherwise.

References ibis::CATEGORY, ibis::DOUBLE, ibis::FLOAT, ibis::INT, ibis::LONG, ibis::array_t< T >::nosharing(), ibis::array_t< T >::resize(), ibis::SHORT, ibis::array_t< T >::size(), ibis::TEXT, ibis::UBYTE, ibis::UINT, ibis::ULONG, and ibis::USHORT.

long ibis::bord::column::patternSearch ( const char *  pat) const
virtual

Compute an estimate of the maximum number of possible matches.

This is a trivial implementation that does not actually perform any meaningful checks. It simply returns the number of strings in memory as the estimate.

Reimplemented from ibis::column.

References ibis::CATEGORY, ibis::array_t< T >::size(), ibis::TEXT, and ibis::TYPESTRING.

int ibis::bord::column::restoreCategoriesAsStrings ( const ibis::category cat)

Convert the integer representation back to the string representation.

The existing data type must be ibis::UINT and the column with the same in in the given ibis::part prt must be of type ibis::CATEGORY.

References ibis::CATEGORY, ibis::category::getString(), ibis::array_t< T >::size(), and ibis::UINT.

ibis::array_t< signed char > * ibis::bord::column::selectBytes ( const ibis::bitvector mask) const
virtual

Retrieve selected 1-byte integer values.

Note that unsigned integers are simply treated as signed integers.

Note
The caller is responsible for freeing the returned array from any of the selectTypes functions.

Reimplemented from ibis::column.

References ibis::BIT, ibis::bitvector::cnt(), ibis::horometer::CPUTime(), ibis::bitvector::indexSet::indices(), ibis::bitvector::indexSet::isRange(), ibis::bitvector::indexSet::nIndices(), ibis::array_t< T >::push_back(), ibis::horometer::realTime(), ibis::array_t< T >::reserve(), ibis::array_t< T >::resize(), ibis::array_t< T >::size(), ibis::bitvector::size(), ibis::horometer::start(), ibis::horometer::stop(), and ibis::array_t< T >::swap().

Referenced by ibis::bord::backup().

ibis::array_t< double > * ibis::bord::column::selectDoubles ( const ibis::bitvector mask) const
virtual

Put the selected values into an array as doubles.

Note
Any column type could be selected as doubles. Other selectXXs function only work with data types that can be safely converted. This is the only function that allows one to convert to a different type. This is mainly to support aggregation operations involving arithmetic operation, however, it will truncate 64-bit integers to only 48-bit precision (because a double only has 48-bit mantisa).

Reimplemented from ibis::column.

References ibis::CATEGORY, ibis::bitvector::cnt(), ibis::horometer::CPUTime(), ibis::DOUBLE, ibis::FLOAT, ibis::bitvector::indexSet::indices(), ibis::INT, ibis::bitvector::indexSet::isRange(), ibis::LONG, ibis::bitvector::indexSet::nIndices(), ibis::horometer::realTime(), ibis::array_t< T >::resize(), ibis::SHORT, ibis::array_t< T >::size(), ibis::bitvector::size(), ibis::horometer::start(), ibis::horometer::stop(), ibis::array_t< T >::swap(), ibis::UBYTE, ibis::UINT, ibis::ULONG, and ibis::USHORT.

Referenced by ibis::bord::backup().

ibis::array_t< int32_t > * ibis::bord::column::selectInts ( const ibis::bitvector mask) const
virtual
ibis::array_t< int64_t > * ibis::bord::column::selectLongs ( const ibis::bitvector mask) const
virtual

Can be called on all integral types.

Note that 64-byte unsigned integers are simply treated as signed integer. This may cause the values to be interperted incorrectly. Shorter version of unsigned integers are treated correctly as positive values.

Reimplemented from ibis::column.

References ibis::CATEGORY, ibis::bitvector::cnt(), ibis::horometer::CPUTime(), ibis::bitvector::indexSet::indices(), ibis::INT, ibis::bitvector::indexSet::isRange(), ibis::LONG, ibis::bitvector::indexSet::nIndices(), ibis::horometer::realTime(), ibis::array_t< T >::resize(), ibis::SHORT, ibis::array_t< T >::size(), ibis::bitvector::size(), ibis::horometer::start(), ibis::horometer::stop(), ibis::array_t< T >::swap(), ibis::TEXT, ibis::UBYTE, ibis::UINT, and ibis::USHORT.

Referenced by ibis::bord::backup().

ibis::array_t< int16_t > * ibis::bord::column::selectShorts ( const ibis::bitvector mask) const
virtual

Return selected rows of the column in an array_t object.

Can convert all integers 2-byte or less in length. Note that unsigned integers are simply treated as signed integers. Shoter types of signed integers are treated correctly as positive values.

Note
The caller is responsible for freeing the returned array from any of the selectTypes functions.

Reimplemented from ibis::column.

References ibis::bitvector::cnt(), ibis::horometer::CPUTime(), ibis::bitvector::indexSet::indices(), ibis::bitvector::indexSet::isRange(), ibis::bitvector::indexSet::nIndices(), ibis::horometer::realTime(), ibis::array_t< T >::resize(), ibis::SHORT, ibis::array_t< T >::size(), ibis::bitvector::size(), ibis::horometer::start(), ibis::horometer::stop(), ibis::array_t< T >::swap(), and ibis::UBYTE.

Referenced by ibis::bord::backup().

std::vector< std::string > * ibis::bord::column::selectStrings ( const bitvector mask) const
virtual
ibis::array_t< unsigned char > * ibis::bord::column::selectUBytes ( const ibis::bitvector mask) const
virtual
ibis::array_t< uint32_t > * ibis::bord::column::selectUInts ( const ibis::bitvector mask) const
virtual
ibis::array_t< uint64_t > * ibis::bord::column::selectULongs ( const ibis::bitvector mask) const
virtual
ibis::array_t< uint16_t > * ibis::bord::column::selectUShorts ( const ibis::bitvector mask) const
virtual
int ibis::bord::column::setMeshShape ( uint64_t *  dims,
uint64_t  nd 
)

Specify the shape of the array.

The name is meant to suggest that the data was originally defined on a mesh.

long ibis::bord::column::stringSearch ( const char *  str,
ibis::bitvector hits 
) const
virtual

Locate the strings that match the given string.

The comaprison is case sensitive. If the incoming string is a nil pointer, it matches nothing.

Reimplemented from ibis::column.

References ibis::bitvector::adjustSize(), ibis::CATEGORY, ibis::bitvector::clear(), ibis::bitvector::cnt(), ibis::bitvector::set(), ibis::bitvector::setBit(), ibis::array_t< T >::size(), ibis::TEXT, ibis::TYPESTRING, and ibis::UINT.

long ibis::bord::column::stringSearch ( const char *  str) const
virtual

Compute an estimate of the maximum number of possible matches.

This is a trivial implementation that does not actually perform any meaningful checks. It simply returns the number of strings in memory as the estimate.

Reimplemented from ibis::column.

References ibis::CATEGORY, ibis::array_t< T >::size(), ibis::TEXT, ibis::TYPESTRING, and ibis::UINT.

long ibis::bord::column::stringSearch ( const std::vector< std::string > &  str) const
virtual

Compute an estimate of the maximum number of possible matches.

This is a trivial implementation that does not actually perform any meaningful checks. It simply returns the number of strings in memory as the estimate.

Reimplemented from ibis::column.

References ibis::CATEGORY, ibis::array_t< T >::size(), ibis::TEXT, ibis::TYPESTRING, and ibis::UINT.

Member Data Documentation

void* ibis::bord::column::buffer
protected

The in-memory storage.

A pointer to an array<T> or std::vector<std::string> depending on the data type.

See also
ibis::table::freeBuffer

Referenced by column(), equal_to(), and less_than().

const ibis::dictionary* ibis::bord::column::dic
protected

A dictionary.

It may be used with a column of type ibis::UINT or ibis::CATEGORY. Normally, it is a nil pointer.

Referenced by append(), getDictionary(), and setDictionary().

ibis::array_t<uint64_t> ibis::bord::column::shape
protected

Shape of the mesh for the data.

If it is empty, the data is assumed to be 1-Dimensional. If the shape array is provided, it is assumed that the data values are in the typical C order, where the 1st dimension is the slowest varying dimension and the last dimension is the fastest varying dimension. It is equivalent to member variable shapeSize in ibis::part.


The documentation for this class was generated from the following files:

Make It A Bit Faster
Contact us
Disclaimers
FastBit source code
FastBit mailing list archive