An in-memory data partition. More...
#include <bord110517.h>
Public Member Functions | |
| int | backup (const char *, const char *, const char *) const |
| Write the content of partition into the specified directory dir. | |
| void | describe (std::ostream &) const |
| int | dump (std::ostream &, uint32_t, const char *) const |
| Print the first nr rows of the data to the given output stream. | |
| int | dump (std::ostream &, uint32_t, uint32_t, const char *) const |
| Print nr rows starting with row offset. | |
| void | dumpNames (std::ostream &, const char *) const |
| ibis::table * | evaluateTerms (const ibis::selectClause &, const char *) const |
| ibis::table * | groupby (const ibis::selectClause &) const |
| The actual function to perform the group by operation. | |
| int | limit (uint32_t) |
| part (const char *tn, const char *td, uint64_t nr, const ibis::table::stringList &cn, const ibis::table::typeList &ct, ibis::table::bufferList &buf, const ibis::table::stringList *cdesc=0) | |
| Constructor. | |
| virtual long | reorder (const ibis::table::stringList &) |
Sort rows according the values of the columns specified in names. | |
| virtual long | reorder () |
| Sort rows with the lowest cardinality column first. | |
| long | reorderStrings (std::vector< std::string > &vals, const array_t< uint32_t > &ind) const |
| Reorder the vector of strings. | |
| template<typename T > | |
| long | reorderValues (array_t< T > &vals, const array_t< uint32_t > &ind) const |
| int | restoreCategoriesAsStrings (const ibis::part &, const char *) |
| Convert the integer representation to string representation. | |
| void | reverseRows () |
| long | sortStrings (std::vector< std::string > &vals, const array_t< uint32_t > &idxin, array_t< uint32_t > &idxout, array_t< uint32_t > &starts) const |
| Sort the string values. | |
| template<typename T > | |
| long | sortValues (array_t< T > &vals, const array_t< uint32_t > &indin, array_t< uint32_t > &indout, array_t< uint32_t > &starts) const |
Detailed Description
An in-memory data partition.
Constructor & Destructor Documentation
| ibis::bord::part::part | ( | const char * | tn, |
| const char * | td, | ||
| uint64_t | nr, | ||
| const ibis::table::stringList & | cn, | ||
| const ibis::table::typeList & | ct, | ||
| ibis::table::bufferList & | buf, | ||
| const ibis::table::stringList * | cdesc = 0 |
||
| ) |
Constructor.
This object can only store upto 4 billion rows because it uses a 32-bit unsigned integer to store the number of rows.
- Note:
- The new object will take ownership of the arrays listed in buf. All pointers in buf will be set to nil afterwards.
References ibis::part::amask, ibis::part::colorder, ibis::part::columns, ibis::table::freeBuffer(), ibis::gVerbose, ibis::part::m_desc, ibis::part::m_name, ibis::column::name(), ibis::part::nEvents, ibis::bitvector::set(), ibis::array_t< T >::size(), and ibis::util::strnewdup().
Member Function Documentation
| int ibis::bord::part::backup | ( | const char * | dir, |
| const char * | tname, | ||
| const char * | tdesc | ||
| ) | const |
Write the content of partition into the specified directory dir.
The directory dir must be writable. If the second and third arguments are valid, the output data will use them as the name and the description of the data partition.
References ibis::BYTE, ibis::CATEGORY, ibis::bitvector::clear(), ibis::bitvector::cnt(), ibis::part::columns, ibis::horometer::CPUTime(), ibis::DOUBLE, ibis::FLOAT, ibis::fileManager::flushDir(), ibis::gParameters(), ibis::gVerbose, ibis::part::idxstr, ibis::column::indexSpec(), ibis::fileManager::instance(), ibis::INT, ibis::LONG, ibis::part::m_desc, ibis::part::m_name, ibis::util::makeDir(), ibis::column::name(), ibis::part::nEvents, ibis::horometer::realTime(), ibis::bord::column::selectBytes(), ibis::bord::column::selectDoubles(), ibis::bord::column::selectFloats(), ibis::bord::column::selectInts(), ibis::bord::column::selectLongs(), ibis::bord::column::selectShorts(), ibis::bord::column::selectStrings(), ibis::bord::column::selectUBytes(), ibis::bord::column::selectUInts(), ibis::bord::column::selectULongs(), ibis::bord::column::selectUShorts(), ibis::bitvector::set(), ibis::SHORT, ibis::bitvector::size(), ibis::horometer::start(), ibis::horometer::stop(), ibis::TEXT, ibis::column::type(), ibis::TYPESTRING, ibis::UBYTE, ibis::UINT, ibis::ULONG, UnixOpen, ibis::USHORT, ibis::part::writeColumn(), and ibis::part::writeString().
| int ibis::bord::part::dump | ( | std::ostream & | out, |
| uint32_t | nr, | ||
| const char * | del | ||
| ) | const |
Print the first nr rows of the data to the given output stream.
The return values:
0 -- normal (successful) completion -1 -- no data in-memory -2 -- unknown data type -3 -- some columns not ibis::bord::column (not in-memory) -4 -- error in the output stream
References ibis::part::colorder, ibis::part::columns, ibis::bord::dump(), ibis::column::name(), and ibis::part::nEvents.
| int ibis::bord::part::dump | ( | std::ostream & | out, |
| uint32_t | offset, | ||
| uint32_t | nr, | ||
| const char * | del | ||
| ) | const |
Print nr rows starting with row offset.
Note that the row number starts with 0, i.e., the first row is row 0.
References ibis::part::colorder, ibis::part::columns, ibis::bord::dump(), ibis::column::name(), and ibis::part::nEvents.
| ibis::table * ibis::bord::part::groupby | ( | const ibis::selectClause & | sel | ) | const |
The actual function to perform the group by operation.
- Note:
- The input argument can only contain column names and support aggregation functions with column names arguments. Not futher arithmetic operations are allowed!
References ibis::BYTE, ibis::CATEGORY, ibis::array_t< T >::clear(), ibis::part::columns, ibis::bundle::create(), ibis::util::guardBase::dismiss(), ibis::DOUBLE, ibis::selectClause::empty(), ibis::FLOAT, ibis::table::freeBuffers(), ibis::selectClause::getAggregator(), ibis::gVerbose, ibis::INT, ibis::LONG, ibis::part::m_desc, ibis::part::m_name, ibis::part::nEvents, ibis::selectClause::print(), ibis::array_t< T >::push_back(), ibis::util::ref(), ibis::array_t< T >::reserve(), ibis::array_t< T >::resize(), ibis::SHORT, ibis::util::shortName(), ibis::selectClause::termName(), ibis::TEXT, ibis::TYPESTRING, ibis::UBYTE, ibis::UINT, ibis::ULONG, and ibis::USHORT.
| virtual long ibis::bord::part::reorder | ( | ) | [inline, virtual] |
Sort rows with the lowest cardinality column first.
The lowest cardinality column is ordered first. Only integer-valued columns are used in sorting. Returns the number of rows reordered when successful, otherwise return a negative number and the base data is corrupt!
- Warning:
- Danger: This function does not update null masks!
Reimplemented from ibis::part.
References ibis::util::reorder().
| long ibis::bord::part::reorderStrings | ( | std::vector< std::string > & | vals, |
| const array_t< uint32_t > & | ind | ||
| ) | const |
Reorder the vector of strings.
To avoid recreating the content of the string values, this function uses swap operations to move the existing strings into their new locations. It only works if ind is a proper permutation of integers between 0 and vals.size() (include 0 but exclude vals.size()), however, it does not check whether the input array is a proper permutation.
References ibis::gVerbose, ibis::part::nEvents, and ibis::array_t< T >::size().
| long ibis::bord::part::sortStrings | ( | std::vector< std::string > & | vals, |
| const array_t< uint32_t > & | idxin, | ||
| array_t< uint32_t > & | idxout, | ||
| array_t< uint32_t > & | starts | ||
| ) | const |
Sort the string values.
It preserves the previous order determined represented by idxin and starts.
References ibis::array_t< T >::clear(), ibis::array_t< T >::copy(), ibis::horometer::CPUTime(), ibis::gVerbose, ibis::part::name(), ibis::part::nEvents, ibis::array_t< T >::push_back(), ibis::horometer::realTime(), ibis::array_t< T >::resize(), ibis::array_t< T >::size(), ibis::bord::sortStrings(), ibis::horometer::start(), ibis::horometer::stop(), and ibis::array_t< T >::swap().
The documentation for this class was generated from the following files:
- bord110517.h
- bord110527.cpp
