array_t< T > Class Template Reference

Template array_t implements a replacement of std::vector. More...

#include <array_t.h>

List of all members.

Public Types

typedef const T * const_iterator
typedef T * iterator

Public Member Functions

 array_t (const char *fn, const off_t begin, const off_t end)
 Retrieve a portion of the named file to an array.
 array_t (const int fdes, const off_t begin, const off_t end)
 Read a portion of an open file into an array.
 array_t (ibis::fileManager::storage *rhs, const uint32_t start, const uint32_t nelm)
 Construct an array from a section of the raw storage.
 array_t (ibis::fileManager::storage &rhs)
 Turn a raw storage object into an array_t object.
 array_t (const array_t< T > &rhs, const uint32_t offset, const uint32_t nelm=0)
 A shallow copy constructor.
 array_t (const array_t< T > &rhs)
 Shallow copy. Should not throw any exception.
 array_t (uint32_t n, const T &val)
 Construct an array with n elements of value val.
 array_t (uint32_t n)
 Construct an array with n elements.
 array_t ()
 The default constructor. It constructs an empty array.
const T & back () const
T & back ()
const T * begin () const
T * begin ()
void bottomk (uint32_t k, array_t< uint32_t > &ind) const
 Return the positions of the k smallest elements.
uint32_t capacity () const
 The maximum number of elements can be stored with the current memory.
void clear ()
void copy (const array_t< T > &rhs)
 The copy function. It performs a shallow copy.
void deepCopy (const array_t< T > &rhs)
 The deep copy function. It makes an in-memory copy of rhs.
int empty () const
const T * end () const
T * end ()
iterator erase (iterator i, iterator j)
iterator erase (iterator p)
uint32_t find (const T &val) const
 Return the smallest i such that [i] >= val.
uint32_t find (const array_t< uint32_t > &ind, const T &val) const
 Return the smallest i such that [ind[i]] >= val.
uint32_t find_upper (const T &val) const
 Return the smallest i such that [i] > val.
const T & front () const
T & front ()
ibis::fileManager::storagegetStorage ()
 Export the actual storage object.
bool incore () const
 Is the content of the array solely in memory?
void insert (iterator p, const_iterator i, const_iterator j)
void insert (iterator p, uint32_t n, const T &val)
iterator insert (iterator pos, const T &val)
 Insert one value or a list of values before p.
void nosharing ()
 Make a not-shared copy of the array if it is actually a shared array.
const array_t< T > & operator= (const array_t< T > &rhs)
 Assignment operator. It performs a shallow copy.
T & operator[] (uint32_t i)
 Modifiable reference to an element of the array.
const T & operator[] (uint32_t i) const
 Non-modifiable reference to an element of the array.
void pop_back ()
 Reset the size to zero.
void printStatus (std::ostream &out) const
 Print internal pointer addresses.
void push_back (const T &elm)
 Add one element.
off_t read (const int fdes, const off_t begin, const off_t end)
 Read a portion of an open file.
void read (const char *)
 Read an array from the name file.
void reserve (uint32_t n)
 Reserve space.
void resize (uint32_t n)
 Remove the last element.
uint32_t size () const
void sort (array_t< uint32_t > &ind) const
 Produce index for ascending order.
void stableSort (array_t< uint32_t > &ind, array_t< T > &sorted) const
 A stable sort.
void stableSort (array_t< uint32_t > &ind) const
 A stable sort that does not modify the current array.
void stableSort (array_t< T > &tmp)
 A stable sort using the provided workspace.
void swap (array_t< T > &rhs)
 Exchange the content.
void topk (uint32_t k, array_t< uint32_t > &ind) const
 Return the positions of the k largest elements.
void write (FILE *fptr) const
 write whole array to an opened file
void write (const char *) const
 write whole array to the named file

Static Public Member Functions

static void stableSort (array_t< T > &val, array_t< uint32_t > &ind, array_t< T > &tmp, array_t< uint32_t > &itmp)
 This function sorts the content of array val.


Detailed Description

template<class T>
class array_t< T >

Template array_t implements a replacement of std::vector.

The main difference is that the underlying memory of this object is managed by ibis::fileManager. In addition, it also implements read and write functions that are not present in std::vector.


Constructor & Destructor Documentation

template<class T>
array_t< T >::array_t ( const array_t< T > &  rhs,
const uint32_t  offset,
const uint32_t  nelm = 0 
) [inline]

A shallow copy constructor.

It makes a new array out of a section of an existing array.

References ibis::fileManager::storage::beginUse(), ibis::gVerbose, and array_t< T >::m_end.

template<class T>
array_t< T >::array_t ( ibis::fileManager::storage rhs  )  [inline]

Turn a raw storage object into an array_t object.

The input storage object is used by the array. No new storage is allocated.

References ibis::fileManager::storage::beginUse().

template<class T>
array_t< T >::array_t ( ibis::fileManager::storage rhs,
const uint32_t  start,
const uint32_t  nelm 
) [inline]

Construct an array from a section of the raw storage.

The argument start is measured in number of bytes, the second argument nelm is the number of element of type T.

References ibis::fileManager::storage::begin(), ibis::fileManager::storage::beginUse(), ibis::fileManager::storage::end(), and ibis::gVerbose.

template<class T>
array_t< T >::array_t ( const int  fdes,
const off_t  begin,
const off_t  end 
) [inline]

Read a portion of an open file into an array.

Construct a new array by reading a part of a binary file.

The argument fdes must be a valid file descriptor (as defined in unistd.h). It attempt to read end - begin bytes from the file starting at offset begin.

References ibis::fileManager::storage::beginUse(), and ibis::gVerbose.

template<class T>
array_t< T >::array_t ( const char *  fn,
const off_t  begin,
const off_t  end 
) [inline]

Retrieve a portion of the named file to an array.

Prefer memory map if possible.

References ibis::fileManager::storage::beginUse(), and ibis::gVerbose.


Member Function Documentation

template<class T>
void array_t< T >::bottomk ( uint32_t  k,
array_t< uint32_t > &  ind 
) const [inline]

Return the positions of the k smallest elements.

Sort the first k elemnent of the array.

Return the indices of the smallest values in array ind.

Note:
The resulting array ind may have more than k elements if the kth smallest value is not a single value. The array ind may have less than k elements if this array has less than k elements.

References ibis::util::logger::buffer(), array_t< T >::resize(), and array_t< T >::size().

template<class T>
uint32_t array_t< T >::find ( const T &  val  )  const [inline]

Return the smallest i such that [i] >= val.

Find the first position where the value is no less than val.

Assuming the array is already sorted in ascending order, it returns the smallest i such that operator[](i) >= val.

References array_t< T >::size().

template<class T>
uint32_t array_t< T >::find ( const array_t< uint32_t > &  ind,
const T &  val 
) const [inline]

Return the smallest i such that [ind[i]] >= val.

Find the position of the first element that is no less than val.

Assuming ind was produced by the sort function, it returns the smallest i such that operator[](ind[i]) >= val.

References array_t< T >::size().

Referenced by ibis::query::countEqualPairs(), ibis::fuge::estimate(), ibis::bylt::estimateCost(), ibis::bylt::evaluate(), ibis::part::mapValues(), ibis::index::mapValues(), ibis::column::searchSortedICC(), ibis::column::searchSortedICD(), and ibis::part::vault::seek().

template<class T>
uint32_t array_t< T >::find_upper ( const T &  val  )  const [inline]

Return the smallest i such that [i] > val.

Find the first position where the value is greater than val.

Assuming the array is already sorted in ascending order, it returns the smallest i such that operator[](i) > val.

Note:
The word upper is used in the same sense as in the STL function std::upper_bound.

References array_t< T >::size().

Referenced by ibis::column::searchSortedICC().

template<class T>
ibis::fileManager::storage* array_t< T >::getStorage (  )  [inline]

Export the actual storage object.

Note:
Very dangarous. Likely to be removed soon. Don't rely on this function!

template<class T>
iterator array_t< T >::insert ( iterator  pos,
const T &  val 
)

template<class T>
void array_t< T >::nosharing (  )  [inline]

Make a not-shared copy of the array if it is actually a shared array.

Note:
This function makes a copy of the current content if the content is shared by two or more clients. This does not guarantee that it would not become shared later. The complete solution is to implement copy-on-write in all functions that modifies an array, but that may decrease performance of this class for rare cases of modifications.

References ibis::fileManager::storage::begin(), ibis::fileManager::storage::beginUse(), ibis::fileManager::storage::end(), ibis::fileManager::storage::endUse(), ibis::fileManager::storage::inUse(), and ibis::fileManager::storage::isFileMap().

Referenced by array_t< T >::reserve(), and array_t< T >::resize().

template<class T>
T& array_t< T >::operator[] ( uint32_t  i  )  [inline]

Modifiable reference to an element of the array.

Note:
For efficiency reasons, this is not a copy-on-write implementation! The caller has to call the function nosharing to make sure the underlying data is not shared with others.

template<class T>
void array_t< T >::push_back ( const T &  elm  )  [inline]

template<class T>
off_t array_t< T >::read ( const int  fdes,
const off_t  begin,
const off_t  end 
) [inline]

Read a portion of an open file.

Read an array from a file already open.

References ibis::fileManager::storage::begin(), ibis::gVerbose, and ibis::fileManager::storage::read().

template<class T>
void array_t< T >::reserve ( uint32_t  n  )  [inline]

template<class T>
void array_t< T >::resize ( uint32_t  n  )  [inline]

Remove the last element.

Change the size of the array so it has no less than
elements.

Resize array.

References ibis::fileManager::storage::begin(), ibis::fileManager::storage::beginUse(), ibis::fileManager::storage::end(), ibis::fileManager::storage::enlarge(), ibis::gVerbose, and array_t< T >::nosharing().

Referenced by ibis::part::adaptive2DBins(), ibis::part::adaptive3DBins(), ibis::part::adaptiveInts(), ibis::part::adaptiveIntsDetailed(), ibis::index::addBins(), ibis::tafel::append(), ibis::relic::append(), ibis::bin::append(), ibis::category::append(), ibis::bin::binning(), ibis::bin::binningT(), array_t< T >::bottomk(), ibis::part::calculate(), ibis::part::coarsenBins(), ibis::ambit::construct(), ibis::relic::construct(), ibis::range::construct(), ibis::mesa::construct(), ibis::egale::construct(), ibis::bin::construct(), ibis::bitvector64::decompress(), ibis::bitvector::decompress(), ibis::index::divideCounts(), ibis::part::fill1DBinsWeighted(), ibis::part::fill2DBinsWeighted(), ibis::part::fill3DBinsWeighted(), ibis::part::get2DDistributionI(), ibis::part::getJointDistribution(), ibis::roster::mergeBlock2(), ibis::tafel::normalize(), ibis::part::old2DDistribution(), ibis::part::quickTest(), ibis::bundle::readRIDs(), ibis::relic::relic(), ibis::part::reorderValues(), ibis::bundle::rowCounts(), ibis::column::selectBytes(), ibis::column::selectDoubles(), ibis::bord::column::selectDoubles(), ibis::column::selectFloats(), ibis::bord::column::selectFloats(), ibis::column::selectInts(), ibis::bord::column::selectInts(), ibis::column::selectLongs(), ibis::bord::column::selectLongs(), ibis::column::selectShorts(), ibis::column::selectUBytes(), ibis::column::selectUInts(), ibis::bord::column::selectUInts(), ibis::column::selectULongs(), ibis::column::selectUShorts(), ibis::column::selectValuesT(), ibis::index::setBases(), array_t< T >::sort(), ibis::util::sortMerge(), array_t< T >::stableSort(), ibis::index::sumBins(), array_t< T >::topk(), ibis::bundles::truncate(), ibis::bundle1::truncate(), ibis::bitvector64::write(), and ibis::bitvector::write().

template<class T>
uint32_t array_t< T >::size (  )  const [inline]

< Return the number of elements.

Referenced by ibis::index::activate(), ibis::column::actualMinMax(), ibis::part::adaptive2DBins(), ibis::part::adaptive3DBins(), ibis::part::adaptiveFloats(), ibis::part::adaptiveFloatsDetailed(), ibis::part::adaptiveInts(), ibis::part::adaptiveIntsDetailed(), ibis::index::addBins(), ibis::part::addColumn(), ibis::mensa::addIncoreData(), ibis::bitvector::all0s(), ibis::bitvector::all1s(), ibis::tafel::append(), ibis::ridHandler::append(), ibis::sbiad::append(), ibis::sapid::append(), ibis::relic::append(), ibis::bin::append(), ibis::category::append(), ibis::column::appendValues(), ibis::zone::binBoundaries(), ibis::pack::binBoundaries(), ibis::ambit::binBoundaries(), ibis::relic::binBoundaries(), ibis::bak2::binBoundaries(), ibis::egale::binBoundaries(), ibis::bin::binning(), ibis::bin::binningT(), ibis::bin::binOrderT(), ibis::slice::binWeights(), ibis::relic::binWeights(), ibis::fade::binWeights(), ibis::bitvector::bitvector(), array_t< T >::bottomk(), ibis::part::calculate(), ibis::part::coarsenBins(), ibis::bitvector64::compress(), ibis::column::computeMax(), ibis::column::computeMin(), ibis::bord::column::computeMinMax(), ibis::column::computeSum(), ibis::ambit::construct(), ibis::relic::construct(), ibis::range::construct(), ibis::egale::construct(), ibis::bin::convertGranules(), ibis::part::count2DBins(), ibis::part::count2DWeights(), ibis::part::count3DBins(), ibis::part::count3DWeights(), ibis::query::countDeltaPairs(), ibis::query::countEqualPairs(), ibis::bitvector64::decompress(), ibis::bitvector::decompress(), array_t< T >::deepCopy(), ibis::index::divideCounts(), ibis::part::doCompare(), ibis::part::doCompare0(), ibis::part::doCount(), ibis::part::doScan(), ibis::fuge::estimate(), ibis::slice::estimate(), ibis::entre::estimate(), ibis::bylt::estimateCost(), ibis::slice::estimateCost(), ibis::relic::estimateCost(), ibis::keywords::estimateCost(), ibis::direkte::estimateCost(), ibis::bin::estimateCost(), ibis::query::evaluate(), ibis::zona::evaluate(), ibis::bylt::evaluate(), ibis::slice::evaluate(), ibis::sbiad::evaluate(), ibis::sapid::evaluate(), ibis::fade::evaluate(), ibis::part::evaluateRIDSet(), ibis::range::expandRange(), ibis::bak2::expandRange(), ibis::bak::expandRange(), ibis::bin::expandRange(), ibis::part::fill1DBins(), ibis::part::fill1DBinsWeighted(), ibis::part::fill2DBins(), ibis::part::fill2DBinsWeighted(), ibis::part::fill3DBins(), ibis::part::fill3DBinsWeighted(), array_t< T >::find(), array_t< T >::find_upper(), ibis::part::get1DBins_(), ibis::part::get1DDistribution(), ibis::part::get2DDistributionI(), ibis::bord::getColumnAsBytes(), ibis::mensa::getColumnAsDoubles(), ibis::bord::getColumnAsDoubles(), ibis::mensa::getColumnAsFloats(), ibis::bord::getColumnAsFloats(), ibis::mensa::getColumnAsInts(), ibis::bord::getColumnAsInts(), ibis::mensa::getColumnAsLongs(), ibis::bord::getColumnAsLongs(), ibis::mensa::getColumnAsShorts(), ibis::bord::getColumnAsShorts(), ibis::mensa::getColumnAsStrings(), ibis::bord::getColumnAsStrings(), ibis::bord::getColumnAsUBytes(), ibis::mensa::getColumnAsUInts(), ibis::bord::getColumnAsUInts(), ibis::mensa::getColumnAsULongs(), ibis::bord::getColumnAsULongs(), ibis::mensa::getColumnAsUShorts(), ibis::bord::getColumnAsUShorts(), ibis::part::getCumulativeDistribution(), ibis::part::getDistribution(), ibis::fileManager::getFile(), ibis::part::getJointDistribution(), ibis::query::getQualifiedBytes(), ibis::query::getQualifiedDoubles(), ibis::query::getQualifiedFloats(), ibis::query::getQualifiedInts(), ibis::query::getQualifiedLongs(), ibis::query::getQualifiedShorts(), ibis::query::getQualifiedUBytes(), ibis::query::getQualifiedUInts(), ibis::query::getQualifiedULongs(), ibis::query::getQualifiedUShorts(), ibis::query::getRIDs(), ibis::part::getRIDs(), ibis::query::getRIDsInBundle(), ibis::pack::getSum(), ibis::ambit::getSum(), ibis::slice::getSum(), ibis::relic::getSum(), ibis::range::getSum(), ibis::mesa::getSum(), ibis::fade::getSum(), ibis::moins::getSum(), ibis::entre::getSum(), ibis::egale::getSum(), ibis::part::loadIndexes(), ibis::relic::locate(), ibis::bak2::locate(), ibis::bak::locate(), ibis::bin::locate(), ibis::part::mapValues(), ibis::index::mapValues(), ibis::bak2::mapValues(), ibis::roster::mergeBlock2(), ibis::part::negativeCompare(), ibis::tafel::normalize(), ibis::part::old2DDistribution(), ibis::roster::operator[](), ibis::zona::print(), ibis::fuzz::print(), ibis::fuge::print(), ibis::bylt::print(), ibis::slice::print(), ibis::sbiad::print(), ibis::sapid::print(), ibis::relic::print(), ibis::fade::print(), ibis::egale::print(), ibis::bundles::printAll(), ibis::bundle1::printAll(), ibis::query::printRIDs(), ibis::qDiscreteRange::qDiscreteRange(), ibis::part::quickTest(), ibis::ridHandler::read(), ibis::fuge::read(), ibis::bylt::read(), ibis::query::readRIDs(), ibis::relic::relic(), ibis::util::reorder(), ibis::part::reorder(), ibis::bundles::reorder(), ibis::bord::part::reorder(), ibis::part::reorderValues(), ibis::roster::roster(), ibis::bundle::rowCounts(), ibis::bin::scanAndPartition(), ibis::column::searchSortedICC(), ibis::column::searchSortedICD(), ibis::colDoubles::segment(), ibis::colFloats::segment(), ibis::colULongs::segment(), ibis::colLongs::segment(), ibis::colUInts::segment(), ibis::colInts::segment(), ibis::joinIN::select(), ibis::column::selectBytes(), ibis::column::selectDoubles(), ibis::bord::column::selectDoubles(), ibis::column::selectFloats(), ibis::bord::column::selectFloats(), ibis::column::selectInts(), ibis::bord::column::selectInts(), ibis::column::selectLongs(), ibis::text::selectLongs(), ibis::bord::column::selectLongs(), ibis::column::selectShorts(), ibis::text::selectStrings(), ibis::column::selectUBytes(), ibis::column::selectUInts(), ibis::bord::column::selectUInts(), ibis::column::selectULongs(), ibis::column::selectUShorts(), ibis::column::selectValuesT(), ibis::query::setRIDs(), array_t< T >::sort(), ibis::util::sort_heap(), ibis::util::sort_insertion(), ibis::util::sort_partition(), ibis::util::sort_partition3(), ibis::util::sort_quick(), ibis::util::sort_quick3(), ibis::util::sort_radix(), ibis::util::sort_shell(), ibis::util::sortAll(), ibis::util::sortAll_shell(), ibis::util::sortAll_split(), ibis::query::sortEquiJoin(), ibis::util::sortKeys(), ibis::util::sortMerge(), ibis::query::sortRangeJoin(), ibis::util::sortRIDs(), ibis::util::sortStrings(), ibis::util::sortStrings_partition(), ibis::util::sortStrings_quick(), ibis::util::sortStrings_shell(), array_t< T >::stableSort(), ibis::column::string2int(), ibis::index::sumBins(), array_t< T >::topk(), ibis::bundles::truncate(), ibis::ridHandler::write(), ibis::zona::write(), ibis::fuzz::write(), ibis::bylt::write(), ibis::sbiad::write(), ibis::sapid::write(), ibis::roster::write(), ibis::relic::write(), ibis::fade::write(), ibis::roster::writeSorted(), and ibis::part::writeValues().

template<class T>
void array_t< T >::sort ( array_t< uint32_t > &  ind  )  const [inline]

Produce index for ascending order.

Sort the array to produce ind so that array_t[ind[i]] is in ascending order.

Uses the quicksort algorithm with introspection.

References ibis::util::logger::buffer(), array_t< T >::clear(), array_t< T >::resize(), and array_t< T >::size().

Referenced by ibis::bord::part::reorder(), ibis::part::reorderValues(), ibis::colDoubles::sort(), ibis::colFloats::sort(), ibis::colULongs::sort(), ibis::colLongs::sort(), ibis::colUInts::sort(), and ibis::colInts::sort().

template<class T>
void array_t< T >::stableSort ( array_t< T > &  val,
array_t< uint32_t > &  ind,
array_t< T > &  tmp,
array_t< uint32_t > &  itmp 
) [inline, static]

This function sorts the content of array val.

The values of ind[i] will be reordered in the same way as the array val. The two other arrays are used as temporary storage.

Note:
On input, if array ind has the same size as array val, the content of array ind will be directly used. Otherwise, array ind will be initialized to be consecutive integers starting from 0.

If the input array val has less than two elements, this function does nothing, i.e., does not change any of the four arguments.

References array_t< T >::resize(), array_t< T >::size(), and array_t< T >::swap().

template<class T>
void array_t< T >::stableSort ( array_t< uint32_t > &  ind,
array_t< T > &  sorted 
) const [inline]

A stable sort.

The content of array ind will be simply moved together with @ this array if it is the same size as this array.

It does not change this array, but produces a sorted version in sorted.

Otherwise, it is initialized to consecutive integers starting from 0 before the actual sorting is performed.

References array_t< T >::clear(), array_t< T >::resize(), array_t< T >::size(), and array_t< T >::stableSort().

template<class T>
void array_t< T >::stableSort ( array_t< uint32_t > &  ind  )  const [inline]

A stable sort that does not modify the current array.

Use merge sort algorithm to produce an index array so that array[ind[i]] would in ascending order.

It uses two additional arrays for temporary storage.

References array_t< T >::clear(), array_t< T >::deepCopy(), array_t< T >::resize(), array_t< T >::size(), and array_t< T >::stableSort().

template<class T>
void array_t< T >::stableSort ( array_t< T > &  tmp  )  [inline]

A stable sort using the provided workspace.

Merge sort algorithm.

The current content is modified to be in ascending order. The argument tmp is only used as temporary storage.

This array is sorted. The argument tmp is only used as temporary storage.

References array_t< T >::resize(), array_t< T >::size(), and array_t< T >::swap().

Referenced by ibis::part::reorder(), ibis::query::sortEquiJoin(), ibis::query::sortRangeJoin(), and array_t< T >::stableSort().

template<class T>
void array_t< T >::swap ( array_t< T > &  rhs  )  [inline]

template<class T>
void array_t< T >::topk ( uint32_t  k,
array_t< uint32_t > &  ind 
) const [inline]

Return the positions of the k largest elements.

Sort the k largest elements of the array.

Return the indices of the in sorted values.

Note:
The resulting array ind may have more than k elements if the kth smallest value is not a single value. The array ind may have less than k elements if this array has less than k elements.

The values are sorted in ascending order, i.e., [ind[i]] <= [ind[i+1]]. This is done so that all sorting routines produce indices in the same ascending order. It should be easy to reverse the order the indices since it only contains the largest values.

References ibis::util::logger::buffer(), array_t< T >::resize(), and array_t< T >::size().

template<class T>
void array_t< T >::write ( FILE *  fptr  )  const [inline]

write whole array to an opened file

Write the content of the array to a file already opened.

References ibis::gVerbose.

template<class T>
void array_t< T >::write ( const char *  file  )  const [inline]

write whole array to the named file

Write the content of array to the named file.

References ibis::gVerbose.

Referenced by ibis::query::writeRIDs().


The documentation for this class was generated from the following files:
Make It A Bit Faster
Disclaimers
FastBit source code
FastBit mailing list archive
Maintainer of this page