LcsClusterReader Class Reference

Reads blocks from a single cluster. More...

#include <LcsClusterReader.h>

Inheritance diagram for LcsClusterReader:

LcsClusterAccessBase SegPageEntryIterSource< LcsRid > LcsClusterVerifier List of all members.

Public Member Functions

 LcsClusterReader (BTreeDescriptor const &treeDescriptor, CircularBuffer< LcsRidRun > *pRidRuns=NULL)
 Constructor.
virtual ~LcsClusterReader ()
void setRootPageId (PageId rootPageId)
 Sets the root pageId of the underlying btree corresponding to the cluster.
void initColumnReaders (uint nClusterColsInit, TupleProjection const &clusterProj)
 Initializes cluster reader with column readers.
void open ()
 Initializes state variables used by cluster reader.
void close ()
 Performs shutdown on cluster reader.
bool getFirstClusterPageForRead (PConstLcsClusterNode &pBlock)
 Gets first page in a cluster.
bool getNextClusterPageForRead (PConstLcsClusterNode &pBlock)
 Gets next page in a cluster, based on current position in btree.
bool isPositioned () const
 Returns true if positioned within some range in a batch.
LcsRid getRangeStartRid () const
 Returns first rid in a range.
LcsRid getRangeEndRid () const
 Returns first rid after the end of the current range.
uint getRangeSize () const
 Returns number of rids in the range.
uint getRangePos () const
 Returns offset within the current range.
LcsRid getCurrentRid () const
 Returns rid currently positioned at.
uint getRangeRowsLeft () const
 Returns number of rids yet to be read.
bool position (LcsRid rid)
 Positions scan on the rid, moving to a new range if necessary.
bool nextRange ()
 Positions scan on the first rid of the next range.
bool advance (uint nRids)
 Moves "nRids" forward in the current range.
void advanceWithinBatch (uint nRids)
 Advances nRids forward in the current batch.
virtual PageId getNextPageForPrefetch (LcsRid &rid, bool &found)
 Determines the next page to be pre-fetched based on the rids that need to be read.
void catchUp (uint parentBufPos, LcsRid parentNextRid)
 Resynchronizes the cluster reader's prefetch position to that of the parent scan.
RecordNum getNumRows ()
 
Returns:
the number of rows in the cluster

uint getNumClusterCols ()
 Returns number of columns in cluster.
void setNumClusterCols (uint nCols)
 Sets number of columns in cluster.
void unlockClusterPage ()
 Unlocks cluster page.
virtual void initPrefetchEntry (LcsRid &entry)
 Initializes a specific entry in the pre-fetch queue.

Static Public Member Functions

static LcsRid getFetchRids (CircularBufferIter< LcsRidRun > &ridRunIter, LcsRid &nextRid, bool remove)
 Retrieves the current and next rids that need to be read, based on the current rid run being processed.

Public Attributes

uint nColsToRead
 Number of cluster columns that will be read.
boost::scoped_array< LcsColumnReaderclusterCols
 Column readers for each cluster column that will be read.

Protected Member Functions

LcsRid readRid ()
 Returns RID from btree tuple.
PageId readClusterPageId ()
 Returns cluster pageid from btree tuple.
void setHdrOffsets (PConstLcsClusterNode pHdr)
 Sets pointers to offset arrays in cluster page header.

Protected Attributes

TupleData bTreeTupleData
 Tuple data representing the btree key corresponding to the cluster page.
SegmentAccessor segmentAccessor
 Accessor for segment storing both btree and cluster pages.
ClusterPageLock clusterLock
 Buffer lock for the actual cluster node pages.
PageId clusterPageId
 Current cluster pageid.
LcsRid bTreeRid
 Current rid in btree used to access current cluster page.
uint nClusterCols
 Number of columns in cluster.
uint16_tlastVal
 Offsets to the last value stored on the page for each column in cluster.
uint16_tfirstVal
 Offsets to the first value stored on the page for each column in cluster.
uintnVal
 Number of distinct values in the page for each column in cluster.
uint16_tdelta
 For each column in the cluster, offset used to get the real offset within the page.

Private Member Functions

void moveToBlock (PageId clusterPageId)
 Reads a cluster block and sets up necessary structures to navigate within the page.
bool moveToBlockWithRid (LcsRid rid)
 Reads a cluster block and sets up necessary structures to navigate within the page corresponding to a rid.
bool positionInBlock (LcsRid rid)
 Finds the range in the current block which contains "Rid".
void positionInRange (uint pos)
 Positions on "pos" in the current range.
void setUpBlock ()
 Updates class status to reflect page just read and sets header values to point within the current page.
LcsClusterNode const & readClusterPage ()
 Reads a cluster page based on current btree position.
bool searchForRid (LcsRid rid)
 Reads btree and locates rid in tree.
void getNextBTreeEntry ()
 Reads the next entry in the rid-to-pageId btree map.

Private Attributes

SharedBTreeReader bTreeReader
 Reads btree corresponding to cluster.
PLcsBatchDir pRangeBatches
 Pointer to first batch in the current range.
LcsRid rangeStartRid
 First rid in the current range.
LcsRid rangeEndRid
 1 rid past the last rid in the current range
uint nRangePos
 Offset from start of current range.
PBuffer pLeaf
 Pointer to current cluster block.
PConstLcsClusterNode pLHdr
 Pointer to header of cluster block.
PLcsBatchDir pBatches
 Batch directory.
bool noPrefetch
 If true, do not use pre-fetch.
CircularBufferIter< LcsRidRunridRunIter
 Iterator over the circular buffer containing rid runs.
SegPageEntryIter< LcsRid > prefetchQueue
 Pre-fetch queue for pages read from this cluster.
LcsRid nextRid
 The next rid that needs to be read.
std::pair< PageId, LcsRid > nextBTreeEntry
 The next entry in the rid-to-pageId btree map following the one that contains the last page that was pre-fetched.
std::pair< PageId, LcsRid > nextPrefetchEntry
 The next entry from the pre-fetch queue.
bool dumbPrefetch
 If true, switch has been made to dumb pre-fetch.
LcsRid currRid
 The current rid that needs to be read.

Friends

class LcsColumnReader

Detailed Description

Reads blocks from a single cluster.

A column reader object (LcsColumnReader) is used to read the value of a particular column at the current rowid of the LcsClusterReader.

A cluster holds the values for one or more columns. The values of these columns are held in rowid ranges. Each range contains one batch (LcsBatchDir) for each column; this batch holds the values of that column for each rowid in the range. A batch may be "fixed", "compressed", or "variable".

Suppose a table has 2 column store clusters: one containing columns A, B, and C, and another containing columns D and E. To read the values of columns A and C, create a LcsClusterReader on the first cluster, and two LcsColumnReader onto this LcsClusterReader, one for each column, A and C.

Here's how to use a LcsClusterReader (scan) and LcsColumnReader (colScan) for doing a full table scan:

scan.init for i in 0..nClusterCols colScan[i].init scan.position(first rid) do for i in 0..nClusterCols colScan[i].sync do for i in 0..nClusterCols colScan[i].getCurrentValue while scan.advance(1) while scan.nextRange != false scan.close()

Definition at line 84 of file LcsClusterReader.h.


Constructor & Destructor Documentation

LcsClusterReader::LcsClusterReader ( BTreeDescriptor const &  treeDescriptor,
CircularBuffer< LcsRidRun > *  pRidRuns = NULL 
) [explicit]

Constructor.

Parameters:
treeDescriptor of btree corresponding to cluster
pRidRuns pointer to circular buffer of rid runs; defaults to NULL, in which case, pre-fetches are disabled

Definition at line 31 of file LcsClusterReader.cpp.

References bTreeReader, noPrefetch, prefetchQueue, and SegPageEntryIter< EntryT >::setPrefetchSource().

00034 :
00035     LcsClusterAccessBase(treeDescriptor),
00036     SegPageEntryIterSource<LcsRid>(),
00037     ridRunIter(pRidRuns),
00038     prefetchQueue(4000)
00039 {
00040     bTreeReader = SharedBTreeReader(new BTreeReader(treeDescriptor));
00041     if (pRidRuns == NULL) {
00042         noPrefetch = true;
00043     } else {
00044         noPrefetch = false;
00045         prefetchQueue.setPrefetchSource(*this);
00046     }
00047 }

LcsClusterReader::~LcsClusterReader (  )  [virtual]

Definition at line 49 of file LcsClusterReader.cpp.

00050 {
00051 }


Member Function Documentation

void LcsClusterReader::moveToBlock ( PageId  clusterPageId  )  [private]

Reads a cluster block and sets up necessary structures to navigate within the page.

Parameters:
clusterPageId pageId of the cluster page to read

Definition at line 197 of file LcsClusterReader.cpp.

References LcsClusterAccessBase::clusterLock, SegNodeLock< Node >::getNodeForRead(), SegPageLock::lockShared(), pLHdr, and setUpBlock().

Referenced by position().

00198 {
00199     // read the desired cluster page and initialize structures to reflect
00200     // page read
00201 
00202     clusterLock.lockShared(clusterPageId);
00203     LcsClusterNode const &page = clusterLock.getNodeForRead();
00204     pLHdr = &page;
00205     setUpBlock();
00206 }

bool LcsClusterReader::moveToBlockWithRid ( LcsRid  rid  )  [private]

Reads a cluster block and sets up necessary structures to navigate within the page corresponding to a rid.

Parameters:
rid the rid that determines the page that needs to be read
Returns:
true if a cluster block was successfully located

Definition at line 208 of file LcsClusterReader.cpp.

References LcsClusterAccessBase::clusterLock, LcsClusterAccessBase::clusterPageId, SegNodeLock< Node >::getNodeForRead(), SegPageLock::lockShared(), MAXU, nextPrefetchEntry, NULL_PAGE_ID, pLHdr, prefetchQueue, and setUpBlock().

Referenced by position().

00209 {
00210     PageId clusterPageId;
00211 
00212     // Read entries from the pre-fetch queue until we either find the entry
00213     // within the range of our desired rid, or we exhaust the contents of
00214     // the queue.
00215     for (;;) {
00216         std::pair<PageId, LcsRid> &prefetchEntry =
00217             (nextPrefetchEntry.second == LcsRid(MAXU)) ?
00218             *prefetchQueue : nextPrefetchEntry;
00219 
00220         clusterPageId = prefetchEntry.first;
00221         if (clusterPageId == NULL_PAGE_ID) {
00222             return false;
00223         }
00224 
00225         LcsRid prefetchRid = prefetchEntry.second;
00226         assert(prefetchRid <= rid);
00227 
00228         // Make sure this is the correct entry by checking that the rid
00229         // is smaller than the next entry.  We can end up with non-matching
00230         // entries if dumb pre-fetches are being used.
00231         ++prefetchQueue;
00232         nextPrefetchEntry = *prefetchQueue;
00233         if (nextPrefetchEntry.first == NULL_PAGE_ID ||
00234             rid < nextPrefetchEntry.second)
00235         {
00236             break;
00237         } else {
00238             continue;
00239         }
00240     }
00241 
00242     // Now that we've located the desired page, read it.
00243     clusterLock.lockShared(clusterPageId);
00244     LcsClusterNode const &page = clusterLock.getNodeForRead();
00245     pLHdr = &page;
00246     setUpBlock();
00247     return true;
00248 }

bool LcsClusterReader::positionInBlock ( LcsRid  rid  )  [private]

Finds the range in the current block which contains "Rid".

Does not search backwards.

Parameters:
rid rid to be located
Returns:
false if "Rid" is beyond the end of the block

Definition at line 250 of file LcsClusterReader.cpp.

References getRangeEndRid(), LcsClusterAccessBase::nClusterCols, LcsBatchDir::nRow, opaqueToInt(), positionInRange(), pRangeBatches, rangeEndRid, and rangeStartRid.

Referenced by position().

00251 {
00252     // Go forward through the ranges in the current block until we find the
00253     // right one, or until we hit the end of the block
00254     while (rid >= getRangeEndRid()
00255            && pRangeBatches + nClusterCols < pBatches + pLHdr->nBatch) {
00256         rangeStartRid += pRangeBatches->nRow;
00257 
00258         pRangeBatches += nClusterCols;            // go to start of next range
00259 
00260         // set end rowid based on already available info (for performance
00261         // reasons)
00262         rangeEndRid = rangeStartRid + pRangeBatches->nRow;
00263     }
00264 
00265     // Try to position within current batch
00266     if (rid < getRangeEndRid()) {
00267         assert(rid >= rangeStartRid);
00268         positionInRange(opaqueToInt(rid) - opaqueToInt(rangeStartRid));
00269         return true;
00270     } else {
00271         return false;
00272     }
00273 }

void LcsClusterReader::positionInRange ( uint  pos  )  [inline, private]

Positions on "pos" in the current range.

Parameters:
pos desired position

Definition at line 203 of file LcsClusterReader.h.

Referenced by positionInBlock().

00204     {
00205         nRangePos = pos;
00206     }

void LcsClusterReader::setUpBlock (  )  [private]

Updates class status to reflect page just read and sets header values to point within the current page.

Definition at line 275 of file LcsClusterReader.cpp.

References nRangePos, LcsBatchDir::nRow, pBatches, pLeaf, pLHdr, pRangeBatches, rangeEndRid, rangeStartRid, and LcsClusterAccessBase::setHdrOffsets().

Referenced by moveToBlock(), and moveToBlockWithRid().

00276 {
00277     // setup pointers to lastVal, firstVal arrays
00278     pLeaf = (PBuffer) pLHdr;
00279     setHdrOffsets(pLHdr);
00280 
00281     assert(pLHdr->nBatch > 0);
00282 
00283     pBatches = (PLcsBatchDir) (pLeaf + pLHdr->oBatch);
00284     rangeStartRid = pLHdr->firstRID;
00285 
00286     // at first range in block
00287     pRangeBatches = pBatches;
00288     // at first rid in range
00289     nRangePos = 0;
00290 
00291     // set end rowid based on already available info (for performance reasons)
00292     rangeEndRid = rangeStartRid + pRangeBatches->nRow;
00293 }

LcsClusterNode const & LcsClusterReader::readClusterPage (  )  [private]

Reads a cluster page based on current btree position.

Returns:
cluster page read

Definition at line 58 of file LcsClusterReader.cpp.

References bTreeReader, LcsClusterAccessBase::bTreeRid, LcsClusterAccessBase::bTreeTupleData, LcsClusterAccessBase::clusterLock, LcsClusterAccessBase::clusterPageId, LcsClusterNode::firstRID, SegNodeLock< Node >::getNodeForRead(), SegPageLock::lockShared(), LcsClusterAccessBase::readClusterPageId(), and LcsClusterAccessBase::readRid().

Referenced by getFirstClusterPageForRead(), getNextClusterPageForRead(), and getNumRows().

00059 {
00060     bTreeReader->getTupleAccessorForRead().unmarshal(bTreeTupleData);
00061     clusterPageId = readClusterPageId();
00062     // REVIEW jvs 27-Dec-2005:  What is bTreeRid used for?  Should probably
00063     // assert that it matches node.firstRID
00064     bTreeRid = readRid();
00065     clusterLock.lockShared(clusterPageId);
00066     LcsClusterNode const &node = clusterLock.getNodeForRead();
00067     assert(bTreeRid == node.firstRID);
00068     return node;
00069 }

bool LcsClusterReader::searchForRid ( LcsRid  rid  )  [private]

Reads btree and locates rid in tree.

Returns:
false if rid is not within the range of the tree; e.g., if it's empty

Definition at line 181 of file LcsClusterReader.cpp.

References bTreeReader, LcsClusterAccessBase::bTreeTupleData, DUP_SEEK_BEGIN, and LcsClusterAccessBase::readRid().

Referenced by getNextPageForPrefetch(), and position().

00182 {
00183     bTreeTupleData[0].pData = (PConstBuffer) &rid;
00184     // position on greatest lower bound of key
00185     bTreeReader->searchForKey(
00186         bTreeTupleData, DUP_SEEK_BEGIN, false);
00187     if (bTreeReader->isSingular()) {
00188         return false;
00189     }
00190     bTreeReader->getTupleAccessorForRead().unmarshal(bTreeTupleData);
00191 
00192     LcsRid key = readRid();
00193     assert(key <= rid);
00194     return true;
00195 }

void LcsClusterReader::getNextBTreeEntry (  )  [private]

Reads the next entry in the rid-to-pageId btree map.

Definition at line 377 of file LcsClusterReader.cpp.

References bTreeReader, LcsClusterAccessBase::bTreeTupleData, nextBTreeEntry, NULL_PAGE_ID, LcsClusterAccessBase::readClusterPageId(), and LcsClusterAccessBase::readRid().

Referenced by getNextPageForPrefetch().

00378 {
00379     bool rc = bTreeReader->searchNext();
00380     if (!rc) {
00381         // Indicate that there are no more pages
00382         nextBTreeEntry.first = NULL_PAGE_ID;
00383     } else {
00384         bTreeReader->getTupleAccessorForRead().unmarshal(bTreeTupleData);
00385         nextBTreeEntry.first = readClusterPageId();
00386         nextBTreeEntry.second = readRid();
00387     }
00388 }

void LcsClusterReader::setRootPageId ( PageId  rootPageId  ) 

Sets the root pageId of the underlying btree corresponding to the cluster.

Parameters:
rootPageId the root pageId that's set

Definition at line 53 of file LcsClusterReader.cpp.

References bTreeReader.

00054 {
00055     bTreeReader->setRootPageId(rootPageId);
00056 }

void LcsClusterReader::initColumnReaders ( uint  nClusterColsInit,
TupleProjection const &  clusterProj 
)

Initializes cluster reader with column readers.

Parameters:
nClusterColsInit total number of columns in the cluster
clusterProj list of columns to be read from cluster; column numbers are 0-based relative to the cluster

Definition at line 100 of file LcsClusterReader.cpp.

References clusterCols, LcsClusterAccessBase::nClusterCols, and nColsToRead.

00103 {
00104     nClusterCols = nClusterColsInit;
00105     nColsToRead = clusterProj.size();
00106     clusterCols.reset(new LcsColumnReader[nColsToRead]);
00107     for (uint i = 0; i < nColsToRead; i++) {
00108         clusterCols[i].init(this, clusterProj[i]);
00109     }
00110 }

void LcsClusterReader::open (  ) 

Initializes state variables used by cluster reader.

Definition at line 112 of file LcsClusterReader.cpp.

References dumbPrefetch, MAXU, nextBTreeEntry, nextPrefetchEntry, nextRid, pLeaf, pRangeBatches, CircularBufferIter< T >::reset(), and ridRunIter.

00113 {
00114     pLeaf = NULL;
00115     pRangeBatches = NULL;
00116     ridRunIter.reset();
00117 
00118     // values of (0, MAXU) indicate that entry has not been filled in yet
00119     nextBTreeEntry.first = PageId(0);
00120     nextBTreeEntry.second = LcsRid(MAXU);
00121 
00122     nextPrefetchEntry.first = PageId(0);
00123     nextPrefetchEntry.second = LcsRid(MAXU);
00124 
00125     dumbPrefetch = false;
00126     nextRid = LcsRid(0);
00127 }

void LcsClusterReader::close (  ) 

Performs shutdown on cluster reader.

Definition at line 129 of file LcsClusterReader.cpp.

References bTreeReader, and LcsClusterAccessBase::unlockClusterPage().

00130 {
00131     bTreeReader->endSearch();
00132     unlockClusterPage();
00133 }

bool LcsClusterReader::getFirstClusterPageForRead ( PConstLcsClusterNode pBlock  ) 

Gets first page in a cluster.

Parameters:
pBlock output param returning cluster page
Returns:
true if page available

Definition at line 71 of file LcsClusterReader.cpp.

References bTreeReader, and readClusterPage().

Referenced by LcsClusterAppendExecStreamTest::verifyClusterPages().

00073 {
00074     bool found;
00075 
00076     found = bTreeReader->searchFirst();
00077     if (!found) {
00078         return false;
00079     }
00080 
00081     LcsClusterNode const &node = readClusterPage();
00082     pBlock = &node;
00083     return true;
00084 }

bool LcsClusterReader::getNextClusterPageForRead ( PConstLcsClusterNode pBlock  ) 

Gets next page in a cluster, based on current position in btree.

Parameters:
pBlock output param returning cluster page
Returns:
true if page available

Definition at line 86 of file LcsClusterReader.cpp.

References bTreeReader, and readClusterPage().

Referenced by LcsClusterAppendExecStreamTest::verifyClusterPages().

00087 {
00088     bool found;
00089 
00090     found = bTreeReader->searchNext();
00091     if (!found) {
00092         return false;
00093     }
00094 
00095     LcsClusterNode const &node = readClusterPage();
00096     pBlock = &node;
00097     return true;
00098 }

bool LcsClusterReader::isPositioned (  )  const [inline]

Returns true if positioned within some range in a batch.

Definition at line 424 of file LcsClusterReader.h.

References pRangeBatches.

00425 {
00426     return pRangeBatches != NULL;
00427 }

LcsRid LcsClusterReader::getRangeStartRid (  )  const [inline]

Returns first rid in a range.

Definition at line 429 of file LcsClusterReader.h.

References rangeStartRid.

Referenced by getCurrentRid().

00430 {
00431     return rangeStartRid;
00432 }

LcsRid LcsClusterReader::getRangeEndRid (  )  const [inline]

Returns first rid after the end of the current range.

Definition at line 434 of file LcsClusterReader.h.

References rangeEndRid.

Referenced by nextRange(), and positionInBlock().

00435 {
00436     return rangeEndRid;
00437 }

uint LcsClusterReader::getRangeSize (  )  const [inline]

Returns number of rids in the range.

Definition at line 439 of file LcsClusterReader.h.

References LcsBatchDir::nRow, and pRangeBatches.

Referenced by getRangeRowsLeft().

00440 {
00441     return pRangeBatches->nRow;
00442 }

uint LcsClusterReader::getRangePos (  )  const [inline]

Returns offset within the current range.

E.g., 0 if at first rid in range

Definition at line 444 of file LcsClusterReader.h.

References nRangePos.

Referenced by getCurrentRid(), LcsColumnReader::getCurrentValueCode(), LcsColumnReader::getFixedValue(), getRangeRowsLeft(), LcsColumnReader::getVariableValue(), and LcsColumnReader::readCompressedBatch().

00445 {
00446     return nRangePos;
00447 }

LcsRid LcsClusterReader::getCurrentRid (  )  const [inline]

Returns rid currently positioned at.

Definition at line 449 of file LcsClusterReader.h.

References getRangePos(), and getRangeStartRid().

00450 {
00451     return getRangeStartRid() + getRangePos();
00452 }

uint LcsClusterReader::getRangeRowsLeft (  )  const [inline]

Returns number of rids yet to be read.

(E.g., 1 if we're at the last rid)

Definition at line 454 of file LcsClusterReader.h.

References getRangePos(), and getRangeSize().

Referenced by LcsColumnReader::readCompressedBatch().

00455 {
00456     return getRangeSize() - getRangePos();
00457 }

bool LcsClusterReader::position ( LcsRid  rid  ) 

Positions scan on the rid, moving to a new range if necessary.

Parameters:
rid rid to position to
Returns:
false if rid is beyond end of cluster

Definition at line 135 of file LcsClusterReader.cpp.

References bTreeReader, currRid, SegPageEntryIter< EntryT >::mapRange(), moveToBlock(), moveToBlockWithRid(), noPrefetch, NULL_PAGE_ID, pLeaf, positionInBlock(), prefetchQueue, LcsClusterAccessBase::readClusterPageId(), searchForRid(), and LcsClusterAccessBase::segmentAccessor.

Referenced by nextRange().

00136 {
00137     bool found;
00138 
00139     currRid = rid;
00140     if (pLeaf) {
00141         // Scan is already in progress. Try to find the row we want in the
00142         // current block.
00143 
00144         found = positionInBlock(rid);
00145         if (found) {
00146             return true;
00147         }
00148     } else {
00149         if (noPrefetch) {
00150             if (!bTreeReader->searchFirst()) {
00151                 bTreeReader->endSearch();
00152             }
00153         } else {
00154             // Initiate the first set of pre-fetches.
00155             prefetchQueue.mapRange(segmentAccessor, NULL_PAGE_ID);
00156         }
00157     }
00158 
00159     if (noPrefetch) {
00160         found = searchForRid(rid);
00161         if (!found) {
00162             return false;
00163         }
00164         moveToBlock(readClusterPageId());
00165     } else {
00166         found = moveToBlockWithRid(rid);
00167         if (!found) {
00168             return false;
00169         }
00170     }
00171 
00172     found = positionInBlock(rid);
00173     // page ends before "rid"; we must be off the last block
00174     if (!found) {
00175         return false;
00176     }
00177 
00178     return true;
00179 }

bool LcsClusterReader::nextRange (  )  [inline]

Positions scan on the first rid of the next range.

Definition at line 459 of file LcsClusterReader.h.

References getRangeEndRid(), and position().

00460 {
00461     return position(getRangeEndRid());
00462 }

bool LcsClusterReader::advance ( uint  nRids  ) 

Moves "nRids" forward in the current range.

Parameters:
nRids number of rids to move forward
Returns:
false if we are at the endof the range and therefore cannot advance the desired number of rids

Definition at line 295 of file LcsClusterReader.cpp.

References nRangePos.

00296 {
00297     uint newPos = nRangePos + nRids;
00298 
00299     if (newPos < pRangeBatches->nRow) {
00300         nRangePos = newPos;
00301         return true;
00302     } else {
00303         return false;
00304     }
00305 }

void LcsClusterReader::advanceWithinBatch ( uint  nRids  )  [inline]

Advances nRids forward in the current batch.

Parameters:
nRids number of rids to advance

Definition at line 464 of file LcsClusterReader.h.

References nRangePos.

00465 {
00466     nRangePos = nRangePos + nRids;
00467 }

PageId LcsClusterReader::getNextPageForPrefetch ( LcsRid &  rid,
bool &  found 
) [virtual]

Determines the next page to be pre-fetched based on the rids that need to be read.

Parameters:
[out] rid returns the rid corresponding to the page to be pre-fetched
[out] found true if a pre-fetch page was found
Returns:
the page to be pre-fetched next

Implements SegPageEntryIterSource< LcsRid >.

Definition at line 307 of file LcsClusterReader.cpp.

References currRid, CircularBufferIter< T >::done(), dumbPrefetch, getFetchRids(), getNextBTreeEntry(), MAXU, nextBTreeEntry, nextRid, NULL_PAGE_ID, LcsClusterAccessBase::readClusterPageId(), LcsClusterAccessBase::readRid(), ridRunIter, and searchForRid().

00308 {
00309     found = true;
00310     for (;;) {
00311         if (dumbPrefetch) {
00312             rid = currRid;
00313         } else {
00314             rid = getFetchRids(ridRunIter, nextRid, false);
00315         }
00316 
00317         // If the rid run buffer is exhausted and there are still rid runs
00318         // to be filled in, we're pre-fetching faster than we can fill up
00319         // the rid run buffers.  So, switch to dumb pre-fetch in that case.
00320         if (rid == LcsRid(MAXU)) {
00321             if (ridRunIter.done()) {
00322                 rid = LcsRid(0);
00323                 return NULL_PAGE_ID;
00324             } else {
00325                 // TODO - Use stricter criteria on when to switch to dumb
00326                 // pre-fetch.  E.g., if there are less than a certain number
00327                 // of pages already pre-fetched.
00328                 dumbPrefetch = true;
00329                 continue;
00330             }
00331         }
00332 
00333         if (!(nextBTreeEntry.second == LcsRid(MAXU) &&
00334             nextBTreeEntry.first == PageId(0)))
00335         {
00336             // If we hit the end of the btree on the last iteration, then
00337             // there are no more pages.
00338             if (nextBTreeEntry.first == NULL_PAGE_ID) {
00339                 rid = LcsRid(0);
00340                 return NULL_PAGE_ID;
00341             }
00342 
00343             // If the next rid to be read is on the same page as the last
00344             // one located, bump up the rid so we move over to the next page.
00345             // In the case of dumb pre-fetch, just use the next page in the
00346             // cluster sequence.
00347             if (rid < nextBTreeEntry.second) {
00348                 if (dumbPrefetch) {
00349                     rid = nextBTreeEntry.second;
00350                     PageId pageId = nextBTreeEntry.first;
00351                     getNextBTreeEntry();
00352                     return pageId;
00353                 }
00354                 nextRid = nextBTreeEntry.second;
00355                 continue;
00356             } else {
00357                 // TODO - optimize by avoiding search from top of btree if
00358                 // desired entry is within a few entries of the current
00359                 break;
00360            }
00361         } else {
00362             // We haven't located any pages yet.  Initiate first btree search.
00363             break;
00364         }
00365     }
00366 
00367     bool rc = searchForRid(rid);
00368     if (!rc) {
00369         return NULL_PAGE_ID;
00370     }
00371     rid = readRid();
00372     PageId pageId = readClusterPageId();
00373     getNextBTreeEntry();
00374     return pageId;
00375 }

LcsRid LcsClusterReader::getFetchRids ( CircularBufferIter< LcsRidRun > &  ridRunIter,
LcsRid &  nextRid,
bool  remove 
) [static]

Retrieves the current and next rids that need to be read, based on the current rid run being processed.

Parameters:
ridRunIter iterator over the circular buffer that provides the rid runs
[in,out] nextRid on input, indicates the minimum rid value to be retrieved; on return, the next rid that will need to be read
remove if true, remove the first element from the circular buffer if the rid run buffer's position is incremented
Returns:
the current rid that needs to be read

Definition at line 390 of file LcsClusterReader.cpp.

References CircularBufferIter< T >::end(), MAXU, LcsRidRun::nRids, CircularBufferIter< T >::removeFront(), ridRunIter, and LcsRidRun::startRid.

Referenced by LcsRowScanExecStream::execute(), and getNextPageForPrefetch().

00394 {
00395     for (;;) {
00396         if (ridRunIter.end()) {
00397             return LcsRid(MAXU);
00398         }
00399 
00400         LcsRidRun &currRidRun = *ridRunIter;
00401         if (nextRid < currRidRun.startRid) {
00402             nextRid = currRidRun.startRid + 1;
00403             return currRidRun.startRid;
00404         } else if (
00405             (currRidRun.nRids == RecordNum(MAXU) &&
00406                 nextRid >= currRidRun.startRid) ||
00407             (nextRid < currRidRun.startRid + currRidRun.nRids))
00408         {
00409             return nextRid++;
00410         } else {
00411             ++ridRunIter;
00412             if (remove) {
00413                 ridRunIter.removeFront();
00414             }
00415         }
00416     }
00417 }

void LcsClusterReader::catchUp ( uint  parentBufPos,
LcsRid  parentNextRid 
)

Resynchronizes the cluster reader's prefetch position to that of the parent scan.

Parameters:
parentBufPos buffer position of the parent scan
parentNextRid next rid of the parent scan

Definition at line 419 of file LcsClusterReader.cpp.

References CircularBufferIter< T >::getCurrPos(), nextRid, ridRunIter, and CircularBufferIter< T >::setCurrPos().

00420 {
00421     if (parentNextRid - 1 > nextRid) {
00422         nextRid = parentNextRid - 1;
00423     }
00424     if (parentBufPos > ridRunIter.getCurrPos()) {
00425         ridRunIter.setCurrPos(parentBufPos);
00426     }
00427 }

RecordNum LcsClusterReader::getNumRows (  ) 

Returns:
the number of rows in the cluster

Definition at line 429 of file LcsClusterReader.cpp.

References bTreeReader, LcsClusterAccessBase::bTreeTupleData, LcsClusterNode::firstRID, LcsClusterNode::nBatch, LcsClusterAccessBase::nClusterCols, LcsClusterNode::oBatch, opaqueToInt(), and readClusterPage().

00430 {
00431     // Read the last cluster page
00432     if (bTreeReader->searchLast() == false) {
00433         bTreeReader->endSearch();
00434         return RecordNum(0);
00435     }
00436     bTreeReader->getTupleAccessorForRead().unmarshal(bTreeTupleData);
00437     LcsClusterNode const &node = readClusterPage();
00438 
00439     // Then count the number of rows in each batch on that page
00440     RecordNum nRows = RecordNum(opaqueToInt(node.firstRID));
00441     PLcsBatchDir pBatch = (PLcsBatchDir) ((PBuffer) &node + node.oBatch);
00442     for (uint i = 0; i < node.nBatch; i += nClusterCols) {
00443         nRows += pBatch[i].nRow;
00444     }
00445 
00446     bTreeReader->endSearch();
00447     return nRows;
00448 }

LcsRid LcsClusterAccessBase::readRid (  )  [protected, inherited]

Returns RID from btree tuple.

Definition at line 37 of file LcsClusterAccessBase.cpp.

References LcsClusterAccessBase::bTreeTupleData.

Referenced by getNextBTreeEntry(), getNextPageForPrefetch(), readClusterPage(), and searchForRid().

00038 {
00039     return *reinterpret_cast<LcsRid const *> (bTreeTupleData[0].pData);
00040 }

PageId LcsClusterAccessBase::readClusterPageId (  )  [protected, inherited]

Returns cluster pageid from btree tuple.

Definition at line 42 of file LcsClusterAccessBase.cpp.

References LcsClusterAccessBase::bTreeTupleData.

Referenced by LcsClusterNodeWriter::getLastClusterPageForWrite(), getNextBTreeEntry(), getNextPageForPrefetch(), position(), and readClusterPage().

00043 {
00044     return *reinterpret_cast<PageId const *> (bTreeTupleData[1].pData);
00045 }

void LcsClusterAccessBase::setHdrOffsets ( PConstLcsClusterNode  pHdr  )  [protected, inherited]

Sets pointers to offset arrays in cluster page header.

Parameters:
pHdr pointer to cluster node header

Definition at line 47 of file LcsClusterAccessBase.cpp.

References LcsClusterAccessBase::delta, LcsClusterAccessBase::firstVal, LcsClusterAccessBase::lastVal, LcsClusterAccessBase::nClusterCols, and LcsClusterAccessBase::nVal.

Referenced by LcsClusterDump::dump(), LcsClusterNodeWriter::init(), and setUpBlock().

00048 {
00049     PBuffer p = (PBuffer) pHdr;
00050 
00051     lastVal = (uint16_t *) (p += sizeof(LcsClusterNode));
00052     firstVal = (uint16_t *) (p += sizeof(uint16_t) * nClusterCols);
00053     nVal = (uint *) (p += sizeof(uint16_t) * nClusterCols);
00054     delta = (uint16_t *) (p += sizeof(uint) * nClusterCols);
00055 }

uint LcsClusterAccessBase::getNumClusterCols (  )  [inline, inherited]

Returns number of columns in cluster.

Definition at line 118 of file LcsClusterAccessBase.h.

00119     {
00120         return nClusterCols;
00121     }

void LcsClusterAccessBase::setNumClusterCols ( uint  nCols  )  [inline, inherited]

Sets number of columns in cluster.

Definition at line 126 of file LcsClusterAccessBase.h.

00127     {
00128         nClusterCols = nCols;
00129     }

void LcsClusterAccessBase::unlockClusterPage (  )  [inherited]

Unlocks cluster page.

Definition at line 57 of file LcsClusterAccessBase.cpp.

References LcsClusterAccessBase::clusterLock, and SegPageLock::unlock().

Referenced by close().

00058 {
00059     clusterLock.unlock();
00060 }

virtual void SegPageEntryIterSource< LcsRid >::initPrefetchEntry ( LcsRid &  entry  )  [inline, virtual, inherited]

Initializes a specific entry in the pre-fetch queue.

Parameters:
entry the entry that will be initialized

Definition at line 46 of file SegPageEntryIterSource.h.

00047     {
00048     }


Friends And Related Function Documentation

friend class LcsColumnReader [friend]

Definition at line 87 of file LcsClusterReader.h.


Member Data Documentation

SharedBTreeReader LcsClusterReader::bTreeReader [private]

Reads btree corresponding to cluster.

Definition at line 92 of file LcsClusterReader.h.

Referenced by close(), getFirstClusterPageForRead(), getNextBTreeEntry(), getNextClusterPageForRead(), getNumRows(), LcsClusterReader(), position(), readClusterPage(), searchForRid(), and setRootPageId().

PLcsBatchDir LcsClusterReader::pRangeBatches [private]

Pointer to first batch in the current range.

Definition at line 97 of file LcsClusterReader.h.

Referenced by getRangeSize(), isPositioned(), open(), positionInBlock(), setUpBlock(), and LcsColumnReader::sync().

LcsRid LcsClusterReader::rangeStartRid [private]

First rid in the current range.

Definition at line 102 of file LcsClusterReader.h.

Referenced by getRangeStartRid(), positionInBlock(), and setUpBlock().

LcsRid LcsClusterReader::rangeEndRid [private]

1 rid past the last rid in the current range

Definition at line 107 of file LcsClusterReader.h.

Referenced by getRangeEndRid(), positionInBlock(), and setUpBlock().

uint LcsClusterReader::nRangePos [private]

Offset from start of current range.

Definition at line 112 of file LcsClusterReader.h.

Referenced by advance(), advanceWithinBatch(), getRangePos(), and setUpBlock().

PBuffer LcsClusterReader::pLeaf [private]

Pointer to current cluster block.

Definition at line 117 of file LcsClusterReader.h.

Referenced by open(), position(), setUpBlock(), and LcsColumnReader::sync().

PConstLcsClusterNode LcsClusterReader::pLHdr [private]

Pointer to header of cluster block.

Definition at line 122 of file LcsClusterReader.h.

Referenced by moveToBlock(), moveToBlockWithRid(), and setUpBlock().

PLcsBatchDir LcsClusterReader::pBatches [private]

Batch directory.

Definition at line 127 of file LcsClusterReader.h.

Referenced by setUpBlock().

bool LcsClusterReader::noPrefetch [private]

If true, do not use pre-fetch.

Definition at line 132 of file LcsClusterReader.h.

Referenced by LcsClusterReader(), and position().

CircularBufferIter<LcsRidRun> LcsClusterReader::ridRunIter [private]

Iterator over the circular buffer containing rid runs.

Definition at line 137 of file LcsClusterReader.h.

Referenced by catchUp(), getFetchRids(), getNextPageForPrefetch(), and open().

SegPageEntryIter<LcsRid> LcsClusterReader::prefetchQueue [private]

Pre-fetch queue for pages read from this cluster.

Definition at line 142 of file LcsClusterReader.h.

Referenced by LcsClusterReader(), moveToBlockWithRid(), and position().

LcsRid LcsClusterReader::nextRid [private]

The next rid that needs to be read.

Definition at line 147 of file LcsClusterReader.h.

Referenced by catchUp(), getNextPageForPrefetch(), and open().

std::pair<PageId, LcsRid> LcsClusterReader::nextBTreeEntry [private]

The next entry in the rid-to-pageId btree map following the one that contains the last page that was pre-fetched.

Definition at line 153 of file LcsClusterReader.h.

Referenced by getNextBTreeEntry(), getNextPageForPrefetch(), and open().

std::pair<PageId, LcsRid> LcsClusterReader::nextPrefetchEntry [private]

The next entry from the pre-fetch queue.

Definition at line 158 of file LcsClusterReader.h.

Referenced by moveToBlockWithRid(), and open().

bool LcsClusterReader::dumbPrefetch [private]

If true, switch has been made to dumb pre-fetch.

Definition at line 163 of file LcsClusterReader.h.

Referenced by getNextPageForPrefetch(), and open().

LcsRid LcsClusterReader::currRid [private]

The current rid that needs to be read.

Definition at line 168 of file LcsClusterReader.h.

Referenced by getNextPageForPrefetch(), and position().

uint LcsClusterReader::nColsToRead

Number of cluster columns that will be read.

Definition at line 238 of file LcsClusterReader.h.

Referenced by initColumnReaders().

boost::scoped_array<LcsColumnReader> LcsClusterReader::clusterCols

Column readers for each cluster column that will be read.

Definition at line 243 of file LcsClusterReader.h.

Referenced by initColumnReaders().

TupleData LcsClusterAccessBase::bTreeTupleData [protected, inherited]

Tuple data representing the btree key corresponding to the cluster page.

Definition at line 42 of file LcsClusterAccessBase.h.

Referenced by LcsClusterNodeWriter::allocateClusterPage(), LcsClusterNodeWriter::getLastClusterPageForWrite(), getNextBTreeEntry(), getNumRows(), LcsClusterAccessBase::LcsClusterAccessBase(), readClusterPage(), LcsClusterAccessBase::readClusterPageId(), LcsClusterAccessBase::readRid(), and searchForRid().

SegmentAccessor LcsClusterAccessBase::segmentAccessor [protected, inherited]

Accessor for segment storing both btree and cluster pages.

Definition at line 47 of file LcsClusterAccessBase.h.

Referenced by LcsClusterNodeWriter::allocateClusterPage(), LcsClusterAccessBase::LcsClusterAccessBase(), and position().

ClusterPageLock LcsClusterAccessBase::clusterLock [protected, inherited]

Buffer lock for the actual cluster node pages.

Shares the same segment as the btree corresponding to the cluster.

Definition at line 53 of file LcsClusterAccessBase.h.

Referenced by LcsClusterNodeWriter::allocateClusterPage(), LcsClusterNodeWriter::close(), LcsClusterNodeWriter::getLastClusterPageForWrite(), LcsClusterAccessBase::LcsClusterAccessBase(), moveToBlock(), moveToBlockWithRid(), readClusterPage(), and LcsClusterAccessBase::unlockClusterPage().

PageId LcsClusterAccessBase::clusterPageId [protected, inherited]

Current cluster pageid.

Definition at line 58 of file LcsClusterAccessBase.h.

Referenced by LcsClusterNodeWriter::allocateClusterPage(), LcsClusterNodeWriter::getLastClusterPageForWrite(), LcsClusterVerifier::getPageData(), LcsClusterNodeWriter::moveFromTempToIndex(), moveToBlockWithRid(), and readClusterPage().

LcsRid LcsClusterAccessBase::bTreeRid [protected, inherited]

Current rid in btree used to access current cluster page.

Definition at line 63 of file LcsClusterAccessBase.h.

Referenced by LcsClusterNodeWriter::allocateClusterPage(), LcsClusterVerifier::getPageData(), and readClusterPage().

uint LcsClusterAccessBase::nClusterCols [protected, inherited]

Number of columns in cluster.

Definition at line 68 of file LcsClusterAccessBase.h.

Referenced by LcsClusterNodeWriter::addValue(), LcsClusterNodeWriter::allocArrays(), LcsClusterDump::dump(), getNumRows(), LcsClusterNodeWriter::init(), initColumnReaders(), LcsClusterNodeWriter::LcsClusterNodeWriter(), LcsClusterNodeWriter::moveFromIndexToTemp(), LcsClusterNodeWriter::moveFromTempToIndex(), LcsClusterNodeWriter::openAppend(), LcsClusterNodeWriter::openNew(), positionInBlock(), and LcsClusterAccessBase::setHdrOffsets().

uint16_t* LcsClusterAccessBase::lastVal [protected, inherited]

Offsets to the last value stored on the page for each column in cluster.

Definition at line 74 of file LcsClusterAccessBase.h.

Referenced by LcsClusterNodeWriter::addValue(), LcsClusterDump::dump(), LcsClusterNodeWriter::moveFromIndexToTemp(), LcsClusterNodeWriter::moveFromTempToIndex(), LcsClusterNodeWriter::openAppend(), LcsClusterNodeWriter::openNew(), LcsClusterNodeWriter::pickCompressionMode(), LcsClusterNodeWriter::putCompressedBatch(), LcsClusterNodeWriter::putFixedVarBatch(), LcsClusterNodeWriter::rollBackLastBatch(), LcsClusterAccessBase::setHdrOffsets(), and LcsClusterNodeWriter::undoValue().

uint16_t* LcsClusterAccessBase::firstVal [protected, inherited]

Offsets to the first value stored on the page for each column in cluster.

Points to the end of the firstVal, so subtracting lastVal from firstVal will tell you the number of bytes taken up by values for a column, since lastVals are appended in front of firstVal

Definition at line 82 of file LcsClusterAccessBase.h.

Referenced by LcsClusterDump::dump(), LcsClusterNodeWriter::moveFromIndexToTemp(), LcsClusterNodeWriter::moveFromTempToIndex(), LcsClusterNodeWriter::openNew(), and LcsClusterAccessBase::setHdrOffsets().

uint* LcsClusterAccessBase::nVal [protected, inherited]

Number of distinct values in the page for each column in cluster.

Definition at line 87 of file LcsClusterAccessBase.h.

Referenced by LcsClusterNodeWriter::addValue(), LcsClusterDump::dump(), LcsClusterNodeWriter::moveFromIndexToTemp(), LcsClusterNodeWriter::moveFromTempToIndex(), LcsClusterNodeWriter::openAppend(), LcsClusterNodeWriter::openNew(), LcsClusterNodeWriter::pickCompressionMode(), LcsClusterNodeWriter::putCompressedBatch(), LcsClusterNodeWriter::putFixedVarBatch(), LcsClusterNodeWriter::rollBackLastBatch(), LcsClusterAccessBase::setHdrOffsets(), and LcsClusterNodeWriter::undoValue().

uint16_t* LcsClusterAccessBase::delta [protected, inherited]

For each column in the cluster, offset used to get the real offset within the page.

Definition at line 93 of file LcsClusterAccessBase.h.

Referenced by LcsClusterDump::dump(), LcsClusterNodeWriter::moveFromTempToIndex(), LcsClusterNodeWriter::openNew(), LcsClusterAccessBase::setHdrOffsets(), and LcsColumnReader::sync().


The documentation for this class was generated from the following files:
Generated on Mon Jun 22 04:00:37 2009 for Fennel by  doxygen 1.5.1