LcsClusterReader Class Reference

Reads blocks from a single cluster. More...

#include <LcsClusterReader.h>

Inheritance diagram for LcsClusterReader:

List of all members.

Public Member Functions
	LcsClusterReader (BTreeDescriptor const &treeDescriptor, CircularBuffer< LcsRidRun > *pRidRuns=NULL)
	Constructor.
virtual	~LcsClusterReader ()
void	setRootPageId (PageId rootPageId)
	Sets the root pageId of the underlying btree corresponding to the cluster.
void	initColumnReaders (uint nClusterColsInit, TupleProjection const &clusterProj)
	Initializes cluster reader with column readers.
void	open ()
	Initializes state variables used by cluster reader.
void	close ()
	Performs shutdown on cluster reader.
bool	getFirstClusterPageForRead (PConstLcsClusterNode &pBlock)
	Gets first page in a cluster.
bool	getNextClusterPageForRead (PConstLcsClusterNode &pBlock)
	Gets next page in a cluster, based on current position in btree.
bool	isPositioned () const
	Returns true if positioned within some range in a batch.
LcsRid	getRangeStartRid () const
	Returns first rid in a range.
LcsRid	getRangeEndRid () const
	Returns first rid after the end of the current range.
uint	getRangeSize () const
	Returns number of rids in the range.
uint	getRangePos () const
	Returns offset within the current range.
LcsRid	getCurrentRid () const
	Returns rid currently positioned at.
uint	getRangeRowsLeft () const
	Returns number of rids yet to be read.
bool	position (LcsRid rid)
	Positions scan on the rid, moving to a new range if necessary.
bool	nextRange ()
	Positions scan on the first rid of the next range.
bool	advance (uint nRids)
	Moves "nRids" forward in the current range.
void	advanceWithinBatch (uint nRids)
	Advances nRids forward in the current batch.
virtual PageId	getNextPageForPrefetch (LcsRid &rid, bool &found)
	Determines the next page to be pre-fetched based on the rids that need to be read.
void	catchUp (uint parentBufPos, LcsRid parentNextRid)
	Resynchronizes the cluster reader's prefetch position to that of the parent scan.
RecordNum	getNumRows ()
	Returns: the number of rows in the cluster
uint	getNumClusterCols ()
	Returns number of columns in cluster.
void	setNumClusterCols (uint nCols)
	Sets number of columns in cluster.
void	unlockClusterPage ()
	Unlocks cluster page.
virtual void	initPrefetchEntry (LcsRid &entry)
	Initializes a specific entry in the pre-fetch queue.
Static Public Member Functions
static LcsRid	getFetchRids (CircularBufferIter< LcsRidRun > &ridRunIter, LcsRid &nextRid, bool remove)
	Retrieves the current and next rids that need to be read, based on the current rid run being processed.
Public Attributes
uint	nColsToRead
	Number of cluster columns that will be read.
boost::scoped_array< LcsColumnReader >	clusterCols
	Column readers for each cluster column that will be read.
Protected Member Functions
LcsRid	readRid ()
	Returns RID from btree tuple.
PageId	readClusterPageId ()
	Returns cluster pageid from btree tuple.
void	setHdrOffsets (PConstLcsClusterNode pHdr)
	Sets pointers to offset arrays in cluster page header.
Protected Attributes
TupleData	bTreeTupleData
	Tuple data representing the btree key corresponding to the cluster page.
SegmentAccessor	segmentAccessor
	Accessor for segment storing both btree and cluster pages.
ClusterPageLock	clusterLock
	Buffer lock for the actual cluster node pages.
PageId	clusterPageId
	Current cluster pageid.
LcsRid	bTreeRid
	Current rid in btree used to access current cluster page.
uint	nClusterCols
	Number of columns in cluster.
uint16_t *	lastVal
	Offsets to the last value stored on the page for each column in cluster.
uint16_t *	firstVal
	Offsets to the first value stored on the page for each column in cluster.
uint *	nVal
	Number of distinct values in the page for each column in cluster.
uint16_t *	delta
	For each column in the cluster, offset used to get the real offset within the page.
Private Member Functions
void	moveToBlock (PageId clusterPageId)
	Reads a cluster block and sets up necessary structures to navigate within the page.
bool	moveToBlockWithRid (LcsRid rid)
	Reads a cluster block and sets up necessary structures to navigate within the page corresponding to a rid.
bool	positionInBlock (LcsRid rid)
	Finds the range in the current block which contains "Rid".
void	positionInRange (uint pos)
	Positions on "pos" in the current range.
void	setUpBlock ()
	Updates class status to reflect page just read and sets header values to point within the current page.
LcsClusterNode const &	readClusterPage ()
	Reads a cluster page based on current btree position.
bool	searchForRid (LcsRid rid)
	Reads btree and locates rid in tree.
void	getNextBTreeEntry ()
	Reads the next entry in the rid-to-pageId btree map.
Private Attributes
SharedBTreeReader	bTreeReader
	Reads btree corresponding to cluster.
PLcsBatchDir	pRangeBatches
	Pointer to first batch in the current range.
LcsRid	rangeStartRid
	First rid in the current range.
LcsRid	rangeEndRid
	1 rid past the last rid in the current range
uint	nRangePos
	Offset from start of current range.
PBuffer	pLeaf
	Pointer to current cluster block.
PConstLcsClusterNode	pLHdr
	Pointer to header of cluster block.
PLcsBatchDir	pBatches
	Batch directory.
bool	noPrefetch
	If true, do not use pre-fetch.
CircularBufferIter< LcsRidRun >	ridRunIter
	Iterator over the circular buffer containing rid runs.
SegPageEntryIter< LcsRid >	prefetchQueue
	Pre-fetch queue for pages read from this cluster.
LcsRid	nextRid
	The next rid that needs to be read.
std::pair< PageId, LcsRid >	nextBTreeEntry
	The next entry in the rid-to-pageId btree map following the one that contains the last page that was pre-fetched.
std::pair< PageId, LcsRid >	nextPrefetchEntry
	The next entry from the pre-fetch queue.
bool	dumbPrefetch
	If true, switch has been made to dumb pre-fetch.
LcsRid	currRid
	The current rid that needs to be read.
Friends
class	LcsColumnReader

---

Detailed Description

Reads blocks from a single cluster.

A column reader object (LcsColumnReader) is used to read the value of a particular column at the current rowid of the LcsClusterReader.

A cluster holds the values for one or more columns. The values of these columns are held in rowid ranges. Each range contains one batch (LcsBatchDir) for each column; this batch holds the values of that column for each rowid in the range. A batch may be "fixed", "compressed", or "variable".

Suppose a table has 2 column store clusters: one containing columns A, B, and C, and another containing columns D and E. To read the values of columns A and C, create a LcsClusterReader on the first cluster, and two LcsColumnReader onto this LcsClusterReader, one for each column, A and C.

Here's how to use a LcsClusterReader (scan) and LcsColumnReader (colScan) for doing a full table scan:

scan.init for i in 0..nClusterCols colScan[i].init scan.position(first rid) do for i in 0..nClusterCols colScan[i].sync do for i in 0..nClusterCols colScan[i].getCurrentValue while scan.advance(1) while scan.nextRange != false scan.close()

Definition at line 84 of file LcsClusterReader.h.

---

Constructor & Destructor Documentation

LcsClusterReader::LcsClusterReader	(	BTreeDescriptor const &	treeDescriptor,
		CircularBuffer< LcsRidRun > *	pRidRuns = NULL
	)			[explicit]

Constructor.

Parameters:

	treeDescriptor	of btree corresponding to cluster
	pRidRuns	pointer to circular buffer of rid runs; defaults to NULL, in which case, pre-fetches are disabled

Definition at line 31 of file LcsClusterReader.cpp.

References bTreeReader, noPrefetch, prefetchQueue, and SegPageEntryIter< EntryT >::setPrefetchSource().

:
    LcsClusterAccessBase(treeDescriptor),
    SegPageEntryIterSource<LcsRid>(),
    ridRunIter(pRidRuns),
    prefetchQueue(4000)
{
    bTreeReader = SharedBTreeReader(new BTreeReader(treeDescriptor));
    if (pRidRuns == NULL) {
        noPrefetch = true;
    } else {
        noPrefetch = false;
        prefetchQueue.setPrefetchSource(*this);
    }
}

LcsClusterReader::~LcsClusterReader

(

)

[virtual]

Definition at line 49 of file LcsClusterReader.cpp.

{
}

---

Member Function Documentation

void LcsClusterReader::moveToBlock

(

PageId

clusterPageId

)

[private]

Reads a cluster block and sets up necessary structures to navigate within the page.

Parameters:

clusterPageId

pageId of the cluster page to read

Definition at line 197 of file LcsClusterReader.cpp.

References LcsClusterAccessBase::clusterLock, SegNodeLock< Node >::getNodeForRead(), SegPageLock::lockShared(), pLHdr, and setUpBlock().

Referenced by position().

{
    // read the desired cluster page and initialize structures to reflect
    // page read

    clusterLock.lockShared(clusterPageId);
    LcsClusterNode const &page = clusterLock.getNodeForRead();
    pLHdr = &page;
    setUpBlock();
}

bool LcsClusterReader::moveToBlockWithRid

(

LcsRid

rid

)

[private]

Reads a cluster block and sets up necessary structures to navigate within the page corresponding to a rid.

Parameters:

rid

the rid that determines the page that needs to be read

Returns:

true if a cluster block was successfully located

Definition at line 208 of file LcsClusterReader.cpp.

References LcsClusterAccessBase::clusterLock, LcsClusterAccessBase::clusterPageId, SegNodeLock< Node >::getNodeForRead(), SegPageLock::lockShared(), MAXU, nextPrefetchEntry, NULL_PAGE_ID, pLHdr, prefetchQueue, and setUpBlock().

Referenced by position().

{
    PageId clusterPageId;

    // Read entries from the pre-fetch queue until we either find the entry
    // within the range of our desired rid, or we exhaust the contents of
    // the queue.
    for (;;) {
        std::pair<PageId, LcsRid> &prefetchEntry =
            (nextPrefetchEntry.second == LcsRid(MAXU)) ?
            *prefetchQueue : nextPrefetchEntry;

        clusterPageId = prefetchEntry.first;
        if (clusterPageId == NULL_PAGE_ID) {
            return false;
        }

        LcsRid prefetchRid = prefetchEntry.second;
        assert(prefetchRid <= rid);

        // Make sure this is the correct entry by checking that the rid
        // is smaller than the next entry.  We can end up with non-matching
        // entries if dumb pre-fetches are being used.
        ++prefetchQueue;
        nextPrefetchEntry = *prefetchQueue;
        if (nextPrefetchEntry.first == NULL_PAGE_ID ||
            rid < nextPrefetchEntry.second)
        {
            break;
        } else {
            continue;
        }
    }

    // Now that we've located the desired page, read it.
    clusterLock.lockShared(clusterPageId);
    LcsClusterNode const &page = clusterLock.getNodeForRead();
    pLHdr = &page;
    setUpBlock();
    return true;
}

bool LcsClusterReader::positionInBlock

(

LcsRid

rid

)

[private]

Finds the range in the current block which contains "Rid".

Does not search backwards.

Parameters:

rid

rid to be located

Returns:

false if "Rid" is beyond the end of the block

Definition at line 250 of file LcsClusterReader.cpp.

References getRangeEndRid(), LcsClusterAccessBase::nClusterCols, LcsBatchDir::nRow, opaqueToInt(), positionInRange(), pRangeBatches, rangeEndRid, and rangeStartRid.

Referenced by position().

{
    // Go forward through the ranges in the current block until we find the
    // right one, or until we hit the end of the block
    while (rid >= getRangeEndRid()
           && pRangeBatches + nClusterCols < pBatches + pLHdr->nBatch) {
        rangeStartRid += pRangeBatches->nRow;

        pRangeBatches += nClusterCols;            // go to start of next range

        // set end rowid based on already available info (for performance
        // reasons)
        rangeEndRid = rangeStartRid + pRangeBatches->nRow;
    }

    // Try to position within current batch
    if (rid < getRangeEndRid()) {
        assert(rid >= rangeStartRid);
        positionInRange(opaqueToInt(rid) - opaqueToInt(rangeStartRid));
        return true;
    } else {
        return false;
    }
}

void LcsClusterReader::positionInRange

(

uint

pos

)

[inline, private]

Positions on "pos" in the current range.

Parameters:

pos

desired position

Definition at line 203 of file LcsClusterReader.h.

Referenced by positionInBlock().

    {
        nRangePos = pos;
    }

void LcsClusterReader::setUpBlock

(

)

[private]

Updates class status to reflect page just read and sets header values to point within the current page.

Definition at line 275 of file LcsClusterReader.cpp.

References nRangePos, LcsBatchDir::nRow, pBatches, pLeaf, pLHdr, pRangeBatches, rangeEndRid, rangeStartRid, and LcsClusterAccessBase::setHdrOffsets().

Referenced by moveToBlock(), and moveToBlockWithRid().

{
    // setup pointers to lastVal, firstVal arrays
    pLeaf = (PBuffer) pLHdr;
    setHdrOffsets(pLHdr);

    assert(pLHdr->nBatch > 0);

    pBatches = (PLcsBatchDir) (pLeaf + pLHdr->oBatch);
    rangeStartRid = pLHdr->firstRID;

    // at first range in block
    pRangeBatches = pBatches;
    // at first rid in range
    nRangePos = 0;

    // set end rowid based on already available info (for performance reasons)
    rangeEndRid = rangeStartRid + pRangeBatches->nRow;
}

LcsClusterNode const & LcsClusterReader::readClusterPage

(

)

[private]

Reads a cluster page based on current btree position.

Returns:

cluster page read

Definition at line 58 of file LcsClusterReader.cpp.

References bTreeReader, LcsClusterAccessBase::bTreeRid, LcsClusterAccessBase::bTreeTupleData, LcsClusterAccessBase::clusterLock, LcsClusterAccessBase::clusterPageId, LcsClusterNode::firstRID, SegNodeLock< Node >::getNodeForRead(), SegPageLock::lockShared(), LcsClusterAccessBase::readClusterPageId(), and LcsClusterAccessBase::readRid().

Referenced by getFirstClusterPageForRead(), getNextClusterPageForRead(), and getNumRows().

{
    bTreeReader->getTupleAccessorForRead().unmarshal(bTreeTupleData);
    clusterPageId = readClusterPageId();
    // REVIEW jvs 27-Dec-2005:  What is bTreeRid used for?  Should probably
    // assert that it matches node.firstRID
    bTreeRid = readRid();
    clusterLock.lockShared(clusterPageId);
    LcsClusterNode const &node = clusterLock.getNodeForRead();
    assert(bTreeRid == node.firstRID);
    return node;
}

bool LcsClusterReader::searchForRid

(

LcsRid

rid

)

[private]

Reads btree and locates rid in tree.

Returns:

false if rid is not within the range of the tree; e.g., if it's empty

Definition at line 181 of file LcsClusterReader.cpp.

References bTreeReader, LcsClusterAccessBase::bTreeTupleData, DUP_SEEK_BEGIN, and LcsClusterAccessBase::readRid().

Referenced by getNextPageForPrefetch(), and position().

{
    bTreeTupleData[0].pData = (PConstBuffer) &rid;
    // position on greatest lower bound of key
    bTreeReader->searchForKey(
        bTreeTupleData, DUP_SEEK_BEGIN, false);
    if (bTreeReader->isSingular()) {
        return false;
    }
    bTreeReader->getTupleAccessorForRead().unmarshal(bTreeTupleData);

    LcsRid key = readRid();
    assert(key <= rid);
    return true;
}

void LcsClusterReader::getNextBTreeEntry

(

)

[private]

Reads the next entry in the rid-to-pageId btree map.

Definition at line 377 of file LcsClusterReader.cpp.

References bTreeReader, LcsClusterAccessBase::bTreeTupleData, nextBTreeEntry, NULL_PAGE_ID, LcsClusterAccessBase::readClusterPageId(), and LcsClusterAccessBase::readRid().

Referenced by getNextPageForPrefetch().

{
    bool rc = bTreeReader->searchNext();
    if (!rc) {
        // Indicate that there are no more pages
        nextBTreeEntry.first = NULL_PAGE_ID;
    } else {
        bTreeReader->getTupleAccessorForRead().unmarshal(bTreeTupleData);
        nextBTreeEntry.first = readClusterPageId();
        nextBTreeEntry.second = readRid();
    }
}

void LcsClusterReader::setRootPageId

(

PageId

rootPageId

)

Sets the root pageId of the underlying btree corresponding to the cluster.

Parameters:

rootPageId

the root pageId that's set

Definition at line 53 of file LcsClusterReader.cpp.

References bTreeReader.

{
    bTreeReader->setRootPageId(rootPageId);
}

void LcsClusterReader::initColumnReaders	(	uint	nClusterColsInit,
		TupleProjection const &	clusterProj
	)

Initializes cluster reader with column readers.

Parameters:

	nClusterColsInit	total number of columns in the cluster
	clusterProj	list of columns to be read from cluster; column numbers are 0-based relative to the cluster

Definition at line 100 of file LcsClusterReader.cpp.

References clusterCols, LcsClusterAccessBase::nClusterCols, and nColsToRead.

{
    nClusterCols = nClusterColsInit;
    nColsToRead = clusterProj.size();
    clusterCols.reset(new LcsColumnReader[nColsToRead]);
    for (uint i = 0; i < nColsToRead; i++) {
        clusterCols[i].init(this, clusterProj[i]);
    }
}

void LcsClusterReader::open

(

)

Initializes state variables used by cluster reader.

Definition at line 112 of file LcsClusterReader.cpp.

References dumbPrefetch, MAXU, nextBTreeEntry, nextPrefetchEntry, nextRid, pLeaf, pRangeBatches, CircularBufferIter< T >::reset(), and ridRunIter.

{
    pLeaf = NULL;
    pRangeBatches = NULL;
    ridRunIter.reset();

    // values of (0, MAXU) indicate that entry has not been filled in yet
    nextBTreeEntry.first = PageId(0);
    nextBTreeEntry.second = LcsRid(MAXU);

    nextPrefetchEntry.first = PageId(0);
    nextPrefetchEntry.second = LcsRid(MAXU);

    dumbPrefetch = false;
    nextRid = LcsRid(0);
}

void LcsClusterReader::close

(

)

Performs shutdown on cluster reader.

Definition at line 129 of file LcsClusterReader.cpp.

References bTreeReader, and LcsClusterAccessBase::unlockClusterPage().

{
    bTreeReader->endSearch();
    unlockClusterPage();
}

bool LcsClusterReader::getFirstClusterPageForRead

(

PConstLcsClusterNode &

pBlock

)

Gets first page in a cluster.

Parameters:

pBlock

output param returning cluster page

Returns:

true if page available

Definition at line 71 of file LcsClusterReader.cpp.

References bTreeReader, and readClusterPage().

Referenced by LcsClusterAppendExecStreamTest::verifyClusterPages().

{
    bool found;

    found = bTreeReader->searchFirst();
    if (!found) {
        return false;
    }

    LcsClusterNode const &node = readClusterPage();
    pBlock = &node;
    return true;
}

bool LcsClusterReader::getNextClusterPageForRead

(

PConstLcsClusterNode &

pBlock

)

Gets next page in a cluster, based on current position in btree.

Parameters:

pBlock

output param returning cluster page

Returns:

true if page available

Definition at line 86 of file LcsClusterReader.cpp.

References bTreeReader, and readClusterPage().

Referenced by LcsClusterAppendExecStreamTest::verifyClusterPages().

{
    bool found;

    found = bTreeReader->searchNext();
    if (!found) {
        return false;
    }

    LcsClusterNode const &node = readClusterPage();
    pBlock = &node;
    return true;
}

bool LcsClusterReader::isPositioned

(

)

const [inline]

Returns true if positioned within some range in a batch.

Definition at line 424 of file LcsClusterReader.h.

References pRangeBatches.

{
    return pRangeBatches != NULL;
}

LcsRid LcsClusterReader::getRangeStartRid

(

)

const [inline]

Returns first rid in a range.

Definition at line 429 of file LcsClusterReader.h.

References rangeStartRid.

Referenced by getCurrentRid().

{
    return rangeStartRid;
}

LcsRid LcsClusterReader::getRangeEndRid

(

)

const [inline]

Returns first rid after the end of the current range.

Definition at line 434 of file LcsClusterReader.h.

References rangeEndRid.

Referenced by nextRange(), and positionInBlock().

{
    return rangeEndRid;
}

uint LcsClusterReader::getRangeSize

(

)

const [inline]

Returns number of rids in the range.

Definition at line 439 of file LcsClusterReader.h.

References LcsBatchDir::nRow, and pRangeBatches.

Referenced by getRangeRowsLeft().

{
    return pRangeBatches->nRow;
}

uint LcsClusterReader::getRangePos

(

)

const [inline]

Returns offset within the current range.

E.g., 0 if at first rid in range

Definition at line 444 of file LcsClusterReader.h.

References nRangePos.

Referenced by getCurrentRid(), LcsColumnReader::getCurrentValueCode(), LcsColumnReader::getFixedValue(), getRangeRowsLeft(), LcsColumnReader::getVariableValue(), and LcsColumnReader::readCompressedBatch().

{
    return nRangePos;
}

LcsRid LcsClusterReader::getCurrentRid

(

)

const [inline]

Returns rid currently positioned at.

Definition at line 449 of file LcsClusterReader.h.

References getRangePos(), and getRangeStartRid().

{
    return getRangeStartRid() + getRangePos();
}

uint LcsClusterReader::getRangeRowsLeft

(

)

const [inline]

Returns number of rids yet to be read.

(E.g., 1 if we're at the last rid)

Definition at line 454 of file LcsClusterReader.h.

References getRangePos(), and getRangeSize().

Referenced by LcsColumnReader::readCompressedBatch().

{
    return getRangeSize() - getRangePos();
}

bool LcsClusterReader::position

(

LcsRid

rid

)

Positions scan on the rid, moving to a new range if necessary.

Parameters:

rid

rid to position to

Returns:

false if rid is beyond end of cluster

Definition at line 135 of file LcsClusterReader.cpp.

References bTreeReader, currRid, SegPageEntryIter< EntryT >::mapRange(), moveToBlock(), moveToBlockWithRid(), noPrefetch, NULL_PAGE_ID, pLeaf, positionInBlock(), prefetchQueue, LcsClusterAccessBase::readClusterPageId(), searchForRid(), and LcsClusterAccessBase::segmentAccessor.

Referenced by nextRange().

{
    bool found;

    currRid = rid;
    if (pLeaf) {
        // Scan is already in progress. Try to find the row we want in the
        // current block.

        found = positionInBlock(rid);
        if (found) {
            return true;
        }
    } else {
        if (noPrefetch) {
            if (!bTreeReader->searchFirst()) {
                bTreeReader->endSearch();
            }
        } else {
            // Initiate the first set of pre-fetches.
            prefetchQueue.mapRange(segmentAccessor, NULL_PAGE_ID);
        }
    }

    if (noPrefetch) {
        found = searchForRid(rid);
        if (!found) {
            return false;
        }
        moveToBlock(readClusterPageId());
    } else {
        found = moveToBlockWithRid(rid);
        if (!found) {
            return false;
        }
    }

    found = positionInBlock(rid);
    // page ends before "rid"; we must be off the last block
    if (!found) {
        return false;
    }

    return true;
}

bool LcsClusterReader::nextRange

(

)

[inline]

Positions scan on the first rid of the next range.

Definition at line 459 of file LcsClusterReader.h.

References getRangeEndRid(), and position().

{
    return position(getRangeEndRid());
}

bool LcsClusterReader::advance

(

uint

nRids

)

Moves "nRids" forward in the current range.

Parameters:

nRids

number of rids to move forward

Returns:

false if we are at the endof the range and therefore cannot advance the desired number of rids

Definition at line 295 of file LcsClusterReader.cpp.

References nRangePos.

{
    uint newPos = nRangePos + nRids;

    if (newPos < pRangeBatches->nRow) {
        nRangePos = newPos;
        return true;
    } else {
        return false;
    }
}

void LcsClusterReader::advanceWithinBatch

(

uint

nRids

)

[inline]

Advances nRids forward in the current batch.

Parameters:

nRids

number of rids to advance

Definition at line 464 of file LcsClusterReader.h.

References nRangePos.

{
    nRangePos = nRangePos + nRids;
}

PageId LcsClusterReader::getNextPageForPrefetch	(	LcsRid &	rid,
		bool &	found
	)			[virtual]

Determines the next page to be pre-fetched based on the rids that need to be read.

Parameters:

[out]	rid	returns the rid corresponding to the page to be pre-fetched
[out]	found	true if a pre-fetch page was found

Returns:

the page to be pre-fetched next

Implements SegPageEntryIterSource< LcsRid >.

Definition at line 307 of file LcsClusterReader.cpp.

References currRid, CircularBufferIter< T >::done(), dumbPrefetch, getFetchRids(), getNextBTreeEntry(), MAXU, nextBTreeEntry, nextRid, NULL_PAGE_ID, LcsClusterAccessBase::readClusterPageId(), LcsClusterAccessBase::readRid(), ridRunIter, and searchForRid().

{
    found = true;
    for (;;) {
        if (dumbPrefetch) {
            rid = currRid;
        } else {
            rid = getFetchRids(ridRunIter, nextRid, false);
        }

        // If the rid run buffer is exhausted and there are still rid runs
        // to be filled in, we're pre-fetching faster than we can fill up
        // the rid run buffers.  So, switch to dumb pre-fetch in that case.
        if (rid == LcsRid(MAXU)) {
            if (ridRunIter.done()) {
                rid = LcsRid(0);
                return NULL_PAGE_ID;
            } else {
                // TODO - Use stricter criteria on when to switch to dumb
                // pre-fetch.  E.g., if there are less than a certain number
                // of pages already pre-fetched.
                dumbPrefetch = true;
                continue;
            }
        }

        if (!(nextBTreeEntry.second == LcsRid(MAXU) &&
            nextBTreeEntry.first == PageId(0)))
        {
            // If we hit the end of the btree on the last iteration, then
            // there are no more pages.
            if (nextBTreeEntry.first == NULL_PAGE_ID) {
                rid = LcsRid(0);
                return NULL_PAGE_ID;
            }

            // If the next rid to be read is on the same page as the last
            // one located, bump up the rid so we move over to the next page.
            // In the case of dumb pre-fetch, just use the next page in the
            // cluster sequence.
            if (rid < nextBTreeEntry.second) {
                if (dumbPrefetch) {
                    rid = nextBTreeEntry.second;
                    PageId pageId = nextBTreeEntry.first;
                    getNextBTreeEntry();
                    return pageId;
                }
                nextRid = nextBTreeEntry.second;
                continue;
            } else {
                // TODO - optimize by avoiding search from top of btree if
                // desired entry is within a few entries of the current
                break;
           }
        } else {
            // We haven't located any pages yet.  Initiate first btree search.
            break;
        }
    }

    bool rc = searchForRid(rid);
    if (!rc) {
        return NULL_PAGE_ID;
    }
    rid = readRid();
    PageId pageId = readClusterPageId();
    getNextBTreeEntry();
    return pageId;
}

LcsRid LcsClusterReader::getFetchRids	(	CircularBufferIter< LcsRidRun > &	ridRunIter,
		LcsRid &	nextRid,
		bool	remove
	)			[static]

Retrieves the current and next rids that need to be read, based on the current rid run being processed.

Parameters:

	ridRunIter	iterator over the circular buffer that provides the rid runs
[in,out]	nextRid	on input, indicates the minimum rid value to be retrieved; on return, the next rid that will need to be read
	remove	if true, remove the first element from the circular buffer if the rid run buffer's position is incremented

Returns:

the current rid that needs to be read

Definition at line 390 of file LcsClusterReader.cpp.

References CircularBufferIter< T >::end(), MAXU, LcsRidRun::nRids, CircularBufferIter< T >::removeFront(), ridRunIter, and LcsRidRun::startRid.

Referenced by LcsRowScanExecStream::execute(), and getNextPageForPrefetch().

{
    for (;;) {
        if (ridRunIter.end()) {
            return LcsRid(MAXU);
        }

        LcsRidRun &currRidRun = *ridRunIter;
        if (nextRid < currRidRun.startRid) {
            nextRid = currRidRun.startRid + 1;
            return currRidRun.startRid;
        } else if (
            (currRidRun.nRids == RecordNum(MAXU) &&
                nextRid >= currRidRun.startRid) ||
            (nextRid < currRidRun.startRid + currRidRun.nRids))
        {
            return nextRid++;
        } else {
            ++ridRunIter;
            if (remove) {
                ridRunIter.removeFront();
            }
        }
    }
}

void LcsClusterReader::catchUp	(	uint	parentBufPos,
		LcsRid	parentNextRid
	)

Resynchronizes the cluster reader's prefetch position to that of the parent scan.

Parameters:

	parentBufPos	buffer position of the parent scan
	parentNextRid	next rid of the parent scan

Definition at line 419 of file LcsClusterReader.cpp.

References CircularBufferIter< T >::getCurrPos(), nextRid, ridRunIter, and CircularBufferIter< T >::setCurrPos().

{
    if (parentNextRid - 1 > nextRid) {
        nextRid = parentNextRid - 1;
    }
    if (parentBufPos > ridRunIter.getCurrPos()) {
        ridRunIter.setCurrPos(parentBufPos);
    }
}

RecordNum LcsClusterReader::getNumRows

(

)

Returns:

the number of rows in the cluster

Definition at line 429 of file LcsClusterReader.cpp.

References bTreeReader, LcsClusterAccessBase::bTreeTupleData, LcsClusterNode::firstRID, LcsClusterNode::nBatch, LcsClusterAccessBase::nClusterCols, LcsClusterNode::oBatch, opaqueToInt(), and readClusterPage().

{
    // Read the last cluster page
    if (bTreeReader->searchLast() == false) {
        bTreeReader->endSearch();
        return RecordNum(0);
    }
    bTreeReader->getTupleAccessorForRead().unmarshal(bTreeTupleData);
    LcsClusterNode const &node = readClusterPage();

    // Then count the number of rows in each batch on that page
    RecordNum nRows = RecordNum(opaqueToInt(node.firstRID));
    PLcsBatchDir pBatch = (PLcsBatchDir) ((PBuffer) &node + node.oBatch);
    for (uint i = 0; i < node.nBatch; i += nClusterCols) {
        nRows += pBatch[i].nRow;
    }

    bTreeReader->endSearch();
    return nRows;
}

LcsRid LcsClusterAccessBase::readRid

(

)

[protected, inherited]

Returns RID from btree tuple.

Definition at line 37 of file LcsClusterAccessBase.cpp.

References LcsClusterAccessBase::bTreeTupleData.

Referenced by getNextBTreeEntry(), getNextPageForPrefetch(), readClusterPage(), and searchForRid().

{
    return *reinterpret_cast<LcsRid const *> (bTreeTupleData[0].pData);
}

PageId LcsClusterAccessBase::readClusterPageId

(

)

[protected, inherited]

Returns cluster pageid from btree tuple.

Definition at line 42 of file LcsClusterAccessBase.cpp.

References LcsClusterAccessBase::bTreeTupleData.

Referenced by LcsClusterNodeWriter::getLastClusterPageForWrite(), getNextBTreeEntry(), getNextPageForPrefetch(), position(), and readClusterPage().

{
    return *reinterpret_cast<PageId const *> (bTreeTupleData[1].pData);
}

void LcsClusterAccessBase::setHdrOffsets

(

PConstLcsClusterNode

pHdr

)

[protected, inherited]

Sets pointers to offset arrays in cluster page header.

Parameters:

pHdr

pointer to cluster node header

Definition at line 47 of file LcsClusterAccessBase.cpp.

References LcsClusterAccessBase::delta, LcsClusterAccessBase::firstVal, LcsClusterAccessBase::lastVal, LcsClusterAccessBase::nClusterCols, and LcsClusterAccessBase::nVal.

Referenced by LcsClusterDump::dump(), LcsClusterNodeWriter::init(), and setUpBlock().

{
    PBuffer p = (PBuffer) pHdr;

    lastVal = (uint16_t *) (p += sizeof(LcsClusterNode));
    firstVal = (uint16_t *) (p += sizeof(uint16_t) * nClusterCols);
    nVal = (uint *) (p += sizeof(uint16_t) * nClusterCols);
    delta = (uint16_t *) (p += sizeof(uint) * nClusterCols);
}

uint LcsClusterAccessBase::getNumClusterCols

(

)

[inline, inherited]

Returns number of columns in cluster.

Definition at line 118 of file LcsClusterAccessBase.h.

    {
        return nClusterCols;
    }

void LcsClusterAccessBase::setNumClusterCols

(

uint

nCols

)

[inline, inherited]

Sets number of columns in cluster.

Definition at line 126 of file LcsClusterAccessBase.h.

    {
        nClusterCols = nCols;
    }

void LcsClusterAccessBase::unlockClusterPage

(

)

[inherited]

Unlocks cluster page.

Definition at line 57 of file LcsClusterAccessBase.cpp.

References LcsClusterAccessBase::clusterLock, and SegPageLock::unlock().

Referenced by close().

{
    clusterLock.unlock();
}

virtual void SegPageEntryIterSource< LcsRid >::initPrefetchEntry

(

LcsRid &

entry

)

[inline, virtual, inherited]

Initializes a specific entry in the pre-fetch queue.

Parameters:

entry

the entry that will be initialized

Definition at line 46 of file SegPageEntryIterSource.h.

    {
    }

---

Friends And Related Function Documentation

friend class LcsColumnReader [friend]

Definition at line 87 of file LcsClusterReader.h.

---

Member Data Documentation

SharedBTreeReader LcsClusterReader::bTreeReader [private]

Reads btree corresponding to cluster.

Definition at line 92 of file LcsClusterReader.h.

Referenced by close(), getFirstClusterPageForRead(), getNextBTreeEntry(), getNextClusterPageForRead(), getNumRows(), LcsClusterReader(), position(), readClusterPage(), searchForRid(), and setRootPageId().

PLcsBatchDir LcsClusterReader::pRangeBatches [private]

Pointer to first batch in the current range.

Definition at line 97 of file LcsClusterReader.h.

Referenced by getRangeSize(), isPositioned(), open(), positionInBlock(), setUpBlock(), and LcsColumnReader::sync().

LcsRid LcsClusterReader::rangeStartRid [private]

First rid in the current range.

Definition at line 102 of file LcsClusterReader.h.

Referenced by getRangeStartRid(), positionInBlock(), and setUpBlock().

LcsRid LcsClusterReader::rangeEndRid [private]

1 rid past the last rid in the current range

Definition at line 107 of file LcsClusterReader.h.

Referenced by getRangeEndRid(), positionInBlock(), and setUpBlock().

uint LcsClusterReader::nRangePos [private]

Offset from start of current range.

Definition at line 112 of file LcsClusterReader.h.

Referenced by advance(), advanceWithinBatch(), getRangePos(), and setUpBlock().

PBuffer LcsClusterReader::pLeaf [private]

Pointer to current cluster block.

Definition at line 117 of file LcsClusterReader.h.

Referenced by open(), position(), setUpBlock(), and LcsColumnReader::sync().

PConstLcsClusterNode LcsClusterReader::pLHdr [private]

Pointer to header of cluster block.

Definition at line 122 of file LcsClusterReader.h.

Referenced by moveToBlock(), moveToBlockWithRid(), and setUpBlock().

PLcsBatchDir LcsClusterReader::pBatches [private]

Batch directory.

Definition at line 127 of file LcsClusterReader.h.

Referenced by setUpBlock().

bool LcsClusterReader::noPrefetch [private]

If true, do not use pre-fetch.

Definition at line 132 of file LcsClusterReader.h.

Referenced by LcsClusterReader(), and position().

CircularBufferIter<LcsRidRun> LcsClusterReader::ridRunIter [private]

Iterator over the circular buffer containing rid runs.

Definition at line 137 of file LcsClusterReader.h.

Referenced by catchUp(), getFetchRids(), getNextPageForPrefetch(), and open().

SegPageEntryIter<LcsRid> LcsClusterReader::prefetchQueue [private]

Pre-fetch queue for pages read from this cluster.

Definition at line 142 of file LcsClusterReader.h.

Referenced by LcsClusterReader(), moveToBlockWithRid(), and position().

LcsRid LcsClusterReader::nextRid [private]

The next rid that needs to be read.

Definition at line 147 of file LcsClusterReader.h.

Referenced by catchUp(), getNextPageForPrefetch(), and open().

std::pair<PageId, LcsRid> LcsClusterReader::nextBTreeEntry [private]

The next entry in the rid-to-pageId btree map following the one that contains the last page that was pre-fetched.

Definition at line 153 of file LcsClusterReader.h.

Referenced by getNextBTreeEntry(), getNextPageForPrefetch(), and open().

std::pair<PageId, LcsRid> LcsClusterReader::nextPrefetchEntry [private]

The next entry from the pre-fetch queue.

Definition at line 158 of file LcsClusterReader.h.

Referenced by moveToBlockWithRid(), and open().

bool LcsClusterReader::dumbPrefetch [private]

If true, switch has been made to dumb pre-fetch.

Definition at line 163 of file LcsClusterReader.h.

Referenced by getNextPageForPrefetch(), and open().

LcsRid LcsClusterReader::currRid [private]

The current rid that needs to be read.

Definition at line 168 of file LcsClusterReader.h.

Referenced by getNextPageForPrefetch(), and position().

uint LcsClusterReader::nColsToRead

Number of cluster columns that will be read.

Definition at line 238 of file LcsClusterReader.h.

Referenced by initColumnReaders().

boost::scoped_array<LcsColumnReader> LcsClusterReader::clusterCols

Column readers for each cluster column that will be read.

Definition at line 243 of file LcsClusterReader.h.

Referenced by initColumnReaders().

TupleData LcsClusterAccessBase::bTreeTupleData [protected, inherited]

Tuple data representing the btree key corresponding to the cluster page.

Definition at line 42 of file LcsClusterAccessBase.h.

Referenced by LcsClusterNodeWriter::allocateClusterPage(), LcsClusterNodeWriter::getLastClusterPageForWrite(), getNextBTreeEntry(), getNumRows(), LcsClusterAccessBase::LcsClusterAccessBase(), readClusterPage(), LcsClusterAccessBase::readClusterPageId(), LcsClusterAccessBase::readRid(), and searchForRid().

SegmentAccessor LcsClusterAccessBase::segmentAccessor [protected, inherited]

Accessor for segment storing both btree and cluster pages.

Definition at line 47 of file LcsClusterAccessBase.h.

Referenced by LcsClusterNodeWriter::allocateClusterPage(), LcsClusterAccessBase::LcsClusterAccessBase(), and position().

ClusterPageLock LcsClusterAccessBase::clusterLock [protected, inherited]

Buffer lock for the actual cluster node pages.

Shares the same segment as the btree corresponding to the cluster.

Definition at line 53 of file LcsClusterAccessBase.h.

Referenced by LcsClusterNodeWriter::allocateClusterPage(), LcsClusterNodeWriter::close(), LcsClusterNodeWriter::getLastClusterPageForWrite(), LcsClusterAccessBase::LcsClusterAccessBase(), moveToBlock(), moveToBlockWithRid(), readClusterPage(), and LcsClusterAccessBase::unlockClusterPage().

PageId LcsClusterAccessBase::clusterPageId [protected, inherited]

Current cluster pageid.

Definition at line 58 of file LcsClusterAccessBase.h.

Referenced by LcsClusterNodeWriter::allocateClusterPage(), LcsClusterNodeWriter::getLastClusterPageForWrite(), LcsClusterVerifier::getPageData(), LcsClusterNodeWriter::moveFromTempToIndex(), moveToBlockWithRid(), and readClusterPage().

LcsRid LcsClusterAccessBase::bTreeRid [protected, inherited]

Current rid in btree used to access current cluster page.

Definition at line 63 of file LcsClusterAccessBase.h.

Referenced by LcsClusterNodeWriter::allocateClusterPage(), LcsClusterVerifier::getPageData(), and readClusterPage().

uint LcsClusterAccessBase::nClusterCols [protected, inherited]

Number of columns in cluster.

Definition at line 68 of file LcsClusterAccessBase.h.

Referenced by LcsClusterNodeWriter::addValue(), LcsClusterNodeWriter::allocArrays(), LcsClusterDump::dump(), getNumRows(), LcsClusterNodeWriter::init(), initColumnReaders(), LcsClusterNodeWriter::LcsClusterNodeWriter(), LcsClusterNodeWriter::moveFromIndexToTemp(), LcsClusterNodeWriter::moveFromTempToIndex(), LcsClusterNodeWriter::openAppend(), LcsClusterNodeWriter::openNew(), positionInBlock(), and LcsClusterAccessBase::setHdrOffsets().

uint16_t* LcsClusterAccessBase::lastVal [protected, inherited]

Offsets to the last value stored on the page for each column in cluster.

Definition at line 74 of file LcsClusterAccessBase.h.

Referenced by LcsClusterNodeWriter::addValue(), LcsClusterDump::dump(), LcsClusterNodeWriter::moveFromIndexToTemp(), LcsClusterNodeWriter::moveFromTempToIndex(), LcsClusterNodeWriter::openAppend(), LcsClusterNodeWriter::openNew(), LcsClusterNodeWriter::pickCompressionMode(), LcsClusterNodeWriter::putCompressedBatch(), LcsClusterNodeWriter::putFixedVarBatch(), LcsClusterNodeWriter::rollBackLastBatch(), LcsClusterAccessBase::setHdrOffsets(), and LcsClusterNodeWriter::undoValue().

uint16_t* LcsClusterAccessBase::firstVal [protected, inherited]

Offsets to the first value stored on the page for each column in cluster.

Points to the end of the firstVal, so subtracting lastVal from firstVal will tell you the number of bytes taken up by values for a column, since lastVals are appended in front of firstVal

Definition at line 82 of file LcsClusterAccessBase.h.

Referenced by LcsClusterDump::dump(), LcsClusterNodeWriter::moveFromIndexToTemp(), LcsClusterNodeWriter::moveFromTempToIndex(), LcsClusterNodeWriter::openNew(), and LcsClusterAccessBase::setHdrOffsets().

uint* LcsClusterAccessBase::nVal [protected, inherited]

Number of distinct values in the page for each column in cluster.

Definition at line 87 of file LcsClusterAccessBase.h.

Referenced by LcsClusterNodeWriter::addValue(), LcsClusterDump::dump(), LcsClusterNodeWriter::moveFromIndexToTemp(), LcsClusterNodeWriter::moveFromTempToIndex(), LcsClusterNodeWriter::openAppend(), LcsClusterNodeWriter::openNew(), LcsClusterNodeWriter::pickCompressionMode(), LcsClusterNodeWriter::putCompressedBatch(), LcsClusterNodeWriter::putFixedVarBatch(), LcsClusterNodeWriter::rollBackLastBatch(), LcsClusterAccessBase::setHdrOffsets(), and LcsClusterNodeWriter::undoValue().

uint16_t* LcsClusterAccessBase::delta [protected, inherited]

For each column in the cluster, offset used to get the real offset within the page.

Definition at line 93 of file LcsClusterAccessBase.h.

Referenced by LcsClusterDump::dump(), LcsClusterNodeWriter::moveFromTempToIndex(), LcsClusterNodeWriter::openNew(), LcsClusterAccessBase::setHdrOffsets(), and LcsColumnReader::sync().

---

The documentation for this class was generated from the following files:

• /home/pub/open/dev/fennel/lucidera/colstore/LcsClusterReader.h
• /home/pub/open/dev/fennel/lucidera/colstore/LcsClusterReader.cpp

---