LcsClusterNodeWriter Class Reference

Constructs a cluster page, managing the amount of space currently in use on the page and determining the offsets where different elements are to be stored. More...

#include <LcsClusterNodeWriter.h>

Inheritance diagram for LcsClusterNodeWriter:

List of all members.


Public Member Functions
	LcsClusterNodeWriter (BTreeDescriptor const &treeDescriptorInit, SegmentAccessor const &accessorInit, TupleDescriptor const &colTupleDescInit, SharedTraceTarget pTraceTargetInit, std::string nameInit)
	~LcsClusterNodeWriter ()
bool	getLastClusterPageForWrite (PLcsClusterNode &pBlock, LcsRid &firstRid)
	Gets the last cluster page.
PLcsClusterNode	allocateClusterPage (LcsRid firstRid)
	Allocates a new cluster page.
void	init (uint nColumns, PBuffer indexBlock, PBuffer *pBlock, uint szBlock)
	Initializes object with parameters relevant to the cluster page that will be written.
void	close ()
void	openNew (LcsRid startRID)
	Prepares a cluster page as a new one.
bool	openAppend (uint nValOffsets, uint16_t lastValOffsets, RecordNum &nrows)
	Prepares an existing cluster page for appending new data, and determines whether the page is already full and cannot accomodate any more data.
void	describeLastBatch (uint column, uint &dRow, uint &recSize)
	Returns parameters describing the last batch for a given column.
uint16_t	getNextVal (uint column, uint16_t thisVal)
	Returns the offset of the next value in a batch.
void	rollBackLastBatch (uint column, PBuffer pVal)
	Rolls back the last 8 value (or less) from a batch.
bool	noCompressMode (uint column) const
	Returns true if the batch is not being forced to compress mode.
PBuffer	getOffsetPtr (uint column, uint16_t offset)
	Translates an offset for a column to the pointer to the actual value.
bool	addValue (uint column, bool bFirstTimeInBatch)
	Adds a value to the page, in the case where the value already exists in the column.
bool	addValue (uint column, PBuffer pVal, uint16_t *oVal)
	Adds a new value to the page.
void	undoValue (uint column, PBuffer pVal, bool bFirstInBatch)
	Undoes the last value added to the current batch for a column.
void	putCompressedBatch (uint column, PBuffer pRows, PBuffer pBuf)
	Writes a compressed mode batch into the temporary cluster page for a column.
void	putFixedVarBatch (uint column, uint16_t *pRows, PBuffer pBuf)
	Writes a fixed or variable mode batch into a temporary cluster page for a column.
void	pickCompressionMode (uint column, uint fixedSize, uint nRows, uint16_t **pValOffset, LcsBatchMode &compressionMode)
	Determines which compression mode to use for a batch.
bool	isEndOfBlock ()
	Returns true if there is no space left in the cluster page.
void	endBlock ()
	Done with the current cluster page.
uint	getNumClusterCols ()
	Returns number of columns in cluster.
void	setNumClusterCols (uint nCols)
	Sets number of columns in cluster.
void	unlockClusterPage ()
	Unlocks cluster page.
virtual void	initTraceSource (SharedTraceTarget pTraceTarget, std::string name)
	For use when initialization has to be deferred until after construction.
void	trace (TraceLevel level, std::string message) const
	Records a trace message.
bool	isTracing () const
	Returns: true iff tracing is enabled for this source
bool	isTracingLevel (TraceLevel level) const
	Determines whether a particular level is being traced.
TraceTarget &	getTraceTarget () const
	Returns: the TraceTarget for this source
SharedTraceTarget	getSharedTraceTarget () const
	Returns: the SharedTraceTarget for this source
std::string	getTraceSourceName () const
	Gets the name of this source.
void	setTraceSourceName (std::string const &n)
	Sets the name of this source.
TraceLevel	getMinimumTraceLevel () const
void	disableTracing ()
Protected Member Functions
LcsRid	readRid ()
	Returns RID from btree tuple.
PageId	readClusterPageId ()
	Returns cluster pageid from btree tuple.
void	setHdrOffsets (PConstLcsClusterNode pHdr)
	Sets pointers to offset arrays in cluster page header.
Protected Attributes
TupleData	bTreeTupleData
	Tuple data representing the btree key corresponding to the cluster page.
SegmentAccessor	segmentAccessor
	Accessor for segment storing both btree and cluster pages.
ClusterPageLock	clusterLock
	Buffer lock for the actual cluster node pages.
PageId	clusterPageId
	Current cluster pageid.
LcsRid	bTreeRid
	Current rid in btree used to access current cluster page.
uint	nClusterCols
	Number of columns in cluster.
uint16_t *	lastVal
	Offsets to the last value stored on the page for each column in cluster.
uint16_t *	firstVal
	Offsets to the first value stored on the page for each column in cluster.
uint *	nVal
	Number of distinct values in the page for each column in cluster.
uint16_t *	delta
	For each column in the cluster, offset used to get the real offset within the page.
Private Member Functions
PBuffer	valueSource (uint16_t lastValOffset, PBuffer pValBank, uint16_t oValBank, PBuffer pBlock, uint16_t f)
	Associates an offset with an address, determining whether a value is stored in the temporary block or the temporary value bank.
RecordNum	moveFromIndexToTemp ()
	Moves all cluster data from cluster page to temporary storage.
void	moveFromTempToIndex ()
	Moves all cluster data from temporary storage to the actual cluster page.
void	allocArrays ()
	Allocates temporary arrays used during cluster writes.
uint32_t	round8Boundary (uint32_t val)
	Rounds a 32-bit value to a boundary of 8.
uint32_t	roundIf8Boundary (uint32_t val)
	Rounds a 32-bit value to a boundary of 8 if it is > 8.
Private Attributes
SharedBTreeWriter	bTreeWriter
	Writes btree corresponding to cluster.
SegmentAccessor	scratchAccessor
	Accessor for scratch segments.
ClusterPageLock	bufferLock
	Lock on scratch page.
PLcsClusterNode	pHdr
	Cluster page header.
uint	hdrSize
	Size of the cluster page header.
PBuffer	pIndexBlock
	Cluster page to be written.
PBuffer *	pBlock
	Array of pointers to temporary blocks, 1 block for each column cluster.
uint	szBlock
	Size of the cluster page.
int	minSzLeft
	Minimum size left on the page.
boost::scoped_array< LcsBatchDir >	batchDirs
	Batch directories for the batches currently being constructed, one per cluster column.
boost::scoped_array< PBuffer >	pValBank
	Temporary storage for values, used for fixed mode batches; one per cluster column.
boost::scoped_array< uint16_t >	oValBank
	First offset in the bank for each column in the cluster value bank.
boost::scoped_array< uint16_t >	valBankStart
	Start of each cluster column in the value bank.
boost::scoped_array< uint16_t >	batchOffset
	Offsets to the batch directories on the temporary pages, one per cluster column.
boost::scoped_array< uint >	batchCount
	Count of the number of batches in the temporary pages, one per cluster column.
int	szLeft
	Number of bytes left on the page.
boost::scoped_array< uint >	nBits
	Number of bits required to store the value codes for each column in the cluster, for the batches currently being constructed.
boost::scoped_array< uint >	nextWidthChange
	Number of values that will cause the next nBit change for the column in the cluster.
bool	arraysAllocated
	Indicates whether temporary arrays have already been allocated.
boost::scoped_array< ForceMode >	bForceMode
	Set when the mode of a batch should be forced to a particular value.
boost::scoped_array< uint >	forceModeCount
	Number of times force mode has been used for each cluster column.
boost::scoped_array< uint >	maxValueSize
	Max value size encountered thus far for each cluster column.
boost::scoped_array< UnalignedAttributeAccessor >	attrAccessors
	Accessors for reading unaligned values.
SharedLcsClusterDump	clusterDump
	Cluster dump.
TupleDescriptor	colTupleDesc
	Tuple descriptor of the columns being loaded.

Detailed Description

Constructs a cluster page, managing the amount of space currently in use on the page and determining the offsets where different elements are to be stored.

Definition at line 46 of file LcsClusterNodeWriter.h.

Constructor & Destructor Documentation

LcsClusterNodeWriter::LcsClusterNodeWriter	(	BTreeDescriptor const &	treeDescriptorInit,
		SegmentAccessor const &	accessorInit,
		TupleDescriptor const &	colTupleDescInit,
		SharedTraceTarget	pTraceTargetInit,
		std::string	nameInit
	)			`[explicit]`

Definition at line 29 of file LcsClusterNodeWriter.cpp.

References SegPageLock::accessSegment(), arraysAllocated, batchCount, batchDirs, batchOffset, bForceMode, bTreeWriter, bufferLock, clusterDump, colTupleDesc, forceModeCount, hdrSize, maxValueSize, minSzLeft, nBits, LcsClusterAccessBase::nClusterCols, nextWidthChange, oValBank, pBlock, pHdr, pIndexBlock, pValBank, scratchAccessor, szBlock, szLeft, TRACE_FINE, and valBankStart.

00034                         :
00035         LcsClusterAccessBase(treeDescriptorInit),
00036         TraceSource(pTraceTargetInit, nameInit)
00037 {
00038     scratchAccessor = accessorInit;
00039     bufferLock.accessSegment(scratchAccessor);
00040     bTreeWriter = SharedBTreeWriter(
00041         new BTreeWriter(treeDescriptorInit, scratchAccessor, true));
00042     colTupleDesc = colTupleDescInit;
00043     clusterDump =
00044         SharedLcsClusterDump(
00045             new LcsClusterDump(
00046                 treeDescriptorInit,
00047                 colTupleDesc,
00048                 TRACE_FINE,
00049                 pTraceTargetInit,
00050                 nameInit));
00051     nClusterCols = 0;
00052     pHdr = 0;
00053     hdrSize = 0;
00054     pIndexBlock = 0;
00055     pBlock = 0;
00056     szBlock = 0;
00057     minSzLeft = 0;
00058     batchDirs.reset();
00059     pValBank.reset();
00060     oValBank.reset();
00061     batchOffset.reset();
00062     batchCount.reset();
00063     szLeft = 0;
00064     nBits.reset();
00065     nextWidthChange.reset();
00066     arraysAllocated = false;
00067     valBankStart.reset();
00068     bForceMode.reset();
00069     forceModeCount.reset();
00070     maxValueSize.reset();
00071 }

LcsClusterNodeWriter::~LcsClusterNodeWriter ( )

Definition at line 73 of file LcsClusterNodeWriter.cpp.

References close().

00074 {
00075     close();
00076 }

Member Function Documentation

PBuffer LcsClusterNodeWriter::valueSource	(	uint16_t	lastValOffset,
		PBuffer	pValBank,
		uint16_t	oValBank,
		PBuffer	pBlock,
		uint16_t	f
	)			`[inline, private]`

Associates an offset with an address, determining whether a value is stored in the temporary block or the temporary value bank.

Parameters:

	lastValOffset	offset of the last value for this particular column
	pValBank	buffer storing values in the value bank
	oValBank	offset of first value for column in the value bank
	pBlock	temporary block for column
	f	desired offset

Returns:: address corresponding to offset

Definition at line 198 of file LcsClusterNodeWriter.h.

Referenced by putFixedVarBatch().

00201     {
00202         // if value not in back use
00203         if (f < lastValOffset) {
00204             return pValBank + f - oValBank;
00205         } else {
00206             return pBlock + f;
00207         }
00208     }

RecordNum LcsClusterNodeWriter::moveFromIndexToTemp ( ) [private]

Moves all cluster data from cluster page to temporary storage.

Returns:: number of rows currently on page

Definition at line 961 of file LcsClusterNodeWriter.cpp.

References batchCount, batchOffset, bitVecPtr(), bitVecWidth(), calcWidth(), LcsClusterAccessBase::firstVal, hdrSize, LcsClusterAccessBase::lastVal, LCS_COMPRESSED, LCS_VARIABLE, myCopy(), LcsClusterNode::nBatch, LcsClusterAccessBase::nClusterCols, LcsBatchDir::nRow, LcsClusterAccessBase::nVal, LcsBatchDir::nVal, LcsClusterNode::oBatch, LcsBatchDir::oVal, pBlock, pHdr, pIndexBlock, LcsBatchDir::recSize, and szBlock.

Referenced by openAppend().

00962 {
00963     PLcsBatchDir pBatch;
00964     boost::scoped_array<uint16_t> batchDirOffset;
00965     uint16_t loc;
00966     uint column;
00967     uint batchCount = pHdr->nBatch / nClusterCols;
00968     uint b;
00969 
00970     batchDirOffset.reset(new uint16_t[pHdr->nBatch]);
00971 
00972     // First move the values
00973     //
00974     // copy values from index for all columns starting with the
00975     // 1st column in cluster.
00976     for (column = 0; column < nClusterCols; column++) {
00977         uint sz = firstVal[column] - lastVal[column];
00978         loc = (uint16_t) (szBlock - sz);
00979         myCopy(pBlock[column] + loc, pIndexBlock + lastVal[column], sz);
00980 
00981         // adjust lastVal and firstVal to offset in temporary block
00982         lastVal[column]  = loc;
00983         firstVal[column] = (uint16_t) szBlock;
00984     }
00985 
00986     // Next move the batches
00987 
00988     pBatch = (PLcsBatchDir)(pIndexBlock + pHdr->oBatch);
00989     for (column = 0; column < nClusterCols; column++) {
00990         uint i;
00991         loc = hdrSize;
00992 
00993         // move every batch for this column
00994         for (b = column, i = 0; i < batchCount; i++, b = b + nClusterCols) {
00995             uint16_t    batchStart = loc;
00996 
00997             if (pBatch[b].mode == LCS_COMPRESSED) {
00998                 uint8_t     *pBit;
00999                 WidthVec    w;      // bitVec m_width vector
01000                 PtrVec      p;      // bitVec offsets
01001                 uint        iV;     // # of bit vectors
01002                 uint        sizeOffsets, nBytes;
01003 
01004                 //copy offsets
01005                 sizeOffsets =  pBatch[b].nVal * sizeof(uint16_t);
01006                 myCopy(
01007                     pBlock[column] + loc, pIndexBlock + pBatch[b].oVal,
01008                     sizeOffsets);
01009 
01010                 // step past offsets
01011                 loc = (uint16_t) (loc + sizeOffsets);
01012 
01013                 // calculate the bit vector widthes
01014                 iV = bitVecWidth(calcWidth(pBatch[b].nVal), w);
01015 
01016                 // this is where the bit vectors start
01017                 pBit = pIndexBlock + pBatch[b].oVal + sizeOffsets;
01018 
01019                 // nByte are taken by the bit vectors
01020                 nBytes = bitVecPtr(pBatch[b].nRow, iV, w, p, pBit);
01021 
01022                 myCopy(pBlock[column] + loc, pBit, nBytes);
01023 
01024                 // step past bit vectors
01025                 loc = (uint16_t) (loc + nBytes);
01026             } else if (pBatch[b].mode == LCS_VARIABLE) {
01027                 uint        sizeOffsets;
01028 
01029                 sizeOffsets = pBatch[b].nRow * sizeof(uint16_t);
01030 
01031                 // variable size record batch
01032                 myCopy(
01033                     pBlock[column] + loc, pIndexBlock + pBatch[b].oVal,
01034                     sizeOffsets);
01035 
01036                 // step past offsets
01037                 loc = (uint16_t) (loc + sizeOffsets);
01038             } else {
01039                 // fixed mode batch
01040                 uint sizeFixed;
01041 
01042                 sizeFixed =  pBatch[b].nRow * pBatch[b].recSize;
01043                 // fixed size record batch
01044                 myCopy(
01045                     pBlock[column] + loc, pIndexBlock + pBatch[b].oVal,
01046                     sizeFixed);
01047 
01048                 //step past fixed records
01049                 loc = (uint16_t) (loc + sizeFixed);
01050             }
01051 
01052             // set offset where values start in temp block
01053             batchDirOffset[b] = batchStart;
01054         }
01055 
01056         // move batch directories for this column
01057 
01058         uint16_t  dirLoc;
01059         b = column;
01060         dirLoc = loc;
01061         batchOffset[column] = dirLoc;
01062 
01063         // move every batch for this column
01064         for (i = 0; i < batchCount; i++) {
01065             PLcsBatchDir pTempBatch = (PLcsBatchDir)(pBlock[column] + dirLoc);
01066             myCopy(pTempBatch, &pBatch[b], sizeof(LcsBatchDir));
01067 
01068             pTempBatch->oVal = batchDirOffset[b];
01069             // increment to next batch and next location in temp block
01070             b = b + nClusterCols;
01071             dirLoc += sizeof(LcsBatchDir);
01072         }
01073     }
01074 
01075     // compute the number of rows on the page
01076     pBatch = (PLcsBatchDir)(pIndexBlock + pHdr->oBatch);
01077     RecordNum nrows = 0;
01078     for (b = 0; b < pHdr->nBatch; b = b + nClusterCols) {
01079         nrows += pBatch[b].nRow;
01080     }
01081 
01082     batchDirOffset.reset();
01083     return nrows;
01084 }

void LcsClusterNodeWriter::moveFromTempToIndex ( ) [private]

Moves all cluster data from temporary storage to the actual cluster page.

Definition at line 1086 of file LcsClusterNodeWriter.cpp.

References batchCount, batchOffset, bitVecPtr(), bitVecWidth(), calcWidth(), clusterDump, LcsClusterAccessBase::clusterPageId, LcsClusterAccessBase::delta, LcsClusterAccessBase::firstVal, hdrSize, TraceSource::isTracingLevel(), LcsClusterAccessBase::lastVal, LCS_COMPRESSED, LCS_VARIABLE, myCopy(), LcsClusterNode::nBatch, LcsClusterAccessBase::nClusterCols, LcsBatchDir::nRow, LcsClusterAccessBase::nVal, LcsBatchDir::nVal, LcsClusterNode::oBatch, opaqueToInt(), LcsBatchDir::oVal, pBlock, pHdr, pIndexBlock, LcsBatchDir::recSize, szBlock, and TRACE_FINE.

01087 {
01088     PLcsBatchDir pBatch;
01089     uint        sz, numBatches = batchCount[0];
01090     uint16_t    offset, loc;
01091     uint        column, b;
01092 
01093     // Copy values from temporary blocks for all columns starting with the
01094     // 1st column in cluster.
01095 
01096     for (offset = (uint16_t) szBlock, column = 0; column < nClusterCols;
01097         column++)
01098     {
01099         sz = szBlock - lastVal[column];
01100         myCopy(
01101             pIndexBlock + (offset - sz), pBlock[column] + lastVal[column], sz);
01102 
01103         //  set delta value to subtract from offsets to get relative offset
01104         delta[column] = (uint16_t)(szBlock - offset);
01105 
01106         // adjust firstVal and lastVal in the leaf block header to appropriate
01107         // offsets in index block (currently base on offsets in temporary block)
01108         firstVal[column] = offset;
01109         offset = (uint16_t) (offset - sz);
01110         lastVal[column] = offset;
01111     }
01112 
01113     // copy batch descriptors (which point to the batches)
01114 
01115     for (loc =  hdrSize, b = 0; b < numBatches; b++) {
01116         for (column = 0; column < nClusterCols; column++) {
01117             uint16_t    batchStart = loc;
01118 
01119             pBatch = (PLcsBatchDir)(pBlock[column] + batchOffset[column]);
01120 
01121             if (pBatch[b].mode == LCS_COMPRESSED) {
01122                 uint8_t     *pBit;
01123                 WidthVec    w;      // bitVec m_width vector
01124                 PtrVec      p;      // bitVec offsets
01125                 uint        iV;     // # of bit vectors
01126                 uint        sizeOffsets, nBytes;
01127 
01128                 sizeOffsets =  pBatch[b].nVal * sizeof(uint16_t);
01129 
01130                 // first copy offsets then bit vectors
01131                 myCopy(
01132                     pIndexBlock + loc, pBlock[column] + pBatch[b].oVal,
01133                     sizeOffsets);
01134 
01135                 // step past offsets
01136                 loc = (uint16_t) (loc + sizeOffsets);
01137 
01138                 // calculate the bit vector widthes
01139                 iV = bitVecWidth(calcWidth(pBatch[b].nVal), w);
01140 
01141                 // this is where the bit vectors start in temporary block
01142                 pBit = pBlock[column] + pBatch[b].oVal + sizeOffsets;
01143 
01144                 // nByte are taken by the bit vectors
01145                 nBytes = bitVecPtr(pBatch[b].nRow, iV, w, p, pBit);
01146 
01147                 myCopy(pIndexBlock + loc, pBit, nBytes);
01148 
01149                 // step past bit vectors
01150                 loc = (uint16_t)(loc + nBytes);
01151 
01152             } else if (pBatch[b].mode == LCS_VARIABLE) {
01153                 uint        sizeOffsets;
01154 
01155                 sizeOffsets =  pBatch[b].nRow * sizeof(uint16_t);
01156 
01157                 // variable size record batch
01158                 myCopy(
01159                     pIndexBlock + loc, pBlock[column] + pBatch[b].oVal,
01160                     sizeOffsets);
01161 
01162                 // step past offsets
01163                 loc = (uint16_t) (loc + sizeOffsets);
01164             } else {
01165                 // Fixed mode
01166                 uint sizeFixed;
01167 
01168                 sizeFixed =  pBatch[b].nRow * pBatch[b].recSize;
01169                 // fixed size record batch
01170                 myCopy(
01171                     pIndexBlock + loc, pBlock[column] + pBatch[b].oVal,
01172                     sizeFixed);
01173 
01174                 //step past fixed records
01175                 loc = (uint16_t) (loc + sizeFixed);
01176             }
01177 
01178             // set offset where values start in indexBlock
01179             pBatch[b].oVal = batchStart;
01180         }
01181     }
01182 
01183     //adjust batch count in leaf block header
01184     pHdr->nBatch = nClusterCols * numBatches;
01185 
01186     // start batch directory at end of last batch
01187     pHdr->oBatch = loc;
01188 
01189     // copy batch directories
01190     for (b = 0; b < numBatches; b++) {
01191         for (column = 0; column < nClusterCols; column++) {
01192             pBatch = (PLcsBatchDir)(pBlock[column] + batchOffset[column]);
01193             myCopy(pIndexBlock + loc, &pBatch[b], sizeof(LcsBatchDir));
01194             loc += sizeof(LcsBatchDir);
01195         }
01196     }
01197 
01198     if (isTracingLevel(TRACE_FINE)) {
01199         FENNEL_TRACE(
01200             TRACE_FINE, "Calling ClusterDump from moveFromTempToIndex");
01201         clusterDump->dump(opaqueToInt(clusterPageId), pHdr, szBlock);
01202     }
01203 }

void LcsClusterNodeWriter::allocArrays ( ) [private]

Allocates temporary arrays used during cluster writes.

Definition at line 1205 of file LcsClusterNodeWriter.cpp.

References SegNodeLock< Node >::allocatePage(), arraysAllocated, attrAccessors, batchCount, batchDirs, batchOffset, bForceMode, bufferLock, colTupleDesc, forceModeCount, SegPageLock::getPage(), CachePage::getWritableData(), maxValueSize, nBits, LcsClusterAccessBase::nClusterCols, nextWidthChange, oValBank, pValBank, SegPageLock::unlock(), and valBankStart.

Referenced by init().

01206 {
01207     // allocate arrays only if they have not been allocated already
01208     if (!arraysAllocated) {
01209         arraysAllocated = true;
01210 
01211         batchDirs.reset(new LcsBatchDir[nClusterCols]);
01212 
01213         pValBank.reset(new PBuffer[nClusterCols]);
01214 
01215         // allocate larger buffers for the individual pages in the value bank
01216 
01217         attrAccessors.reset(new UnalignedAttributeAccessor[nClusterCols]);
01218 
01219         for (uint col = 0; col < nClusterCols; col++) {
01220             bufferLock.allocatePage();
01221             pValBank[col] = bufferLock.getPage().getWritableData();
01222             // Similar to what's done in external sorter, we rely on the fact
01223             // that the underlying ScratchSegment keeps the page pinned for us.
01224             // The pages will be released when all other pages associated with
01225             // the ScratchSegment are released.
01226             bufferLock.unlock();
01227 
01228             attrAccessors[col].compute(colTupleDesc[col]);
01229         }
01230 
01231         valBankStart.reset(new uint16_t[nClusterCols]);
01232 
01233         forceModeCount.reset(new uint[nClusterCols]);
01234 
01235         bForceMode.reset(new ForceMode[nClusterCols]);
01236 
01237         oValBank.reset(new uint16_t[nClusterCols]);
01238 
01239         batchOffset.reset(new uint16_t[nClusterCols]);
01240 
01241         batchCount.reset(new uint[nClusterCols]);
01242 
01243         nBits.reset(new uint[nClusterCols]);
01244 
01245         nextWidthChange.reset(new uint[nClusterCols]);
01246 
01247         maxValueSize.reset(new uint[nClusterCols]);
01248     }
01249 
01250     memset(valBankStart.get(), 0, nClusterCols * sizeof(uint16_t));
01251     memset(forceModeCount.get(), 0, nClusterCols * sizeof(uint));
01252     memset(bForceMode.get(), 0, nClusterCols * sizeof(ForceMode));
01253     memset(oValBank.get(), 0, nClusterCols * sizeof(uint16_t));
01254     memset(batchOffset.get(), 0, nClusterCols * sizeof(uint16_t));
01255     memset(batchCount.get(), 0, nClusterCols * sizeof(uint));
01256     memset(nBits.get(), 0, nClusterCols * sizeof(uint));
01257     memset(nextWidthChange.get(), 0, nClusterCols * sizeof(uint));
01258     memset(maxValueSize.get(), 0, nClusterCols * sizeof(uint));
01259 }

uint32_t LcsClusterNodeWriter::round8Boundary ( uint32_t val ) [inline, private]

Rounds a 32-bit value to a boundary of 8.

Parameters:

val

value to be rounded

Definition at line 233 of file LcsClusterNodeWriter.h.

Referenced by putCompressedBatch().

00234     {
00235         return val & 0xfffffff8;
00236     }

uint32_t LcsClusterNodeWriter::roundIf8Boundary ( uint32_t val ) [inline, private]

Rounds a 32-bit value to a boundary of 8 if it is > 8.

Parameters:

val

value to be rounded

Definition at line 243 of file LcsClusterNodeWriter.h.

00244     {
00245         if (val > 8) {
00246             return round8Boundary(val);
00247         }
00248     }

bool LcsClusterNodeWriter::getLastClusterPageForWrite	(	PLcsClusterNode &	pBlock,
		LcsRid &	firstRid
	)

Gets the last cluster page.

Parameters:

	pBlock	output param returning the cluster page
	firstRid	output param returning first rid stored on cluster page

Returns:: true if cluster is non-empty

Definition at line 101 of file LcsClusterNodeWriter.cpp.

References LcsClusterAccessBase::bTreeTupleData, bTreeWriter, clusterDump, LcsClusterAccessBase::clusterLock, LcsClusterAccessBase::clusterPageId, SegNodeLock< Node >::getNodeForWrite(), TraceSource::isTracingLevel(), SegPageLock::lockExclusive(), opaqueToInt(), pBlock, LcsClusterAccessBase::readClusterPageId(), szBlock, and TRACE_FINE.

00103 {
00104     // get the last key in the btree (if it exists) and read the cluster
00105     // page based on the pageid stored in that btree record
00106 
00107     if (bTreeWriter->searchLast() == false) {
00108         bTreeWriter->endSearch();
00109         return false;
00110     }
00111 
00112     bTreeWriter->getTupleAccessorForRead().unmarshal(bTreeTupleData);
00113     clusterPageId = readClusterPageId();
00114     clusterLock.lockExclusive(clusterPageId);
00115     pBlock = &(clusterLock.getNodeForWrite());
00116     firstRid = pBlock->firstRID;
00117 
00118     // End the search so the BTreeWriter doesn't think it's positioned within
00119     // the btree.  We'll position properly on the first monotonic insert.
00120     bTreeWriter->endSearch();
00121 
00122     if (isTracingLevel(TRACE_FINE)) {
00123         FENNEL_TRACE(
00124             TRACE_FINE,
00125             "Calling ClusterDump from getLastClusterPageForWrite");
00126         clusterDump->dump(opaqueToInt(clusterPageId), pBlock, szBlock);
00127     }
00128 
00129     return true;
00130 }

PLcsClusterNode LcsClusterNodeWriter::allocateClusterPage ( LcsRid firstRid )

Allocates a new cluster page.

Parameters:

firstRid

first rid to be stored on cluster page

Returns:: page allocated

Definition at line 132 of file LcsClusterNodeWriter.cpp.

References SegNodeLock< Node >::allocatePage(), LcsClusterAccessBase::bTreeRid, LcsClusterAccessBase::bTreeTupleData, bTreeWriter, LcsClusterAccessBase::clusterLock, LcsClusterAccessBase::clusterPageId, DUP_FAIL, SegPageLock::flushPage(), SegNodeLock< Node >::getNodeForWrite(), SegPageLock::getPageId(), SegPageLock::isLocked(), NULL_PAGE_ID, SegmentAccessor::pSegment, and LcsClusterAccessBase::segmentAccessor.

00133 {
00134     // allocate a new cluster page and insert the corresponding rid, pageid
00135     // record into the btree
00136 
00137     PageId prevPageId = NULL_PAGE_ID;
00138 
00139     if (clusterLock.isLocked()) {
00140         // Remember the predecessor so that we can chain it below.
00141         prevPageId = clusterLock.getPageId();
00142 
00143         // Kick off an asynchronous write on the page we've just finished
00144         // so that when it comes time to checkpoint or victimize,
00145         // maybe it will be on disk already.
00146         clusterLock.flushPage(true);
00147     }
00148 
00149     clusterPageId = clusterLock.allocatePage();
00150     if (prevPageId != NULL_PAGE_ID) {
00151         segmentAccessor.pSegment->setPageSuccessor(prevPageId, clusterPageId);
00152     }
00153     bTreeRid = firstRid;
00154     bTreeTupleData[0].pData = reinterpret_cast<uint8_t *> (&firstRid);
00155     bTreeTupleData[1].pData = reinterpret_cast<uint8_t *> (&clusterPageId);
00156     bTreeWriter->insertTupleData(bTreeTupleData, DUP_FAIL);
00157     return &(clusterLock.getNodeForWrite());
00158 }

void LcsClusterNodeWriter::init	(	uint	nColumns,
		PBuffer	indexBlock,
		PBuffer *	pBlock,
		uint	szBlock
	)

Initializes object with parameters relevant to the cluster page that will be written.

Parameters:

	nColumns	number of columns in the cluster
	indexBlock	pointer to the cluster page to be written
	pBlock	array of pointers to temporary pages to be used while writing this cluster page
	szBlock	size of cluster page, reflecting max amount of space available to write cluster data

Definition at line 160 of file LcsClusterNodeWriter.cpp.

References allocArrays(), getClusterSubHeaderSize(), hdrSize, LcsMaxLeftOver, minSzLeft, LcsClusterAccessBase::nClusterCols, pBlock, pHdr, pIndexBlock, LcsClusterAccessBase::setHdrOffsets(), and szBlock.

00162 {
00163     nClusterCols = nColumn;
00164     pIndexBlock = iBlock;
00165     pBlock = pB;
00166     szBlock = szB;
00167     pHdr = (PLcsClusterNode) pIndexBlock;
00168 
00169     hdrSize = getClusterSubHeaderSize(nClusterCols);
00170 
00171     // initialize lastVal, firstVal, and nVal fields in the header
00172     // to point to the appropriate positions in the indexBlock
00173 
00174     setHdrOffsets(pHdr);
00175 
00176     minSzLeft = nClusterCols * (LcsMaxLeftOver * sizeof(uint16_t) +
00177                      sizeof(LcsBatchDir));
00178 
00179     allocArrays();
00180 }

void LcsClusterNodeWriter::close ( )

Definition at line 78 of file LcsClusterNodeWriter.cpp.

References attrAccessors, batchCount, batchDirs, batchOffset, bForceMode, bTreeWriter, LcsClusterAccessBase::clusterLock, SegPageLock::flushPage(), forceModeCount, SegPageLock::isLocked(), maxValueSize, nBits, nextWidthChange, oValBank, pValBank, SegPageLock::unlock(), and valBankStart.

Referenced by ~LcsClusterNodeWriter().

00079 {
00080     // flush and unlock last page written
00081     if (clusterLock.isLocked()) {
00082         clusterLock.flushPage(true);
00083     }
00084     clusterLock.unlock();
00085 
00086     bTreeWriter.reset();
00087     batchDirs.reset();
00088     pValBank.reset();
00089     valBankStart.reset();
00090     forceModeCount.reset();
00091     bForceMode.reset();
00092     oValBank.reset();
00093     batchOffset.reset();
00094     batchCount.reset();
00095     nBits.reset();
00096     nextWidthChange.reset();
00097     maxValueSize.reset();
00098     attrAccessors.reset();
00099 }

void LcsClusterNodeWriter::openNew ( LcsRid startRID )

Prepares a cluster page as a new one.

Parameters:

startRID

first RID on the page

Definition at line 182 of file LcsClusterNodeWriter.cpp.

References batchCount, batchDirs, batchOffset, LcsClusterAccessBase::delta, LcsClusterNode::firstRID, LcsClusterAccessBase::firstVal, hdrSize, LcsClusterAccessBase::lastVal, LCS_COMPRESSED, max(), LcsClusterNode::nBatch, nBits, LcsClusterAccessBase::nClusterCols, LcsClusterNode::nColumn, nextWidthChange, LcsClusterAccessBase::nVal, LcsClusterNode::oBatch, pHdr, szBlock, and szLeft.

00183 {
00184     int i;
00185 
00186     // Inialize block header and batches
00187     pHdr->firstRID = startRID;
00188     pHdr->nColumn = nClusterCols;
00189     pHdr->nBatch = 0;
00190     pHdr->oBatch = hdrSize;
00191 
00192     for (i = 0; i < nClusterCols; i++) {
00193         lastVal[i] = szBlock;
00194         firstVal[i] = (uint16_t) szBlock;
00195         nVal[i] = 0;
00196         delta[i] = 0;
00197         batchDirs[i].mode = LCS_COMPRESSED;
00198         batchDirs[i].nVal = 0;
00199         batchDirs[i].nRow = 0;
00200         batchDirs[i].oVal = 0;
00201         batchDirs[i].oLastValHighMark = lastVal[i];
00202         batchDirs[i].nValHighMark = nVal[i];
00203         batchOffset[i] = hdrSize;
00204         // # of bits it takes to represent 0 values
00205         nBits[i] = 0;
00206         nextWidthChange[i] = 1;
00207         batchCount[i] = 0;
00208     }
00209 
00210     // account for the header size, account for at least 1 batch for each column
00211     // and leave space for one addtional batch for a "left-over" batch
00212 
00213     szLeft = szBlock - hdrSize -
00214                 (2 * sizeof(LcsBatchDir)) * nClusterCols;
00215     szLeft = std::max(szLeft, 0);
00216     assert(szLeft >= 0);
00217 }

bool LcsClusterNodeWriter::openAppend	(	uint *	nValOffsets,
		uint16_t *	lastValOffsets,
		RecordNum &	nrows
	)

Prepares an existing cluster page for appending new data, and determines whether the page is already full and cannot accomodate any more data.

Parameters:

	nValOffsets	pointer to output array reflecting the number of values currently in each column on this page
	lastValOffsets	pointer to output array reflecting the offset of the last value currently on the page for each cluster column
	nrows	returns number of rows currently on page

Returns:: true if the page is already full

Definition at line 219 of file LcsClusterNodeWriter.cpp.

References batchCount, batchDirs, LcsClusterAccessBase::lastVal, LCS_COMPRESSED, max(), moveFromIndexToTemp(), LcsClusterNode::nBatch, nBits, LcsClusterAccessBase::nClusterCols, nextWidthChange, LcsClusterAccessBase::nVal, LcsClusterNode::oBatch, oValBank, pHdr, and szLeft.

00221 {
00222     int i;
00223 
00224     // leave space for one batch for each column entry
00225     szLeft = lastVal[nClusterCols - 1] - pHdr->oBatch -
00226                 (pHdr->nBatch + 2* nClusterCols) * sizeof(LcsBatchDir);
00227     szLeft = std::max(szLeft, 0);
00228     assert(szLeft >= 0);
00229 
00230     // Let's move the values, batch directories, and batches to
00231     // temporary blocks from index block
00232     nrows = moveFromIndexToTemp();
00233 
00234     for (i = 0; i < nClusterCols; i++) {
00235         nValOffsets[i] = nVal[i];
00236         lastValOffsets[i] = lastVal[i];
00237         memset(&batchDirs[i], 0, sizeof(LcsBatchDir));
00238 
00239         batchDirs[i].oLastValHighMark = lastVal[i];
00240         batchDirs[i].nValHighMark = nVal[i];
00241         batchDirs[i].mode = LCS_COMPRESSED;
00242 
00243         // # of bits it takes to represent 0 values
00244         nBits[i] = 0;
00245         nextWidthChange[i] = 1;
00246 
00247         oValBank[i] = 0;
00248         batchCount[i] = pHdr->nBatch / nClusterCols;
00249     }
00250 
00251     return (szLeft == 0);
00252 }

void LcsClusterNodeWriter::describeLastBatch	(	uint	column,
		uint &	dRow,
		uint &	recSize
	)

Returns parameters describing the last batch for a given column.

Parameters:

	column	the column to be described
	dRow	output parameter returning the number of rows over the multiple of 8 boundary
	recSize	output parameter returning the record size for the batch

Definition at line 254 of file LcsClusterNodeWriter.cpp.

References batchCount, batchOffset, LcsBatchDir::nRow, pBlock, and LcsBatchDir::recSize.

00256 {
00257     PLcsBatchDir pBatch;
00258 
00259     pBatch = (PLcsBatchDir) (pBlock[column] + batchOffset[column]);
00260     dRow = pBatch[batchCount[column] -1].nRow % 8;
00261     recSize = pBatch[batchCount[column] -1].recSize;
00262 }

uint16_t LcsClusterNodeWriter::getNextVal	(	uint	column,
		uint16_t	thisVal
	)

Returns the offset of the next value in a batch.

Parameters:

	column	column we want the value for
	thisVal	offset of the value currently positioned at

Definition at line 264 of file LcsClusterNodeWriter.cpp.

References attrAccessors, pBlock, and szBlock.

00265 {
00266     if (thisVal && thisVal != szBlock) {
00267         return
00268             (uint16_t) (thisVal +
00269                 attrAccessors[column].getStoredByteCount(
00270                     pBlock[column] + thisVal));
00271     } else {
00272         return 0;
00273     }
00274 }

void LcsClusterNodeWriter::rollBackLastBatch	(	uint	column,
		PBuffer	pVal
	)

Rolls back the last 8 value (or less) from a batch.

Parameters:

	column	column to be rolled back
	pVal	buffer where the rolled back values will be copied; the buffer is assumed to be fixedRec * (nRows % 8) in size, as determined by the last call to describeLastBatch

Definition at line 276 of file LcsClusterNodeWriter.cpp.

References attrAccessors, batchCount, batchDirs, batchOffset, bitVecPtr(), bitVecWidth(), calcWidth(), LcsClusterAccessBase::lastVal, LCS_COMPRESSED, LCS_FIXED, LcsMaxRollBack, max(), nBits, nextWidthChange, LcsClusterAccessBase::nVal, pBlock, readBitVecs(), and szLeft.

00277 {
00278     uint i;
00279     PLcsBatchDir pBatch;
00280     uint16_t *pValOffsets;
00281 
00282     uint8_t *pBit;                      // bitVecs start address
00283     WidthVec w;                         // bitVec width vector
00284     PtrVec p;                           // bitVec offsets
00285     uint iV;                            // # of bit vectors
00286 
00287     uint16_t rows[LcsMaxRollBack];      // row index storage
00288     int origSzLeft;
00289     uint len;
00290 
00291     // load last batch, nBatch must be at least 1
00292     pBatch = (PLcsBatchDir)(pBlock[column] + batchOffset[column]);
00293     batchDirs[column]  = pBatch[batchCount[column] -1];
00294 
00295     // compute size left in temporary block before roll back
00296     origSzLeft = lastVal[column] - batchOffset[column] -
00297                     (batchCount[column]+2)*sizeof(LcsBatchDir);
00298 
00299     if ((batchDirs[column].nRow > 8) || (batchDirs[column].nRow % 8) == 0) {
00300         return;
00301     }
00302 
00303     if (batchDirs[column].mode == LCS_COMPRESSED) {
00304         // calculate the bit vector widthes
00305         iV = bitVecWidth(calcWidth(batchDirs[column].nVal), w);
00306 
00307         // this is where the bit vectors start
00308         pBit = pBlock[column] + batchDirs[column].oVal +
00309                 batchDirs[column].nVal * sizeof(uint16_t);
00310 
00311         // nByte are taken by the bit vectors
00312         bitVecPtr(batchDirs[column].nRow, iV, w, p, pBit);
00313 
00314         // there are at most 8 rows in this batch
00315         readBitVecs(rows, iV, w, p, 0, batchDirs[column].nRow);
00316 
00317         // get the address of the batches value offsets
00318         pValOffsets = (uint16_t *)(pBlock[column] + batchDirs[column].oVal);
00319 
00320         // fill up buffer with batches values
00321         for (i = 0; i < batchDirs[column].nRow;
00322             i++, pBuf += batchDirs[column].recSize)
00323         {
00324             len =
00325                 attrAccessors[column].getStoredByteCount(
00326                     pBlock[column] + pValOffsets[rows[i]]);
00327             memcpy(pBuf, pBlock[column] + pValOffsets[rows[i]], len);
00328         }
00329 
00330     } else if (batchDirs[column].mode == LCS_FIXED) {
00331         // fixed size record batch
00332         // copy the values into the given buffer
00333         memcpy(
00334             pBuf,
00335             pBlock[column] + batchDirs[column].oVal,
00336             batchDirs[column].nRow * batchDirs[column].recSize);
00337     } else {
00338         // variable sized records (batch.mode == LCS_VARIABLE)
00339         // get the address of the batches value offsets
00340         pValOffsets = (uint16_t *)(pBlock[column] + batchDirs[column].oVal);
00341 
00342         // fill up buffer with batches values
00343         for (i = 0; i < batchDirs[column].nRow;
00344             i++, pBuf += batchDirs[column].recSize)
00345         {
00346             len =
00347                 attrAccessors[column].getStoredByteCount(
00348                     pBlock[column] + pValOffsets[i]);
00349             memcpy(pBuf, pBlock[column] + pValOffsets[i], len);
00350         }
00351     }
00352 
00353     // Reset the last batch
00354     batchCount[column]--;
00355     // batch dir offset points to the beginning of the rolled back batch
00356     batchOffset[column] = batchDirs[column].oVal;
00357 
00358     // copy the batch dir back to the end of the prev batch.
00359     memmove(
00360         pBlock[column] + batchOffset[column],
00361         pBatch,
00362         batchCount[column] * sizeof(LcsBatchDir));
00363 
00364     // recalc size left
00365     // leave place for one new batch(the rolled back one will be rewriten)
00366     // and possibley one to follow.  Subtract the difference of the new size
00367     // and the original size and add this to szLeft in index variable
00368     int newSz;
00369     newSz = lastVal[column] - batchOffset[column] -
00370             (batchCount[column] + 2) * sizeof(LcsBatchDir);
00371     szLeft += (newSz - origSzLeft);
00372     szLeft = std::max(szLeft, 0);
00373     assert(szLeft >= 0);
00374 
00375     // # of bits it takes to represent 0 values
00376     nBits[column] = 0;
00377     nextWidthChange[column] = 1;
00378 
00379     // set batch parameters
00380     batchDirs[column].mode = LCS_COMPRESSED;
00381     batchDirs[column].nVal = 0;
00382     batchDirs[column].nRow = 0;
00383     batchDirs[column].oVal = 0;
00384     batchDirs[column].recSize = 0;
00385 }

bool LcsClusterNodeWriter::noCompressMode ( uint column ) const [inline]

Returns true if the batch is not being forced to compress mode.

Parameters:

column

column being described

Definition at line 361 of file LcsClusterNodeWriter.h.

References fixed, and variable.

00362     {
00363         return bForceMode[column] == fixed ||
00364                 bForceMode[column] == variable;
00365     };

PBuffer LcsClusterNodeWriter::getOffsetPtr	(	uint	column,
		uint16_t	offset
	)			`[inline]`

Translates an offset for a column to the pointer to the actual value.

Parameters:

	column	offset corresponds to this column
	offset	offset to be translated

Returns:: pointer to value

Definition at line 376 of file LcsClusterNodeWriter.h.

00377     {
00378         return pBlock[column] + offset;
00379     };

bool LcsClusterNodeWriter::addValue	(	uint	column,
		bool	bFirstTimeInBatch
	)

Adds a value to the page, in the case where the value already exists in the column.

Parameters:

	column	column corresponding to the value being added
	bFirstTimeInBatch	true if this is the first time the value is encountered for this batch

Returns:: true if there is enough room in the page for the value

Definition at line 389 of file LcsClusterNodeWriter.cpp.

References batchDirs, calcWidth(), LcsMaxSzLeftError, nBits, LcsClusterAccessBase::nClusterCols, nextWidthChange, LcsClusterAccessBase::nVal, and szLeft.

00390 {
00391     // Calculate szleft assuming the value gets added.
00392     szLeft -= sizeof(uint16_t);
00393 
00394     // if there is not enough space left, reject value
00395     if (szLeft < ((int) nClusterCols * LcsMaxSzLeftError)) {
00396         // set szLeft to its previous value
00397         szLeft += sizeof(uint16_t);
00398         assert(szLeft >= 0);
00399         return false;
00400     }
00401 
00402     if (bFirstTimeInBatch) {
00403         // there is enough space to house the value, increment batch
00404         // value count
00405         batchDirs[column].nVal++;
00406 
00407         // check if nBits needs to change by comparing the value count
00408         // the change point count
00409         if (batchDirs[column].nVal == nextWidthChange[column]) {
00410             // calculate the next nBits value, and the count of values
00411             // for the next chane
00412             nBits[column] = calcWidth(batchDirs[column].nVal);
00413             nextWidthChange[column] = (1 << nBits[column]) + 1;
00414         }
00415     }
00416 
00417     return true;
00418 }

bool LcsClusterNodeWriter::addValue	(	uint	column,
		PBuffer	pVal,
		uint16_t *	oVal
	)

Adds a new value to the page.

In the case of compressed or variable mode, adds the value to the bottom of the page. In the case of fixed mode, adds the value to the "value bank".

Parameters:

	column	column corresponding to the value being added
	pVal	value to be added
	oVal	returns the offset where the value has been added

Returns:: true if there is enough room in the page for the value

Definition at line 422 of file LcsClusterNodeWriter.cpp.

References attrAccessors, batchDirs, bForceMode, calcWidth(), fixed, LcsClusterAccessBase::lastVal, LcsMaxSzLeftError, maxValueSize, nBits, LcsClusterAccessBase::nClusterCols, nextWidthChange, LcsClusterAccessBase::nVal, pBlock, pValBank, and szLeft.

00423 {
00424     uint16_t lastValOffset;
00425     int oldSzLeft = szLeft;
00426     uint szVal = attrAccessors[column].getStoredByteCount(pVal);
00427 
00428     // if we are in forced fixed compression mode,
00429     // see if the maximum record size in this batch has increased.
00430     // if so, adjust the szLeft based on the idea that each previous element
00431     // will now be taking more space
00432     if (bForceMode[column] == fixed) {
00433         if (szVal > maxValueSize[column]) {
00434             szLeft -= batchDirs[column].nVal *
00435                 (szVal - maxValueSize[column]);
00436             maxValueSize[column] = szVal;
00437         }
00438     }
00439 
00440     // adjust szleft (upper bound on amount of space left in the block).
00441     // If we are in a forced fixed compression mode then we can calculate this
00442     // exactly.  If we are in "none" mode, then we calculate szLeft according to
00443     // variable mode compression (which should be an upper bound), and adjust it
00444     // later in pickCompressionMode
00445     if (bForceMode[column] == fixed) {
00446         szLeft -= maxValueSize[column];
00447     } else {
00448         // assume value is being added in variable mode
00449         // (note: this assumes that whenever we convert from
00450         // variable mode to compressed mode, the compressed mode will
00451         // take less space, for only in this case is szLeft an upper bound)
00452         szLeft -= (sizeof(uint16_t) + szVal) ;
00453     }
00454 
00455     // if there is not enough space left reject value
00456     if (szLeft < ((int) nClusterCols * LcsMaxSzLeftError)) {
00457         // set szLeft to its previous value
00458         szLeft = oldSzLeft;
00459         assert(szLeft >= 0);
00460         return false;
00461     }
00462 
00463     // otherwise, go ahead and add the value...
00464 
00465     lastValOffset = lastVal[column] - szVal;
00466 
00467     // there is enough space to house the value, increment batch value count
00468     batchDirs[column].nVal++;
00469 
00470     // check if nBits needs to change by comparing the value count
00471     // the change point count
00472     if (batchDirs[column].nVal == nextWidthChange[column]) {
00473         // calculate the next nBits value, and the count of values
00474         // for the next chane
00475         nBits[column] = calcWidth(batchDirs[column].nVal);
00476         nextWidthChange[column] = (1 << nBits[column]) + 1;
00477     }
00478 
00479     lastVal[column] = lastValOffset;
00480 
00481     // Save the value being inserted.  If we are building the
00482     // block in fixed mode then save the value into pValBank
00483     // rather than saving it into the block
00484     if (fixed == bForceMode[column]) {
00485         memcpy(pValBank[column] + lastValOffset, pVal, szVal);
00486     } else {
00487         memcpy(pBlock[column] + lastValOffset, pVal, szVal);
00488     }
00489 
00490     // return the block offset of the new value;
00491     *oVal = lastValOffset;
00492 
00493     nVal[column]++;
00494 
00495     return true;
00496 }

void LcsClusterNodeWriter::undoValue	(	uint	column,
		PBuffer	pVal,
		bool	bFirstInBatch
	)

Undoes the last value added to the current batch for a column.

Parameters:

	column	column corresponding to the value to be undone
	pVal	value to be undone
	bFirstInBatch	true if the value being undone is the first such value for the batch

Definition at line 498 of file LcsClusterNodeWriter.cpp.

References attrAccessors, batchDirs, calcWidth(), LcsClusterAccessBase::lastVal, nBits, nextWidthChange, LcsClusterAccessBase::nVal, and szLeft.

00500 {
00501     // pVal may be null if the value already exists, in which case, it wasn't
00502     // added to the value list.  However, if it was the first such value for
00503     // the batch, addValue was called to bump-up the batch value count
00504     // so we still need to call undoValue
00505     uint szVal =
00506         (pVal) ? attrAccessors[column].getStoredByteCount(pVal) : 0;
00507 
00508     // add back size subtracted for offset
00509     szLeft += (sizeof(uint16_t) + szVal) ;
00510     assert(szLeft >= 0);
00511 
00512     // If value was new to the batch, then adjust counters
00513     if (bFirstInBatch) {
00514         // decrement batch count
00515         batchDirs[column].nVal--;
00516 
00517         //reset nextWidthChange
00518         if (batchDirs[column].nVal == 0) {
00519             nextWidthChange[column] = 1;
00520         } else {
00521             // calculate the next nBits value, and the count of values
00522             // for the next chane
00523             nBits[column] = calcWidth(batchDirs[column].nVal);
00524             nextWidthChange[column] = (1 << nBits[column]) + 1;
00525         }
00526     }
00527 
00528     if (pVal) {
00529         // upgrage header
00530         lastVal[column] += szVal;
00531         nVal[column]--;
00532     }
00533 }

void LcsClusterNodeWriter::putCompressedBatch	(	uint	column,
		PBuffer	pRows,
		PBuffer	pBuf
	)

Writes a compressed mode batch into the temporary cluster page for a column.

Only a multiple of 8 rows is written, if this is not the last batch in the cluster.

Excess rows are written into a temporary buffer. If this is the last batch in the load, then it is ok to have < 8 rows, as the next load will roll it back to fill it up with more rows.

Note that it is assumed that the caller has already copied the key offsets for this batch into the cluster page. This call will only copy the bit vectors and batch directory corresponding to this batch

Parameters:

	column	column corresponding to the batch
	pRows	array mapping rows to key offsets
	pBuf	temporary buffer where excess row values will be copied; assumed to be (nRow % 8)*fixedRec big

Definition at line 535 of file LcsClusterNodeWriter.cpp.

References attrAccessors, batchCount, batchDirs, batchOffset, bitVecPtr(), bitVecWidth(), LcsClusterAccessBase::lastVal, LCS_COMPRESSED, nBits, nByte, nextWidthChange, LcsClusterAccessBase::nVal, pBlock, round8Boundary(), and setBits().

00537 {
00538     uint        i, j, b;
00539     uint        iRow;
00540     uint        nByte;
00541     uint8_t     *pBit;
00542     uint16_t    *pOffs;
00543     PLcsBatchDir pBatch;
00544 
00545     WidthVec    w;      // bitVec width vector
00546     PtrVec      p;      // bitVec offsets
00547     uint        iV;     // number of bit vectors
00548 
00549     // pickCompressionMode() was called prior to putCompressedBatch,
00550     // and the following has been already done:
00551     // -- the batch descriptors were moved to the back of the batch
00552     // -- a batch descriptor for this batch has been placed in the batch
00553     //    directory
00554     // -- this->batch contains up to date info
00555     // -- the caller has copied nVal value offsets to the head of this batch
00556 
00557     // write to buffer values for rows over the 8 boundary if nrow is
00558     // greater then 8
00559 
00560     if (batchDirs[column].nRow > 8) {
00561         uint len;
00562         pOffs = (uint16_t *)(pBlock[column] + batchDirs[column].oVal);
00563         for (i = round8Boundary((uint32_t) batchDirs[column].nRow);
00564             i < batchDirs[column].nRow; i++, pBuf += batchDirs[column].recSize)
00565         {
00566             iRow = ((uint16_t *) pRows)[i];
00567             len =
00568                 attrAccessors[column].getStoredByteCount(
00569                     pBlock[column] + pOffs[iRow]);
00570             memcpy(pBuf, pBlock[column] + pOffs[iRow], len);
00571         }
00572         batchDirs[column].nRow =
00573             round8Boundary((uint32_t) batchDirs[column].nRow);
00574     }
00575 
00576     // calculate the bit vector widthes, sum(w[i]) is nBits
00577     iV = bitVecWidth(nBits[column], w);
00578 
00579     // this is where the bit vectors start
00580     pBit = pBlock[column] + batchDirs[column].oVal +
00581             batchDirs[column].nVal*sizeof(uint16_t);
00582 
00583     // nByte are taken by the bit vectors, clear them befor OR-ing
00584     nByte = bitVecPtr(batchDirs[column].nRow, iV, w, p, pBit);
00585     memset(pBit, 0, nByte);
00586 
00587     for (j = 0, b = 0; j < iV ; j++) {
00588         switch (w[j]) {
00589         case 16:
00590             memcpy(p[j], pRows, batchDirs[column].nRow * sizeof(uint16_t));
00591             break;
00592 
00593         case 8:
00594             for (i = 0; i < batchDirs[column].nRow ; i++) {
00595                 (p[j])[i] = (uint8_t)((uint16_t *) pRows)[i];
00596             }
00597             break;
00598 
00599         case 4:
00600             for (i = 0; i < batchDirs[column].nRow ; i++) {
00601                 setBits(
00602                     p[j] + i / 2 ,
00603                     4,
00604                     (i % 2) * 4,
00605                     (uint16_t)(((uint16_t *) pRows)[i] >> b));
00606             }
00607             break;
00608 
00609         case 2:
00610             for (i = 0; i < batchDirs[column].nRow ; i++) {
00611                 setBits(
00612                     p[j] + i / 4 ,
00613                     2,
00614                     (i % 4) * 2,
00615                     (uint16_t)(((uint16_t *) pRows)[i] >> b));
00616             }
00617             break;
00618 
00619         case 1:
00620             for (i = 0; i < batchDirs[column].nRow ; i++) {
00621                 setBits(
00622                     p[j] + i / 8 ,
00623                     1,
00624                     (i % 8),
00625                     (uint16_t)(((uint16_t *)pRows)[i] >> b));
00626             }
00627             break;
00628 
00629         default:
00630                 ;
00631         }
00632         b += w[j];
00633     }
00634 
00635     // put the batch in the batch directory
00636     pBatch = (PLcsBatchDir)(pBlock[column] + batchOffset[column]);
00637     pBatch[batchCount[column]] = batchDirs[column];
00638     batchCount[column]++;
00639 
00640     // reset the batch state
00641     batchDirs[column].mode = LCS_COMPRESSED;
00642     batchDirs[column].oLastValHighMark = lastVal[column];
00643     batchDirs[column].nValHighMark = nVal[column];
00644     batchDirs[column].nVal = 0;
00645     batchDirs[column].oVal = batchOffset[column];
00646     batchDirs[column].nRow = 0;
00647 
00648     // # of bits it takes to represent 0 values
00649     nBits[column] = 0;
00650     nextWidthChange[column] = 1 ;
00651 }

void LcsClusterNodeWriter::putFixedVarBatch	(	uint	column,
		uint16_t *	pRows,
		PBuffer	pBuf
	)

Writes a fixed or variable mode batch into a temporary cluster page for a column.

Only a multiple of 8 rows is written, if this is not the last batch in the cluster.

Excess rows are written into a temporary buffer. If this is the last batch in the load, then it is ok to have < 8 rows, as the next load will roll it back to fill it up with more rows.

In the variable mode case, the key offsets are written to the batch area on the page. In the fixed mode case, the values themselves are written to the batch area. In both cases, the batch directory is also written out.

Parameters:

	column	column corresponding to the batch
	pRows	array of offsets to values
	pBuf	temporary buffer where excess row values will be copied; assumed to be (nRow % 8)*fixedRec big

Definition at line 653 of file LcsClusterNodeWriter.cpp.

References attrAccessors, batchCount, batchDirs, batchOffset, bForceMode, fixed, forceModeCount, LcsClusterAccessBase::lastVal, LCS_COMPRESSED, LCS_FIXED, LCS_VARIABLE, maxValueSize, nBits, nextWidthChange, none, LcsClusterAccessBase::nVal, oValBank, pBlock, pValBank, valBankStart, valueSource(), and variable.

00655 {
00656     uint        i;
00657     uint        batchRows;
00658     PBuffer     pVal;
00659     PLcsBatchDir pBatch;
00660     PBuffer     src;
00661     uint        batchRecSize;
00662     uint16_t    localLastVal;
00663     uint16_t    localoValBank;
00664     PBuffer     localpValBank, localpBlock;
00665 
00666 
00667     // The # of rows in a batch will be smaller then 8 or a multiple
00668     // of 8.  All but the last batch in a load will be greater then 8.
00669     // Round to nearest 8 boundary, unless this is the last batch (which
00670     // we determine by the nRows in the batch being less than 8).
00671     // If it turns it isn't the latch batch, then our caller (WriteBatch)
00672     // will roll back the whole batch and add it to a new block instead.
00673     batchRows = (batchDirs[column].nRow > 8)
00674         ? batchDirs[column].nRow & 0xfffffff8 : batchDirs[column].nRow;
00675 
00676     // the value destination
00677     pVal = pBlock[column] + batchDirs[column].oVal;
00678     if (batchDirs[column].mode == LCS_VARIABLE) {
00679         // For a variable mode, copy in the value offsets into the RI block.
00680         // The left over i.e. the rows over the highest 8 boundary will be
00681         // copied to pBuf
00682         memcpy(pVal, pRows, batchRows * sizeof(uint16_t));
00683     } else {
00684         // it's a fixed record batch
00685         assert(batchDirs[column].mode == LCS_FIXED);
00686 
00687         batchRecSize  = batchDirs[column].recSize;
00688         localLastVal  = lastVal[column];
00689         localpValBank = pValBank[column] + valBankStart[column];
00690         localoValBank = oValBank[column];
00691         localpBlock   = pBlock[column];
00692 
00693         // Copy the values themselves into the block.
00694         // The values are currently stored in pValBank
00695         for (i = 0; i < batchRows; i++) {
00696             // valueSource determines by the offset whether the value comes from
00697             // the bank of from the block
00698             src = valueSource(
00699                 localLastVal, localpValBank, localoValBank,
00700                 localpBlock, pRows[i]);
00701             uint len = attrAccessors[column].getStoredByteCount(src);
00702             memcpy(pVal, src, len);
00703             pVal += batchRecSize;
00704         }
00705     }
00706 
00707     // if forced fixed mode is true we need to periodically set it false, so
00708     // we can at least check if the data can be compressed
00709     if (bForceMode[column] != none) {
00710         if (forceModeCount[column] > 20) {
00711             bForceMode[column] = none;
00712             forceModeCount[column] = 0;
00713         }
00714     }
00715 
00716     batchRecSize  = batchDirs[column].recSize;
00717     localLastVal  = lastVal[column];
00718     localpValBank = pValBank[column] + valBankStart[column];
00719     localoValBank = oValBank[column];
00720     localpBlock   = pBlock[column];
00721 
00722     // copy the tail of the batch (the last nRow % 8 values) to pBuf
00723     pVal = pBuf;
00724     for (i = batchRows; i < batchDirs[column].nRow; i++) {
00725         // valueSource determines by the offset whether the value comes from
00726         // the bank or from the block. if the value bank is not used
00727         // valueSource will get all the values fron the block
00728         src = valueSource(
00729             localLastVal, localpValBank, localoValBank,
00730             localpBlock, pRows[i]);
00731         uint len = attrAccessors[column].getStoredByteCount(src);
00732         memcpy(pVal, src, len);
00733         pVal += batchRecSize;
00734     }
00735 
00736     if (pValBank[column]) {
00737         oValBank[column] = 0;
00738     }
00739 
00740     // Put batch descriptor in batch directory
00741     batchDirs[column].nRow = batchRows;
00742     pBatch = (PLcsBatchDir)(pBlock[column] + batchOffset[column]);
00743     pBatch[batchCount[column]] = batchDirs[column];
00744 
00745     // inc. batch count
00746     batchCount[column]++;
00747 
00748     // reset the batch state.  set batch back to compressed mode
00749     // unless the flag has been set that next
00750     switch (bForceMode[column]) {
00751     case none:
00752         batchDirs[column].mode = LCS_COMPRESSED;
00753         break;
00754     case fixed:
00755         batchDirs[column].mode = LCS_FIXED;
00756         break;
00757     case variable:
00758         batchDirs[column].mode = LCS_VARIABLE;
00759         break;
00760     default:
00761         assert(false);
00762     }
00763     batchDirs[column].oLastValHighMark = lastVal[column];
00764     batchDirs[column].nValHighMark = nVal[column];
00765     batchDirs[column].nVal = 0;
00766     batchDirs[column].oVal = batchOffset[column];
00767     batchDirs[column].nRow = 0;
00768 
00769     // # of bits it takes to represent 0 values
00770     nBits[column] = 0;
00771     nextWidthChange[column] = 1 ;
00772 
00773     maxValueSize[column] = 0;
00774 }

void LcsClusterNodeWriter::pickCompressionMode	(	uint	column,
		uint	fixedSize,
		uint	nRows,
		uint16_t **	pValOffset,
		LcsBatchMode &	compressionMode
	)

Determines which compression mode to use for a batch.

Parameters:

	column	column for which compression mode is being determined
	fixedSize	size of record in the case of fixed size compression
	nRows	number of rows in the batch
	pValOffset	returns a pointer to the offset of the start of the batch
	compressionMode	returns the chosen compression mode

Definition at line 776 of file LcsClusterNodeWriter.cpp.

References batchCount, batchDirs, batchOffset, bForceMode, bitVecWidth(), fixed, forceModeCount, LcsClusterAccessBase::lastVal, LCS_FIXED, LCS_VARIABLE, LcsMaxLeftOver, LcsMaxSzLeftError, max(), min(), nBits, nByte, LcsClusterAccessBase::nVal, oValBank, pBlock, pValBank, sizeofBitVec(), szLeft, valBankStart, and variable.

00779 {
00780     uint        nByte;
00781     PLcsBatchDir pBatch;
00782     WidthVec    w;      // bitVec m_width vector
00783     uint        iV;     // number of bit vectors
00784 
00785 
00786     uint        szCompressed;   // size of the compressed batch
00787     uint        szVariable;     // size of the variable sized batch
00788     uint        szFixed;        // size of the fixed batch
00789     uint        szNonCompressed;
00790     uint        deltaVal;
00791     uint        batchRows;      // # of rows in the batch that is nRows rounded
00792                                 // down to the nearest 8 boundary
00793 
00794     // update batch fields
00795     batchDirs[column].nRow = nRow;
00796     batchDirs[column].recSize = recSize;
00797 
00798     // calculate the size required for a compressed and sorted batch
00799     // by summing the spcae required for the value offsets, the bit vectors
00800     // and the values that were put in since the batch started
00801 
00802     // the # of rows in a batch will be smaller then 8 or a multiple
00803     // of 8. all but the last batch in a load will be greater then 8.
00804     batchRows = (nRow > 8) ? nRow & 0xfffffff8 : nRow;
00805 
00806     szCompressed = batchDirs[column].nVal*sizeof(uint16_t) +
00807                         (nBits[column]*nRow + LcsMaxSzLeftError * 8) / 8
00808                    + (batchDirs[column].oLastValHighMark - lastVal[column]);
00809 
00810     // the variable size batch does not have the bit vectors
00811     szVariable = batchDirs[column].nRow * sizeof(uint16_t)
00812                    + (batchDirs[column].oLastValHighMark - lastVal[column]);
00813 
00814     // calculate the size required for the non compressed fixed mode
00815     // add max left overs to allow left overs to be added back
00816     uint    leftOverSize;
00817     leftOverSize = LcsMaxLeftOver * sizeof(uint16_t) +
00818                     (3 * LcsMaxLeftOver + LcsMaxSzLeftError * 8) / 8
00819                        + LcsMaxLeftOver * recSize;
00820     szFixed = nRow * recSize + leftOverSize;
00821 
00822     szNonCompressed = std::min(szFixed, szVariable);
00823 
00824     // Should we store this block in one of the non-compressed modes (fixed or
00825     // variable)?  We do this if either
00826     //    1) the non-compressed size is smaller than the compressed size
00827     // or 2) we built the block in forced uncompress mode
00828     //
00829     // test if the compressed size is bigger then the non compressed size
00830 
00831     if ((fixed == bForceMode[column] || variable == bForceMode[column])
00832         || szCompressed > szNonCompressed) {
00833         // switch to one of the noncompressed modes
00834         *pValOffset = NULL;
00835         batchDirs[column].nVal = 0;
00836 
00837         forceModeCount[column]++;
00838 
00839         // If we are storing it in fixed mode...
00840         if (fixed == bForceMode[column] || szNonCompressed == szFixed) {
00841             // batch will be stored in fixed mode
00842             // change mode
00843             batchDirs[column].mode = LCS_FIXED;
00844 
00845             // Are we switching from variable mode to fixed mode?
00846             if (bForceMode[column] != fixed) {
00847                 // We are going to store the batch in fixed mode.  But currently
00848                 // it is saved in variable mode.  Save the value pointers into
00849                 // pValBank and we will use them later to help in conversion
00850 
00851                 // number of bytes taken by value over the batch high mark point
00852                 deltaVal = batchDirs[column].oLastValHighMark - lastVal[column];
00853 
00854                 // save the values obtained while this batch was in
00855                 // variable mode
00856                 if (deltaVal) {
00857                     memcpy(
00858                         pValBank[column],
00859                         pBlock[column] + lastVal[column],
00860                         deltaVal);
00861                 }
00862 
00863                 valBankStart[column] = 0;
00864 
00865                 // mark that for the next few times, automatically go to
00866                 // fixed mode without checking if it is the best
00867                 bForceMode[column] = fixed;
00868 
00869                 // Adjust szLeft since we have freed up some space in
00870                 // the block
00871                 assert(szVariable >= szFixed);
00872                 szLeft += (szVariable - szFixed);
00873                 assert(szLeft >= 0);
00874             } else {
00875                 valBankStart[column] = lastVal[column];
00876             }
00877 
00878             // Reclaim the space at the bottom of the block that
00879             // used to hold the values
00880 
00881             // first offset included in the bank
00882             oValBank[column] = lastVal[column];
00883             lastVal[column] = batchDirs[column].oLastValHighMark;
00884             nVal[column] = batchDirs[column].nValHighMark;
00885 
00886             // the number of bytes taken by the batch
00887             nByte = batchRows * batchDirs[column].recSize;
00888 
00889         } else {
00890             // batch will be stored in variable mode
00891 
00892             batchDirs[column].mode = LCS_VARIABLE;
00893 
00894             // the number of bytes taken by the batch
00895             nByte = batchRows*sizeof(uint16_t);
00896 
00897             // mark that for the next few times, automatically go to
00898             // fixed mode without checking if it is the best
00899             bForceMode[column] = variable;
00900         }
00901     } else {
00902         // batch will be stored in compressed mode
00903         // values will be put at the start of the new batch
00904 
00905         *pValOffset = (uint16_t *)(pBlock[column] + batchOffset[column]);
00906 
00907         // calculate the bit vector widthes
00908         iV = bitVecWidth(nBits[column], w);
00909 
00910         // nByte is the # bytes taken by the batch, it is the sum of
00911         // the bit vectors size and the value offsets
00912         nByte = sizeofBitVec(batchRows, iV, w) +
00913                 batchDirs[column].nVal * sizeof(uint16_t);
00914 
00915         // Adjust szLeft since we have freed up some space in the block
00916         assert(szVariable >= szCompressed);
00917         szLeft += (szVariable - szCompressed);
00918         assert(szLeft >= 0);
00919     }
00920 
00921     compressionMode = batchDirs[column].mode;
00922 
00923     // Slide down the batch directories to make room for new batch data (batch
00924     // directories occur after the batches).
00925     pBatch = (PLcsBatchDir)(pBlock[column] + batchOffset[column] + nByte);
00926     memmove(
00927         pBatch,
00928         pBlock[column] + batchOffset[column],
00929         batchCount[column] * sizeof(LcsBatchDir));
00930 
00931     // adjust szLeft for the space used by the next batch to reflect only
00932     // its batch directory
00933     szLeft -= sizeof(LcsBatchDir);
00934     szLeft = std::max(szLeft, 0);
00935     assert(szLeft >= 0);
00936 
00937     // batchDirs[column].oVal points to where the batch dir used to be,
00938     // and this is where the batch records will start
00939     batchDirs[column].oVal = batchOffset[column];
00940 
00941     // set batchOffset[column] to point to the start of the batch
00942     // directores (if we have another batch then this will become the
00943     // offset of the new batch)
00944     batchOffset[column] = (batchOffset[column] + nByte);
00945 }

bool LcsClusterNodeWriter::isEndOfBlock ( ) [inline]

Returns true if there is no space left in the cluster page.

Definition at line 489 of file LcsClusterNodeWriter.h.

References LcsMaxLeftOver.

00490     {
00491         uint col;
00492         int valueSizeNeeded;
00493 
00494         for (valueSizeNeeded = 0, col = 0; col < nClusterCols; col++) {
00495             valueSizeNeeded += batchDirs[col].recSize * LcsMaxLeftOver;
00496         }
00497 
00498         return szLeft <= (minSzLeft + valueSizeNeeded);
00499     }

void LcsClusterNodeWriter::endBlock ( ) [inline]

Done with the current cluster page.

Moves all data from temporary pages into the real cluster page

Definition at line 505 of file LcsClusterNodeWriter.h.

00506     {
00507         moveFromTempToIndex();
00508     }

LcsRid LcsClusterAccessBase::readRid ( ) [protected, inherited]

Returns RID from btree tuple.

Definition at line 37 of file LcsClusterAccessBase.cpp.

References LcsClusterAccessBase::bTreeTupleData.

Referenced by LcsClusterReader::getNextBTreeEntry(), LcsClusterReader::getNextPageForPrefetch(), LcsClusterReader::readClusterPage(), and LcsClusterReader::searchForRid().

00038 {
00039     return *reinterpret_cast<LcsRid const *> (bTreeTupleData[0].pData);
00040 }

PageId LcsClusterAccessBase::readClusterPageId ( ) [protected, inherited]

Returns cluster pageid from btree tuple.

Definition at line 42 of file LcsClusterAccessBase.cpp.

References LcsClusterAccessBase::bTreeTupleData.

Referenced by getLastClusterPageForWrite(), LcsClusterReader::getNextBTreeEntry(), LcsClusterReader::getNextPageForPrefetch(), LcsClusterReader::position(), and LcsClusterReader::readClusterPage().

00043 {
00044     return *reinterpret_cast<PageId const *> (bTreeTupleData[1].pData);
00045 }

void LcsClusterAccessBase::setHdrOffsets ( PConstLcsClusterNode pHdr ) [protected, inherited]

Sets pointers to offset arrays in cluster page header.

Parameters:

pHdr

pointer to cluster node header

Definition at line 47 of file LcsClusterAccessBase.cpp.

References LcsClusterAccessBase::delta, LcsClusterAccessBase::firstVal, LcsClusterAccessBase::lastVal, LcsClusterAccessBase::nClusterCols, and LcsClusterAccessBase::nVal.

Referenced by LcsClusterDump::dump(), init(), and LcsClusterReader::setUpBlock().

00048 {
00049     PBuffer p = (PBuffer) pHdr;
00050 
00051     lastVal = (uint16_t *) (p += sizeof(LcsClusterNode));
00052     firstVal = (uint16_t *) (p += sizeof(uint16_t) * nClusterCols);
00053     nVal = (uint *) (p += sizeof(uint16_t) * nClusterCols);
00054     delta = (uint16_t *) (p += sizeof(uint) * nClusterCols);
00055 }

uint LcsClusterAccessBase::getNumClusterCols ( ) [inline, inherited]

Returns number of columns in cluster.

Definition at line 118 of file LcsClusterAccessBase.h.

00119     {
00120         return nClusterCols;
00121     }

void LcsClusterAccessBase::setNumClusterCols ( uint nCols ) [inline, inherited]

Sets number of columns in cluster.

Definition at line 126 of file LcsClusterAccessBase.h.

00127     {
00128         nClusterCols = nCols;
00129     }

void LcsClusterAccessBase::unlockClusterPage ( ) [inherited]

Unlocks cluster page.

Definition at line 57 of file LcsClusterAccessBase.cpp.

References LcsClusterAccessBase::clusterLock, and SegPageLock::unlock().

Referenced by LcsClusterReader::close().

00058 {
00059     clusterLock.unlock();
00060 }

void TraceSource::initTraceSource	(	SharedTraceTarget	pTraceTarget,
		std::string	name
	)			`[virtual, inherited]`

For use when initialization has to be deferred until after construction.

Parameters:

	pTraceTarget	the TraceTarget to which messages will be sent
	name	the name of this source

Definition at line 46 of file TraceSource.cpp.

References TraceSource::isTracing(), TraceSource::minimumLevel, TraceSource::name, TraceSource::pTraceTarget, and TRACE_OFF.

Referenced by TestBase::beforeTestCase(), TestBase::TestBase(), and TraceSource::TraceSource().

00049 {
00050     assert(!pTraceTarget.get());
00051 
00052     pTraceTarget = pTraceTargetInit;
00053     name = nameInit;
00054     if (isTracing()) {
00055         minimumLevel = pTraceTarget->getSourceTraceLevel(name);
00056     } else {
00057         minimumLevel = TRACE_OFF;
00058     }
00059 }

void TraceSource::trace	(	TraceLevel	level,
		std::string	message
	)			const `[inherited]`

Records a trace message.

Normally only called via FENNEL_TRACE.

Parameters:

	level	severity level of event being trace
	message	the text of the message

Definition at line 61 of file TraceSource.cpp.

References TraceSource::getTraceTarget(), TraceSource::isTracing(), TraceSource::name, and TraceTarget::notifyTrace().

Referenced by Calculator::exec(), and ExecStreamScheduler::traceStreamBufferContents().

00062 {
00063     if (isTracing()) {
00064         getTraceTarget().notifyTrace(name,level,message);
00065     }
00066 }

bool TraceSource::isTracing ( ) const [inline, inherited]

Returns:: true iff tracing is enabled for this source

Definition at line 88 of file TraceSource.h.

Referenced by TraceSource::initTraceSource(), CalcExecStream::prepare(), and TraceSource::trace().

00089     {
00090         return pTraceTarget.get() ? true : false;
00091     }

bool TraceSource::isTracingLevel ( TraceLevel level ) const [inline, inherited]

Determines whether a particular level is being traced.

Parameters:

level

trace level to test

Returns:: true iff tracing is enabled for the given level

Definition at line 100 of file TraceSource.h.

Referenced by ExecStreamScheduler::addGraph(), SimpleExecStreamGovernor::assignCachePages(), SimpleExecStreamGovernor::distributeCachePages(), Calculator::exec(), ExecStreamScheduler::ExecStreamScheduler(), getLastClusterPageForWrite(), moveFromTempToIndex(), JavaSinkExecStream::stuffByteBuffer(), and ExecStreamScheduler::traceStreamBuffers().

00101     {
00102         return level >= minimumLevel;
00103     }

TraceTarget& TraceSource::getTraceTarget ( ) const [inline, inherited]

Returns:: the TraceTarget for this source

Definition at line 108 of file TraceSource.h.

Referenced by TraceSource::trace().

00109     {
00110         assert(isTracing());
00111         return *(pTraceTarget.get());
00112     }

SharedTraceTarget TraceSource::getSharedTraceTarget ( ) const [inline, inherited]

Returns:: the SharedTraceTarget for this source

Definition at line 117 of file TraceSource.h.

Referenced by Database::init(), LcsClusterAppendExecStream::initLoad(), and CalcExecStream::prepare().

00118     {
00119         return pTraceTarget;
00120     }

std::string TraceSource::getTraceSourceName ( ) const [inline, inherited]

Gets the name of this source.

Useful to construct nested names for subcomponents that are also TraceSources.

Returns:: the name

Definition at line 127 of file TraceSource.h.

Referenced by LcsClusterAppendExecStream::initLoad().

00128     {
00129         return name;
00130     }

void TraceSource::setTraceSourceName ( std::string const & n ) [inline, inherited]

Sets the name of this source.

Useful to construct dynamic names for fine-grained filtering.

Definition at line 136 of file TraceSource.h.

00137     {
00138         name = n;
00139     }

TraceLevel TraceSource::getMinimumTraceLevel ( ) const [inline, inherited]

Definition at line 141 of file TraceSource.h.

00142     {
00143         return minimumLevel;
00144     }

void TraceSource::disableTracing ( ) [inherited]

Definition at line 68 of file TraceSource.cpp.

References TraceSource::minimumLevel, TraceSource::pTraceTarget, and TRACE_OFF.

Referenced by TestBase::afterTestCase().

00069 {
00070     pTraceTarget.reset();
00071     minimumLevel = TRACE_OFF;
00072 }

Member Data Documentation

SharedBTreeWriter LcsClusterNodeWriter::bTreeWriter [private]

Writes btree corresponding to cluster.

Definition at line 53 of file LcsClusterNodeWriter.h.

Referenced by allocateClusterPage(), close(), getLastClusterPageForWrite(), and LcsClusterNodeWriter().

SegmentAccessor LcsClusterNodeWriter::scratchAccessor [private]

Accessor for scratch segments.

Definition at line 58 of file LcsClusterNodeWriter.h.

Referenced by LcsClusterNodeWriter().

ClusterPageLock LcsClusterNodeWriter::bufferLock [private]

Lock on scratch page.

Definition at line 63 of file LcsClusterNodeWriter.h.

Referenced by allocArrays(), and LcsClusterNodeWriter().

PLcsClusterNode LcsClusterNodeWriter::pHdr [private]

Cluster page header.

Definition at line 68 of file LcsClusterNodeWriter.h.

Referenced by init(), LcsClusterNodeWriter(), moveFromIndexToTemp(), moveFromTempToIndex(), openAppend(), and openNew().

uint LcsClusterNodeWriter::hdrSize [private]

Size of the cluster page header.

Definition at line 73 of file LcsClusterNodeWriter.h.

Referenced by init(), LcsClusterNodeWriter(), moveFromIndexToTemp(), moveFromTempToIndex(), and openNew().

PBuffer LcsClusterNodeWriter::pIndexBlock [private]

Cluster page to be written.

Definition at line 78 of file LcsClusterNodeWriter.h.

Referenced by init(), LcsClusterNodeWriter(), moveFromIndexToTemp(), and moveFromTempToIndex().

PBuffer* LcsClusterNodeWriter::pBlock [private]

Array of pointers to temporary blocks, 1 block for each column cluster.

Definition at line 83 of file LcsClusterNodeWriter.h.

Referenced by addValue(), describeLastBatch(), getLastClusterPageForWrite(), getNextVal(), init(), LcsClusterNodeWriter(), moveFromIndexToTemp(), moveFromTempToIndex(), pickCompressionMode(), putCompressedBatch(), putFixedVarBatch(), and rollBackLastBatch().

uint LcsClusterNodeWriter::szBlock [private]

Size of the cluster page.

Definition at line 88 of file LcsClusterNodeWriter.h.

Referenced by getLastClusterPageForWrite(), getNextVal(), init(), LcsClusterNodeWriter(), moveFromIndexToTemp(), moveFromTempToIndex(), and openNew().

int LcsClusterNodeWriter::minSzLeft [private]

Minimum size left on the page.

Definition at line 93 of file LcsClusterNodeWriter.h.

Referenced by init(), and LcsClusterNodeWriter().

boost::scoped_array<LcsBatchDir> LcsClusterNodeWriter::batchDirs [private]

Batch directories for the batches currently being constructed, one per cluster column.

Definition at line 99 of file LcsClusterNodeWriter.h.

Referenced by addValue(), allocArrays(), close(), LcsClusterNodeWriter(), openAppend(), openNew(), pickCompressionMode(), putCompressedBatch(), putFixedVarBatch(), rollBackLastBatch(), and undoValue().

boost::scoped_array<PBuffer> LcsClusterNodeWriter::pValBank [private]

Temporary storage for values, used for fixed mode batches; one per cluster column.

Definition at line 105 of file LcsClusterNodeWriter.h.

Referenced by addValue(), allocArrays(), close(), LcsClusterNodeWriter(), pickCompressionMode(), and putFixedVarBatch().

boost::scoped_array<uint16_t> LcsClusterNodeWriter::oValBank [private]

First offset in the bank for each column in the cluster value bank.

Definition at line 111 of file LcsClusterNodeWriter.h.

Referenced by allocArrays(), close(), LcsClusterNodeWriter(), openAppend(), pickCompressionMode(), and putFixedVarBatch().

boost::scoped_array<uint16_t> LcsClusterNodeWriter::valBankStart [private]

Start of each cluster column in the value bank.

Definition at line 116 of file LcsClusterNodeWriter.h.

Referenced by allocArrays(), close(), LcsClusterNodeWriter(), pickCompressionMode(), and putFixedVarBatch().

boost::scoped_array<uint16_t> LcsClusterNodeWriter::batchOffset [private]

Offsets to the batch directories on the temporary pages, one per cluster column.

Definition at line 122 of file LcsClusterNodeWriter.h.

Referenced by allocArrays(), close(), describeLastBatch(), LcsClusterNodeWriter(), moveFromIndexToTemp(), moveFromTempToIndex(), openNew(), pickCompressionMode(), putCompressedBatch(), putFixedVarBatch(), and rollBackLastBatch().

boost::scoped_array<uint> LcsClusterNodeWriter::batchCount [private]

Count of the number of batches in the temporary pages, one per cluster column.

Definition at line 128 of file LcsClusterNodeWriter.h.

Referenced by allocArrays(), close(), describeLastBatch(), LcsClusterNodeWriter(), moveFromIndexToTemp(), moveFromTempToIndex(), openAppend(), openNew(), pickCompressionMode(), putCompressedBatch(), putFixedVarBatch(), and rollBackLastBatch().

int LcsClusterNodeWriter::szLeft [private]

Number of bytes left on the page.

Definition at line 133 of file LcsClusterNodeWriter.h.

Referenced by addValue(), LcsClusterNodeWriter(), openAppend(), openNew(), pickCompressionMode(), rollBackLastBatch(), and undoValue().

boost::scoped_array<uint> LcsClusterNodeWriter::nBits [private]

Number of bits required to store the value codes for each column in the cluster, for the batches currently being constructed.

Definition at line 139 of file LcsClusterNodeWriter.h.

boost::scoped_array<uint> LcsClusterNodeWriter::nextWidthChange [private]

Number of values that will cause the next nBit change for the column in the cluster.

Definition at line 145 of file LcsClusterNodeWriter.h.

Referenced by addValue(), allocArrays(), close(), LcsClusterNodeWriter(), openAppend(), openNew(), putCompressedBatch(), putFixedVarBatch(), rollBackLastBatch(), and undoValue().

bool LcsClusterNodeWriter::arraysAllocated [private]

Indicates whether temporary arrays have already been allocated.

Definition at line 150 of file LcsClusterNodeWriter.h.

Referenced by allocArrays(), and LcsClusterNodeWriter().

boost::scoped_array<ForceMode> LcsClusterNodeWriter::bForceMode [private]

Set when the mode of a batch should be forced to a particular value.

Definition at line 155 of file LcsClusterNodeWriter.h.

Referenced by addValue(), allocArrays(), close(), LcsClusterNodeWriter(), pickCompressionMode(), and putFixedVarBatch().

boost::scoped_array<uint> LcsClusterNodeWriter::forceModeCount [private]

Number of times force mode has been used for each cluster column.

Definition at line 160 of file LcsClusterNodeWriter.h.

Referenced by allocArrays(), close(), LcsClusterNodeWriter(), pickCompressionMode(), and putFixedVarBatch().

boost::scoped_array<uint> LcsClusterNodeWriter::maxValueSize [private]

Max value size encountered thus far for each cluster column.

Definition at line 165 of file LcsClusterNodeWriter.h.

Referenced by addValue(), allocArrays(), close(), LcsClusterNodeWriter(), and putFixedVarBatch().

boost::scoped_array<UnalignedAttributeAccessor> LcsClusterNodeWriter::attrAccessors [private]

Accessors for reading unaligned values.

Definition at line 170 of file LcsClusterNodeWriter.h.

Referenced by addValue(), allocArrays(), close(), getNextVal(), putCompressedBatch(), putFixedVarBatch(), rollBackLastBatch(), and undoValue().

SharedLcsClusterDump LcsClusterNodeWriter::clusterDump [private]

Cluster dump.

Definition at line 175 of file LcsClusterNodeWriter.h.

Referenced by getLastClusterPageForWrite(), LcsClusterNodeWriter(), and moveFromTempToIndex().

TupleDescriptor LcsClusterNodeWriter::colTupleDesc [private]

Tuple descriptor of the columns being loaded.

Definition at line 180 of file LcsClusterNodeWriter.h.

Referenced by allocArrays(), and LcsClusterNodeWriter().

TupleData LcsClusterAccessBase::bTreeTupleData [protected, inherited]

Tuple data representing the btree key corresponding to the cluster page.

Definition at line 42 of file LcsClusterAccessBase.h.

Referenced by allocateClusterPage(), getLastClusterPageForWrite(), LcsClusterReader::getNextBTreeEntry(), LcsClusterReader::getNumRows(), LcsClusterAccessBase::LcsClusterAccessBase(), LcsClusterReader::readClusterPage(), LcsClusterAccessBase::readClusterPageId(), LcsClusterAccessBase::readRid(), and LcsClusterReader::searchForRid().

SegmentAccessor LcsClusterAccessBase::segmentAccessor [protected, inherited]

Accessor for segment storing both btree and cluster pages.

Definition at line 47 of file LcsClusterAccessBase.h.

Referenced by allocateClusterPage(), LcsClusterAccessBase::LcsClusterAccessBase(), and LcsClusterReader::position().

ClusterPageLock LcsClusterAccessBase::clusterLock [protected, inherited]

Buffer lock for the actual cluster node pages.

Shares the same segment as the btree corresponding to the cluster.

Definition at line 53 of file LcsClusterAccessBase.h.

Referenced by allocateClusterPage(), close(), getLastClusterPageForWrite(), LcsClusterAccessBase::LcsClusterAccessBase(), LcsClusterReader::moveToBlock(), LcsClusterReader::moveToBlockWithRid(), LcsClusterReader::readClusterPage(), and LcsClusterAccessBase::unlockClusterPage().

PageId LcsClusterAccessBase::clusterPageId [protected, inherited]

Current cluster pageid.

Definition at line 58 of file LcsClusterAccessBase.h.

Referenced by allocateClusterPage(), getLastClusterPageForWrite(), LcsClusterVerifier::getPageData(), moveFromTempToIndex(), LcsClusterReader::moveToBlockWithRid(), and LcsClusterReader::readClusterPage().

LcsRid LcsClusterAccessBase::bTreeRid [protected, inherited]

Current rid in btree used to access current cluster page.

Definition at line 63 of file LcsClusterAccessBase.h.

Referenced by allocateClusterPage(), LcsClusterVerifier::getPageData(), and LcsClusterReader::readClusterPage().

uint LcsClusterAccessBase::nClusterCols [protected, inherited]

Number of columns in cluster.

Definition at line 68 of file LcsClusterAccessBase.h.

Referenced by addValue(), allocArrays(), LcsClusterDump::dump(), LcsClusterReader::getNumRows(), init(), LcsClusterReader::initColumnReaders(), LcsClusterNodeWriter(), moveFromIndexToTemp(), moveFromTempToIndex(), openAppend(), openNew(), LcsClusterReader::positionInBlock(), and LcsClusterAccessBase::setHdrOffsets().

uint16_t* LcsClusterAccessBase::lastVal [protected, inherited]

Offsets to the last value stored on the page for each column in cluster.

Definition at line 74 of file LcsClusterAccessBase.h.

Referenced by addValue(), LcsClusterDump::dump(), moveFromIndexToTemp(), moveFromTempToIndex(), openAppend(), openNew(), pickCompressionMode(), putCompressedBatch(), putFixedVarBatch(), rollBackLastBatch(), LcsClusterAccessBase::setHdrOffsets(), and undoValue().

uint16_t* LcsClusterAccessBase::firstVal [protected, inherited]

Offsets to the first value stored on the page for each column in cluster.

Points to the end of the firstVal, so subtracting lastVal from firstVal will tell you the number of bytes taken up by values for a column, since lastVals are appended in front of firstVal

Definition at line 82 of file LcsClusterAccessBase.h.

Referenced by LcsClusterDump::dump(), moveFromIndexToTemp(), moveFromTempToIndex(), openNew(), and LcsClusterAccessBase::setHdrOffsets().

uint* LcsClusterAccessBase::nVal [protected, inherited]

Number of distinct values in the page for each column in cluster.

Definition at line 87 of file LcsClusterAccessBase.h.

uint16_t* LcsClusterAccessBase::delta [protected, inherited]

For each column in the cluster, offset used to get the real offset within the page.

Definition at line 93 of file LcsClusterAccessBase.h.

Referenced by LcsClusterDump::dump(), moveFromTempToIndex(), openNew(), LcsClusterAccessBase::setHdrOffsets(), and LcsColumnReader::sync().

The documentation for this class was generated from the following files:

/home/pub/open/dev/fennel/lucidera/colstore/LcsClusterNodeWriter.h
/home/pub/open/dev/fennel/lucidera/colstore/LcsClusterNodeWriter.cpp

Generated on Mon Jun 22 04:00:37 2009 for Fennel by

1.5.1

LcsClusterNodeWriter Class Reference

Public Member Functions

Protected Member Functions

Protected Attributes

Private Member Functions

Private Attributes

Detailed Description

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation