LbmGeneratorExecStream Class Reference

LbmGeneratorExecStream reads the column store clusters corresponding to the columns of an index, and creates bitmap entries for those rows. More...

#include <LbmGeneratorExecStream.h>

Inheritance diagram for LbmGeneratorExecStream:

List of all members.


Public Member Functions
virtual void	prepare (LbmGeneratorExecStreamParams const &params)
virtual void	open (bool restart)
	Opens this stream, acquiring any resources needed in order to be able to fetch data.
virtual ExecStreamResult	execute (ExecStreamQuantum const &quantum)
	Executes this stream.
virtual void	getResourceRequirements (ExecStreamResourceQuantity &minQuantity, ExecStreamResourceQuantity &optQuantity, ExecStreamResourceSettingType &optType)
	Determines resource requirements for this stream.
virtual void	setResourceAllocation (ExecStreamResourceQuantity &quantity)
	Sets current resource allocation for this stream.
virtual void	closeImpl ()
	Implements ClosableObject.
virtual void	prepare (BTreeExecStreamParams const &params)
virtual void	prepare (SingleOutputExecStreamParams const &params)
virtual void	prepare (ExecStreamParams const &params)
	Prepares this stream for execution.
virtual void	setInputBufAccessors (std::vector< SharedExecStreamBufAccessor > const &inAccessors)
	Initializes the buffer accessors for inputs to this stream.
virtual void	setOutputBufAccessors (std::vector< SharedExecStreamBufAccessor > const &outAccessors)
	Initializes the buffer accessors for outputs from this stream.
virtual ExecStreamBufProvision	getOutputBufProvision () const
	Queries the BufferProvision which this stream is capable of when producing tuples.
virtual bool	canEarlyClose ()
	Returns: true if the stream can be closed early
ExecStreamGraph &	getGraph () const
	Returns: reference to containing graph
ExecStreamId	getStreamId () const
	Returns: the identifier for this stream within containing graph
virtual void	getResourceRequirements (ExecStreamResourceQuantity &minQuantity, ExecStreamResourceQuantity &optQuantity)
virtual void	setName (std::string const &)
	Sets unique name of this stream.
virtual std::string const &	getName () const
	Returns: the name of this stream, as known by the optimizer
virtual bool	mayBlock () const
	Queries whether this stream's implementation may block when execute() is called.
virtual void	checkAbort () const
	Checks whether there is an abort request for this stream's scheduler.
virtual ExecStreamBufProvision	getOutputBufConversion () const
	Queries the BufferProvision to which this stream needs its output to be converted, if any.
virtual ExecStreamBufProvision	getInputBufProvision () const
	Queries the BufferProvision which this stream requires of its inputs when consuming their tuples.
bool	isClosed () const
	Returns: whether the object has been closed
void	close ()
	Closes this object, releasing any unallocated resources.
virtual void	initTraceSource (SharedTraceTarget pTraceTarget, std::string name)
	For use when initialization has to be deferred until after construction.
void	trace (TraceLevel level, std::string message) const
	Records a trace message.
bool	isTracing () const
	Returns: true iff tracing is enabled for this source
bool	isTracingLevel (TraceLevel level) const
	Determines whether a particular level is being traced.
TraceTarget &	getTraceTarget () const
	Returns: the TraceTarget for this source
SharedTraceTarget	getSharedTraceTarget () const
	Returns: the SharedTraceTarget for this source
std::string	getTraceSourceName () const
	Gets the name of this source.
void	setTraceSourceName (std::string const &n)
	Sets the name of this source.
TraceLevel	getMinimumTraceLevel () const
void	disableTracing ()
virtual void	initErrorSource (SharedErrorTarget pErrorTarget, const std::string &name)
	For use when initialization has to be deferred until after construction.
void	postError (ErrorLevel level, const std::string &message, void *address, long capacity, int index)
	Posts an exception, such as a row exception.
void	postError (ErrorLevel level, const std::string &message, const TupleDescriptor &errorDesc, const TupleData &errorTuple, int index)
	Posts an exception, such as a row exception.
bool	hasTarget () const
	Returns: true iff an error target has been set
ErrorTarget &	getErrorTarget () const
	Returns: the ErrorTarget for this source
SharedErrorTarget	getSharedErrorTarget () const
	Returns: the SharedErrorTarget for this source
std::string	getErrorSourceName () const
	Gets the name of this source.
void	setErrorSourceName (std::string const &n)
	Sets the name of this source.
void	disableTarget ()
Static Public Member Functions
static SharedBTreeWriter	newWriter (BTreeExecStreamParams const &params)
static void	copyParamsToDescriptor (BTreeDescriptor &, BTreeParams const &, SharedCacheAccessor const &)
Protected Member Functions
virtual SharedBTreeReader	newReader ()
SharedBTreeWriter	newWriter (bool monotonic=false)
virtual void	endSearch ()
	Forgets the current reader or writer's search, releasing any page locks.
Protected Attributes
BTreeDescriptor	treeDescriptor
SegmentAccessor	scratchAccessor
BTreeOwnerRootMap *	pRootMap
SharedBTreeAccessBase	pBTreeAccessBase
SharedBTreeReader	pBTreeReader
DynamicParamId	rootPageIdParamId
SharedExecStreamBufAccessor	pOutAccessor
bool	isOpen
	Whether this stream is currently open.
ExecStreamGraph *	pGraph
	Dataflow graph containing this stream.
ExecStreamId	id
	Identifier for this stream; local to its containing graph.
std::string	name
	Name of stream, as known by optimizer.
SharedDynamicParamManager	pDynamicParamManager
	The dynamic parameter manager available to this stream.
SharedLogicalTxn	pTxn
	The transaction embracing the stream.
ExecStreamResourceQuantity	resourceAllocation
	Resource quantities currently allocated to this stream.
SharedCacheAccessor	pQuotaAccessor
	CacheAccessor used for quota tracking.
SharedCacheAccessor	pScratchQuotaAccessor
	CacheAccessor used for scratch page quota tracking.
bool	needsClose
Private Member Functions
ExecStreamResult	generateSingleKeyBitmaps (ExecStreamQuantum const &quantum)
	Generates bitmaps for a single column index.
ExecStreamResult	generateMultiKeyBitmaps (ExecStreamQuantum const &quantum)
	Generates bitmaps for multi-column indexes.
void	createSingletonBitmapEntry ()
	Creates a singleton bitmap entry and resets state to indicate that the current row has been processed if successful.
bool	generateBitmaps ()
	Reads values from a single batch corresponding to a compressed batch and generates bitmap entries.
bool	generateSingletons ()
	Reads values from a single batch corresponding to a non-compressed batch and generates singleton bitmap entries.
bool	advanceReader (SharedLcsClusterReader &pScan)
	Advances a single cluster and its corresponding column readers one rid forward in the current batch.
bool	initBitmapTable (uint nEntries)
	Initializes bitmap table, increasing the size, as needed and assigns buffers to bitmap entries.
void	initRidAndBitmap (TupleData &bitmapTuple, LcsRid *pCurrRid)
	Sets rid in a bitmap tupledata.
bool	addRidToBitmap (uint keycode, TupleData &keyvalue, LcsRid rid)
	Adds rid to a bitmap entry being constructing for a specified key.
PBuffer	flushBuffer (LcsRid addRid)
	Flushes out an existing buffer currently in use by another LbmEntry.
bool	flushTable (uint start)
	Flushes out entire table of bitmap entries.
bool	flushEntry (uint keycode)
	Flushes a single entry in the bitmap table and resets the LbmEntry associated with the table entry.
void	syncColumns (SharedLcsClusterReader &pScan)
	Positions column readers based on new cluster reader position.
bool	readColVals (SharedLcsClusterReader &pScan, TupleDataWithBuffer &tupleData, uint colStart)
	Reads column values based on current position of cluster reader.
virtual void	buildOutputProj (TupleProjection &outputProj, LcsRowScanBaseExecStreamParams const &params)
	Builds outputProj from params.
virtual void	prepare (LcsRowScanBaseExecStreamParams const &params)
virtual void	prepare (ConfluenceExecStreamParams const &params)
virtual void	getResourceRequirements (ExecStreamResourceQuantity &minQuantity, ExecStreamResourceQuantity &optQuantity)
virtual void	setInputBufAccessors (std::vector< SharedExecStreamBufAccessor > const &inAccessors)
	Initializes the buffer accessors for inputs to this stream.
virtual ExecStreamBufProvision	getInputBufProvision () const
	Queries the BufferProvision which this stream requires of its inputs when consuming their tuples.
Private Attributes
uint	numMiscScratchPages
	Number of misc scratch pages needed; excludes pages for generating bitmap entries.
uint	maxNumScratchPages
	Number of scratch pages to allocate for constructing bitmaps as determined by the resource governor.
DynamicParamId	insertRowCountParamId
	Dynamic parameter id used to pass along number of rows loaded.
bool	createIndex
	True if index is being loaded as part of a create index statement.
std::vector< DynamicParamId >	parameterIds
	Ordered list of dynamic parameter ids for the clusters that provide source data for this stream.
uint	scratchPageSize
	Size of a scratch page.
uint	entrySize
	Size of a bitmap entry buffer.
uint	maxBitmapSize
	Max size of a bitmap entry.
uint	minBitmapSize
	Min size of a bitmap entry.
ClusterPageLock	scratchLock
	Lock for scratch accessor.
TupleData	inputTuple
	Input tuple data.
RecordNum	numRowsToLoad
	Number of rows to load.
RecordNum	rowCount
	Running count of number of rows read.
LcsRid	startRid
	Starting rid;.
LcsRid	currRid
	Current rid being loaded.
TupleDataWithBuffer	bitmapTuple
	Tuple data with buffer for the bitmap tuple.
TupleDescriptor	bitmapTupleDesc
	Tuple descriptor representing bitmap tuple.
TupleData	outputTuple
	Pointer to generated tuple data.
uint	nIdxKeys
	Number of keys in the bitmap index, excluding the starting rid.
bool	batchRead
	True if current batch has been read.
uint	currBatch
	Current batch entry being processed.
std::vector< uint16_t >	keyCodes
	Keycodes read from a batch.
std::vector< LbmEntryInfo >	bitmapTable
	Table of bitmap entries under construction.
uint	nBitmapEntries
	Number of entries in the bitmap table.
uint	flushIdx
	Index of buffer entry to flush.
uint	nBitmapBuffers
	Number of entries in the bitmap buffer table.
uint	nScratchPagesAllocated
	Number of scratch pages allocated.
std::vector< PBuffer >	scratchPages
	Vector of pointers to scratch pages allocated.
LbmPendingProduceType	producePending
	Produce of one or more output tuples pending: LBM_TABLEFLUSH_PENDING, LBM_ENTRYFLUSH_PENDING, LBM_FINALFLUSH_PENDING, LBM_NOFLUSH_PENDING.
uint	flushStart
	Index into bitmap table from which to start a pending table flush or the single entry currently being flushed.
bool	skipRead
	If true, skip the initial read the next time generator is called since we haven't finished processing the current rowid.
bool	doneReading
	If true, all rows from the column store table have been read.
bool	revertToSingletons
	If true, even though the current batch is compressed and the index has a single key, generate singleton bitmaps for the batch due to lack of buffer space.
VectorOfUint	projMap
	Projection map that maps columns read from cluster to their position in the output projection.
uint	nClusters
	Number of clusters to be scanned.
boost::scoped_array< SharedLcsClusterReader >	pClusters
	Array containing cluster readers.
TupleDescriptor	projDescriptor
	Tuple descriptor representing columns to be projected from scans.
std::vector< int >	nonClusterCols
	List of the non-cluster columns that need to be projected.
bool	allSpecial
	True in the special case where we are only reading special columns.
CircularBuffer< LcsRidRun >	ridRuns
	Circular buffer of rid runs.
std::vector< UnalignedAttributeAccessor >	attrAccessors
	Accessors used for loading actual column values.
std::vector< SharedExecStreamBufAccessor >	inAccessors

Detailed Description

LbmGeneratorExecStream reads the column store clusters corresponding to the columns of an index, and creates bitmap entries for those rows.

Its input is a rowcount, indicating the number of rows it needs to create entries for, and the startRid corresponding to the first row. It's assumed that the rows it needs to create entries for are all sequential, starting at that start value.

If possible, LbmGeneratorExecStream will try to take advantage of compressed column store batches to create bitmap entries spanning multiple rows.

As output, LbmGeneratorExecStream writes out the bitmap entries it has created. It also passes along the rowcount passed in as its input, using a dynamic parameter that will be read downstream.

Author:: Zelaine Fong

Version:

Id: //open/dev/fennel/lucidera/bitmap/LbmGeneratorExecStream.h#15

Definition at line 92 of file LbmGeneratorExecStream.h.

Member Function Documentation

ExecStreamResult LbmGeneratorExecStream::generateSingleKeyBitmaps ( ExecStreamQuantum const & quantum ) [private]

Generates bitmaps for a single column index.

Parameters:

quantum

quantum for stream

Returns:: either EXECRC_BUF_OVERFLOW or EXECRC_EOS

Definition at line 260 of file LbmGeneratorExecStream.cpp.

References batchRead, EXECRC_BUF_OVERFLOW, EXECRC_EOS, EXECRC_QUANTUM_EXPIRED, generateBitmaps(), generateSingletons(), ExecStreamQuantum::nTuplesMax, LcsRowScanBaseExecStream::pClusters, and revertToSingletons.

Referenced by execute().

00262 {
00263     // read from the current batch until either the end of the batch
00264     // is reached, or there is an overflow in a write to the output stream
00265     for (uint i = 0; i < quantum.nTuplesMax; i++) {
00266         if (!revertToSingletons &&
00267             pClusters[0]->clusterCols[0].batchIsCompressed())
00268         {
00269             if (!generateBitmaps()) {
00270                 return EXECRC_BUF_OVERFLOW;
00271             }
00272         } else if (!batchRead) {
00273             if (!generateSingletons()) {
00274                 return EXECRC_BUF_OVERFLOW;
00275             }
00276         }
00277 
00278         // move to the next batch
00279         batchRead = false;
00280         revertToSingletons = false;
00281         SharedLcsClusterReader &pScan = pClusters[0];
00282         if (!pScan->nextRange()) {
00283             return EXECRC_EOS;
00284         }
00285         pScan->clusterCols[0].sync();
00286     }
00287     return EXECRC_QUANTUM_EXPIRED;
00288 }

ExecStreamResult LbmGeneratorExecStream::generateMultiKeyBitmaps ( ExecStreamQuantum const & quantum ) [private]

Generates bitmaps for multi-column indexes.

Parameters:

quantum

quantum for stream

Returns:: either EXECRC_BUF_OVERFLOW or EXECRC_EOS

Definition at line 290 of file LbmGeneratorExecStream.cpp.

References bitmapTuple, createSingletonBitmapEntry(), currRid, EXECRC_BUF_OVERFLOW, EXECRC_EOS, EXECRC_QUANTUM_EXPIRED, flushEntry(), LcsRowScanBaseExecStream::nClusters, ExecStreamQuantum::nTuplesMax, opaqueToInt(), LcsRowScanBaseExecStream::pClusters, LcsRowScanBaseExecStream::readColVals(), TupleDataWithBuffer::resetBuffer(), skipRead, and LcsRowScanBaseExecStream::syncColumns().

Referenced by execute().

00292 {
00293     // read through all rows until the end of the table has been reached,
00294     // or there is an overflow in a write to the output stream
00295     for (uint i = 0; i < quantum.nTuplesMax; i++) {
00296         uint prevClusterEnd = 0;
00297         if (!skipRead) {
00298             // reset buffer before loading new values, in case previous
00299             // row contained nulls
00300             bitmapTuple.resetBuffer();
00301             for (uint iClu = 0; iClu < nClusters; iClu++) {
00302                 SharedLcsClusterReader &pScan = pClusters[iClu];
00303 
00304                 if (currRid >= pScan->getRangeEndRid()) {
00305                     // move to the next batch if this particular cluster
00306                     // reader has reached the end of its batch
00307                     if (!pScan->nextRange()) {
00308                         assert(
00309                             iClu == 0 &&
00310                                 (nClusters == 1 || !pClusters[1]->nextRange()));
00311                         return EXECRC_EOS;
00312                     }
00313                     assert(
00314                         currRid >= pScan->getRangeStartRid() &&
00315                         currRid < pScan->getRangeEndRid());
00316                     syncColumns(pScan);
00317                 } else {
00318                     assert(currRid >= pScan->getRangeStartRid());
00319                     pScan->advanceWithinBatch(
00320                         opaqueToInt(currRid - pScan->getCurrentRid()));
00321                 }
00322                 readColVals(
00323                     pScan,
00324                     bitmapTuple,
00325                     prevClusterEnd);
00326                 prevClusterEnd += pScan->nColsToRead;
00327             }
00328         }
00329 
00330         createSingletonBitmapEntry();
00331         if (!flushEntry(0)) {
00332             return EXECRC_BUF_OVERFLOW;
00333         }
00334     }
00335     return EXECRC_QUANTUM_EXPIRED;
00336 }

void LbmGeneratorExecStream::createSingletonBitmapEntry ( ) [private]

Creates a singleton bitmap entry and resets state to indicate that the current row has been processed if successful.

Definition at line 338 of file LbmGeneratorExecStream.cpp.

References addRidToBitmap(), bitmapTuple, currRid, initRidAndBitmap(), rowCount, and skipRead.

Referenced by generateMultiKeyBitmaps(), and generateSingletons().

00339 {
00340     // create the singleton bitmap entry and then flush it out
00341     // right away; should never fail trying to create a singleton
00342     // entry
00343     initRidAndBitmap(bitmapTuple, &currRid);
00344     bool rc = addRidToBitmap(0, bitmapTuple, currRid);
00345     assert(rc);
00346     skipRead = false;
00347     ++currRid;
00348     ++rowCount;
00349 }

bool LbmGeneratorExecStream::generateBitmaps ( ) [private]

Reads values from a single batch corresponding to a compressed batch and generates bitmap entries.

Returns:: true if successfully processed batch; false if overflow occurred while writing out tuples

Definition at line 365 of file LbmGeneratorExecStream.cpp.

References addRidToBitmap(), LcsRowScanBaseExecStream::attrAccessors, batchRead, bitmapTuple, currBatch, currRid, flushTable(), generateSingletons(), LcsColumnReader::getBatchValCount(), LcsColumnReader::getBatchValue(), initBitmapTable(), initRidAndBitmap(), keyCodes, LcsRowScanBaseExecStream::pClusters, LcsColumnReader::readCompressedBatch(), TupleDataWithBuffer::resetBuffer(), revertToSingletons, rowCount, and skipRead.

Referenced by generateSingleKeyBitmaps().

00366 {
00367     // in the single key case, the column reader is always the first
00368     // one, in the first cluster reader
00369     LcsColumnReader &colReader = pClusters[0]->clusterCols[0];
00370     uint nDistinctVals = colReader.getBatchValCount();
00371 
00372     // only read rows beginning at startRid
00373     uint nRows = pClusters[0]->getRangeRowsLeft();
00374 
00375     // if first time through, setup the keycode array and read the batch
00376     if (!batchRead) {
00377         uint nRead;
00378 
00379         // if there's insufficient buffer space, revert to generating
00380         // singletons for this batch
00381         if (!initBitmapTable(nDistinctVals)) {
00382             revertToSingletons = true;
00383             return generateSingletons();
00384         }
00385 
00386         keyCodes.resize(nRows);
00387         colReader.readCompressedBatch(nRows, &keyCodes[0], &nRead);
00388         assert(nRows == nRead);
00389 
00390         batchRead = true;
00391         currBatch = 0;
00392     }
00393 
00394     // resume reading batch values based on where we last left off;
00395     // if the value has been read but not yet processed, skip the read
00396     for (uint i = currBatch; i < nRows; i++) {
00397         if (!skipRead) {
00398             PBuffer curValue = colReader.getBatchValue(keyCodes[i]);
00399             // reset buffer before loading new value, in case previous
00400             // row had nulls
00401             bitmapTuple.resetBuffer();
00402 
00403             attrAccessors[0].loadValue(bitmapTuple[0], curValue);
00404             initRidAndBitmap(bitmapTuple, &currRid);
00405         }
00406         if (!addRidToBitmap(keyCodes[i], bitmapTuple, currRid)) {
00407             currBatch = i;
00408             skipRead = true;
00409             return false;
00410         }
00411         ++currRid;
00412         ++rowCount;
00413         skipRead = false;
00414     }
00415 
00416     // flush out table since the next batch will have a different set
00417     // of keycodes
00418     if (!flushTable(0)) {
00419         // set currBatch to avoid re-reading column values above
00420         // when return back into this method
00421         currBatch = nRows;
00422         return false;
00423     }
00424 
00425     return true;
00426 }

bool LbmGeneratorExecStream::generateSingletons ( ) [private]

Reads values from a single batch corresponding to a non-compressed batch and generates singleton bitmap entries.

Returns:: true if successfully processed batch; false if overflow occurred while writing out tuples

Definition at line 428 of file LbmGeneratorExecStream.cpp.

References advanceReader(), LcsRowScanBaseExecStream::attrAccessors, batchRead, bitmapTuple, createSingletonBitmapEntry(), flushEntry(), LcsRowScanBaseExecStream::pClusters, LcsRowScanBaseExecStream::projMap, TupleDataWithBuffer::resetBuffer(), and skipRead.

Referenced by generateBitmaps(), and generateSingleKeyBitmaps().

00429 {
00430     SharedLcsClusterReader &pScan = pClusters[0];
00431 
00432     do {
00433         // if we've already read the row but haven't processed it yet,
00434         // skip the read
00435         if (!skipRead) {
00436             uint prevClusterEnd = 0;
00437 
00438             // reset buffer before loading new values, in case previous
00439             // row contained nulls
00440             bitmapTuple.resetBuffer();
00441 
00442             for (uint iCluCol = 0; iCluCol < pScan->nColsToRead; iCluCol++) {
00443                 PBuffer curValue =
00444                     pScan->clusterCols[iCluCol].getCurrentValue();
00445                 uint idx = projMap[prevClusterEnd + iCluCol];
00446 
00447                 attrAccessors[idx].loadValue(bitmapTuple[idx], curValue);
00448             }
00449             prevClusterEnd += pScan->nColsToRead;
00450         }
00451 
00452         createSingletonBitmapEntry();
00453         if (!flushEntry(0)) {
00454             // advance now so the next time we come in here, we'll
00455             // be correctly positioned on the next rid
00456             if (!advanceReader(pScan)) {
00457                 // if we're at the end of the batch, avoid coming
00458                 // back in here until the new batch is read
00459                 batchRead = true;
00460             }
00461             return false;
00462         }
00463 
00464         // advance to the next rid; if at the end of the batch,
00465         // return to caller; else, continue reading from current
00466         // batch
00467         if (!advanceReader(pScan)) {
00468             return true;
00469         }
00470     } while (true);
00471 }

bool LbmGeneratorExecStream::advanceReader ( SharedLcsClusterReader & pScan ) [private]

Advances a single cluster and its corresponding column readers one rid forward in the current batch.

Parameters:

pScan

cluster reader to advance

Returns:: true if still within current batch; false if reached end of batch

Definition at line 473 of file LbmGeneratorExecStream.cpp.

References LcsRowScanBaseExecStream::syncColumns().

Referenced by generateSingletons().

00474 {
00475     if (!pScan->advance(1)) {
00476         return false;
00477     }
00478     syncColumns(pScan);
00479     return true;
00480 }

bool LbmGeneratorExecStream::initBitmapTable ( uint nEntries ) [private]

Initializes bitmap table, increasing the size, as needed and assigns buffers to bitmap entries.

Parameters:

nEntries

desired size of the table

Returns:: true if the table was successfully initialized, indicating that there was enough space to accomodate the number of keys in the current batch

Definition at line 482 of file LbmGeneratorExecStream.cpp.

References SegNodeLock< Node >::allocatePage(), bitmapTable, entrySize, flushIdx, SegPageLock::getPage(), CachePage::getWritableData(), maxBitmapSize, maxNumScratchPages, minBitmapSize, nBitmapEntries, nScratchPagesAllocated, scratchLock, scratchPages, and scratchPageSize.

Referenced by execute(), and generateBitmaps().

00483 {
00484     // compute the size of the bitmap buffers, based on the number
00485     // of scratch pages available and the number of distinct values
00486     // in the batch
00487     uint nBufsPerPage = (uint) ceil((double) nEntries / maxNumScratchPages);
00488     uint currSize = scratchPageSize / nBufsPerPage;
00489 
00490     if (currSize < minBitmapSize) {
00491         currSize = minBitmapSize;
00492         nBufsPerPage = scratchPageSize / currSize;
00493     } else if (currSize > maxBitmapSize) {
00494         currSize = maxBitmapSize;
00495         nBufsPerPage = scratchPageSize / currSize;
00496     }
00497 
00498     // If there are less than 8 buffers, then there cannot be more keys than
00499     // buffers.  That's because we need to avoid flushing those buffers that
00500     // potentially overlap in the last byte with the upcoming rids being
00501     // processed.
00502     uint nBuffers = nBufsPerPage * maxNumScratchPages;
00503     if (nBuffers < 8 && nEntries > nBuffers) {
00504         return false;
00505     }
00506 
00507     if (nEntries > nBitmapEntries) {
00508         // resize bitmap table to accomodate new batch, which has more
00509         // distinct values
00510         bitmapTable.resize(nEntries);
00511         for (uint i = nBitmapEntries; i < nEntries; i++) {
00512             bitmapTable[i].pBitmap = SharedLbmEntry(new LbmEntry());
00513         }
00514     }
00515 
00516     if (nEntries != nBitmapEntries) {
00517         // divide up the buffers across the bitmap table; if there are
00518         // not enough buffers, the bitmap entries at the end of the table
00519         // won't have assigned buffers yet; no need to re-divide the buffers
00520         // if the previous bitmap table had the same number of entries
00521 
00522         uint nPages = (uint) ceil((double) nEntries / nBufsPerPage);
00523         if (nPages > maxNumScratchPages) {
00524             nPages = maxNumScratchPages;
00525         }
00526         while (nPages > nScratchPagesAllocated) {
00527             scratchLock.allocatePage();
00528             PBuffer newPage = scratchLock.getPage().getWritableData();
00529             scratchPages.push_back(newPage);
00530             ++nScratchPagesAllocated;
00531         }
00532         uint idx = 0;
00533         for (uint i = 0; i < nPages; i++) {
00534             uint offset = 0;
00535             for (uint j = 0; j < nBufsPerPage; j++) {
00536                 idx = i * nBufsPerPage + j;
00537                 if (idx == nEntries) {
00538                     break;
00539                 }
00540                 bitmapTable[idx].bufferPtr = scratchPages[i] + offset;
00541                 offset += currSize;
00542             }
00543             if (idx == nEntries) {
00544                 break;
00545             }
00546         }
00547         // entries without assigned buffers
00548         for (uint i = idx + 1; i < nEntries; i++) {
00549             bitmapTable[i].bufferPtr = NULL;
00550         }
00551     }
00552 
00553     for (uint i = 0; i < nEntries; i++) {
00554         bitmapTable[i].inuse = false;
00555     }
00556     flushIdx = 0;
00557     nBitmapEntries = nEntries;
00558     entrySize = currSize;
00559 
00560     return true;
00561 }

void LbmGeneratorExecStream::initRidAndBitmap	(	TupleData &	bitmapTuple,
		LcsRid *	pCurrRid
	)			`[private]`

Sets rid in a bitmap tupledata.

Also initializes bitmap fields to NULL

Parameters:

	bitmapTuple	tupledata to be set
	pCurrRid	pointer to the rid value that will be set in tupledata

Definition at line 563 of file LbmGeneratorExecStream.cpp.

References bitmapTuple, and nIdxKeys.

Referenced by createSingletonBitmapEntry(), and generateBitmaps().

00565 {
00566     bitmapTuple[nIdxKeys].pData = (PConstBuffer) pCurrRid;
00567     bitmapTuple[nIdxKeys + 1].pData = NULL;
00568     bitmapTuple[nIdxKeys + 2].pData = NULL;
00569 }

bool LbmGeneratorExecStream::addRidToBitmap	(	uint	keycode,
		TupleData &	keyvalue,
		LcsRid	rid
	)			`[private]`

Adds rid to a bitmap entry being constructing for a specified key.

Creates a new bitmap entry, as needed, and flushes out entries as buffer space fills out.

Parameters:

	keycode	keycode corresponding to the value of the rid being added; always 0 in the case of a batch that is either non-compressed or has more than 1 key
	keyvalue	tupledata containing the initial bitmap entry value; i.e., only the key value and rid are set
	rid	rid to be added

Returns:: false if an error was encountered flushing out a bitmap entry

Definition at line 571 of file LbmGeneratorExecStream.cpp.

References bitmapTable, bitmapTupleDesc, entrySize, flushBuffer(), flushEntry(), and nBitmapEntries.

Referenced by createSingletonBitmapEntry(), and generateBitmaps().

00573 {
00574     assert(keycode <= nBitmapEntries);
00575 
00576     if (bitmapTable[keycode].inuse) {
00577         assert(bitmapTable[keycode].bufferPtr);
00578 
00579         bool maxedOut = !bitmapTable[keycode].pBitmap->setRID(rid);
00580         if (maxedOut) {
00581             if (!flushEntry(keycode)) {
00582                 return false;
00583             }
00584             assert(!bitmapTable[keycode].inuse);
00585             bitmapTable[keycode].inuse = true;
00586             // buffer should now have enough space; create a singleton
00587             // entry
00588             bitmapTable[keycode].pBitmap->setEntryTuple(initBitmap);
00589         }
00590     } else {
00591         if (!bitmapTable[keycode].bufferPtr) {
00592             // no assigned buffer yet; get a buffer by flushing
00593             // out an existing entry
00594             PBuffer bufPtr = flushBuffer(rid);
00595             if (!bufPtr) {
00596                 return false;
00597             }
00598             bitmapTable[keycode].bufferPtr = bufPtr;
00599         }
00600         // now that we have a buffer, initialize the entry
00601         bitmapTable[keycode].pBitmap->init(
00602             bitmapTable[keycode].bufferPtr, NULL, entrySize, bitmapTupleDesc);
00603         bitmapTable[keycode].pBitmap->setEntryTuple(initBitmap);
00604         bitmapTable[keycode].inuse = true;
00605     }
00606 
00607     return true;
00608 }

PBuffer LbmGeneratorExecStream::flushBuffer ( LcsRid addRid ) [private]

Flushes out an existing buffer currently in use by another LbmEntry.

Parameters:

addRid

the current rid being added that requires a buffer flush

Returns:: pointer to buffer if flush of entry was successful; otherwise NULL if an overflow occurred when writing the entry to the output stream

Definition at line 610 of file LbmGeneratorExecStream.cpp.

References bitmapTable, flushEntry(), flushIdx, and nBitmapEntries.

Referenced by addRidToBitmap().

00611 {
00612     // need to flush a buffer out and return that buffer for use; for now,
00613     // cycle through the entries in round robin order in determining which
00614     // to flush
00615     //
00616     // NOTE zfong 6-Jan-2006: We may want to change this to a more
00617     // sophisticated scheme where we flush on a LRU basis
00618     PBuffer retPtr;
00619     uint nAttempts = 0;
00620     do {
00621         ++nAttempts;
00622         if (nAttempts > nBitmapEntries) {
00623             // we should always have enough buffers so we can flush at least
00624             // one existing entry
00625             permAssert(false);
00626         }
00627         if (bitmapTable[flushIdx].bufferPtr) {
00628             retPtr = bitmapTable[flushIdx].bufferPtr;
00629             if (bitmapTable[flushIdx].inuse) {
00630                 // skip over entries whose rid range overlaps with the rid
00631                 // that will be added next, since we potentially may need
00632                 // to add that rid (or one that follows and is within the
00633                 // same rid range) into that entry
00634                 if (bitmapTable[flushIdx].pBitmap->inRange(addRid)) {
00635                     flushIdx = ++flushIdx % nBitmapEntries;
00636                     continue;
00637                 }
00638                 if (!flushEntry(flushIdx)) {
00639                     return NULL;
00640                 }
00641             }
00642             bitmapTable[flushIdx].bufferPtr = NULL;
00643             break;
00644         }
00645         flushIdx = ++flushIdx % nBitmapEntries;
00646     } while (true);
00647 
00648     flushIdx = ++flushIdx % nBitmapEntries;
00649     return retPtr;
00650 }

bool LbmGeneratorExecStream::flushTable ( uint start ) [private]

Flushes out entire table of bitmap entries.

Parameters:

start

index into bitmap table from which to start flushing entries

Returns:: true if all entries successfully written to output stream

Definition at line 652 of file LbmGeneratorExecStream.cpp.

References bitmapTable, flushEntry(), LBM_TABLEFLUSH_PENDING, nBitmapEntries, and producePending.

Referenced by execute(), and generateBitmaps().

00653 {
00654     assert(start <= nBitmapEntries);
00655     for (uint i = start; i < nBitmapEntries; i++) {
00656         if (bitmapTable[i].inuse) {
00657             if (!flushEntry(i)) {
00658                 producePending = LBM_TABLEFLUSH_PENDING;
00659                 return false;
00660             }
00661         }
00662     }
00663 
00664     return true;
00665 }

bool LbmGeneratorExecStream::flushEntry ( uint keycode ) [private]

Flushes a single entry in the bitmap table and resets the LbmEntry associated with the table entry.

Parameters:

keycode

index of the bitmap entry to be flushed

Returns:: true if entries was successfully written to output stream

Definition at line 667 of file LbmGeneratorExecStream.cpp.

References bitmapTable, flushStart, LBM_ENTRYFLUSH_PENDING, outputTuple, SingleOutputExecStream::pOutAccessor, producePending, LbmEntry::toString(), and TRACE_FINE.

Referenced by addRidToBitmap(), flushBuffer(), flushTable(), generateMultiKeyBitmaps(), and generateSingletons().

00668 {
00669     assert(bitmapTable[keycode].inuse && bitmapTable[keycode].bufferPtr);
00670 
00671     // retrieve the generated bitmap entry and write it to the output
00672     // stream
00673 
00674     outputTuple = bitmapTable[keycode].pBitmap->produceEntryTuple();
00675 
00676     FENNEL_TRACE(TRACE_FINE, LbmEntry::toString(outputTuple));
00677 
00678     if (!pOutAccessor->produceTuple(outputTuple)) {
00679         flushStart = keycode;
00680         producePending = LBM_ENTRYFLUSH_PENDING;
00681         return false;
00682     }
00683 
00684     // entry no longer is associated with an entry tuple
00685     bitmapTable[keycode].inuse = false;
00686 
00687     return true;
00688 }

void LbmGeneratorExecStream::prepare ( LbmGeneratorExecStreamParams const & params ) [virtual]

Definition at line 29 of file LbmGeneratorExecStream.cpp.

References SegPageLock::accessSegment(), LcsRowScanBaseExecStream::attrAccessors, bitmapTuple, bitmapTupleDesc, TupleData::compute(), TupleDataWithBuffer::computeAndAllocate(), LbmGeneratorExecStreamParams::createIndex, createIndex, LbmEntry::getSizeBounds(), ConfluenceExecStream::inAccessors, inputTuple, LbmGeneratorExecStreamParams::insertRowCountParamId, insertRowCountParamId, BTreeDescriptor::keyProjection, LcsRowScanBaseExecStreamParams::lcsClusterScanDefs, maxBitmapSize, minBitmapSize, LcsRowScanBaseExecStream::nClusters, nIdxKeys, opaqueToInt(), parameterIds, SingleOutputExecStream::pOutAccessor, LcsRowScanBaseExecStream::prepare(), BTreeExecStream::prepare(), SegmentAccessor::pSegment, CircularBuffer< T >::resize(), LcsRowScanBaseExecStream::ridRuns, BTreeExecStream::scratchAccessor, scratchLock, scratchPageSize, BTreeDescriptor::segmentAccessor, BTreeExecStream::treeDescriptor, and BTreeDescriptor::tupleDescriptor.

00030 {
00031     BTreeExecStream::prepare(params);
00032     LcsRowScanBaseExecStream::prepare(params);
00033 
00034     insertRowCountParamId = params.insertRowCountParamId;
00035     assert(opaqueToInt(insertRowCountParamId) > 0);
00036 
00037     createIndex = params.createIndex;
00038     parameterIds.resize(nClusters);
00039     for (uint i = 0; i < nClusters; i++) {
00040         parameterIds[i] = params.lcsClusterScanDefs[i].rootPageIdParamId;
00041     }
00042 
00043     scratchLock.accessSegment(scratchAccessor);
00044     scratchPageSize = scratchAccessor.pSegment->getUsablePageSize();
00045 
00046     // setup input tuple -- numRowsToLoad, startRid
00047     assert(inAccessors[0]->getTupleDesc().size() == 2);
00048     inputTuple.compute(inAccessors[0]->getTupleDesc());
00049 
00050     // setup tuple used to store key values read from clusters
00051     bitmapTuple.computeAndAllocate(pOutAccessor->getTupleDesc());
00052 
00053     // setup output tuple
00054     assert(treeDescriptor.tupleDescriptor == pOutAccessor->getTupleDesc());
00055     bitmapTupleDesc = treeDescriptor.tupleDescriptor;
00056 
00057     attrAccessors.resize(bitmapTupleDesc.size());
00058     for (int i = 0; i < bitmapTupleDesc.size(); ++i) {
00059         attrAccessors[i].compute(bitmapTupleDesc[i]);
00060     }
00061 
00062     nIdxKeys = treeDescriptor.keyProjection.size() - 1;
00063 
00064     // determine min and max size of bitmap entries
00065 
00066     LbmEntry::getSizeBounds(
00067         bitmapTupleDesc,
00068         treeDescriptor.segmentAccessor.pSegment->getUsablePageSize(),
00069         minBitmapSize,
00070         maxBitmapSize);
00071 
00072     ridRuns.resize(1);
00073 }

void LbmGeneratorExecStream::open ( bool restart ) [virtual]

Opens this stream, acquiring any resources needed in order to be able to fetch data.

A precondition is that input streams must already be opened. A stream can be closed and reopened.

Parameters:

restart

if true, the stream must be already open, and should reset itself to start from the beginning of its result set

Reimplemented from BTreeExecStream.

Definition at line 75 of file LbmGeneratorExecStream.cpp.

References batchRead, CircularBuffer< T >::clear(), doneReading, flushIdx, ConfluenceExecStream::inAccessors, insertRowCountParamId, LBM_NOFLUSH_PENDING, nBitmapBuffers, nBitmapEntries, LcsRowScanBaseExecStream::nClusters, nScratchPagesAllocated, opaqueToInt(), LcsRowScanBaseExecStream::open(), BTreeExecStream::open(), parameterIds, LcsRowScanBaseExecStream::pClusters, ExecStream::pDynamicParamManager, producePending, revertToSingletons, LcsRowScanBaseExecStream::ridRuns, rowCount, and skipRead.

00076 {
00077     BTreeExecStream::open(restart);
00078     LcsRowScanBaseExecStream::open(restart);
00079     nBitmapEntries = 0;
00080     flushIdx = 0;
00081     nBitmapBuffers = 0;
00082     nScratchPagesAllocated = 0;
00083     producePending = LBM_NOFLUSH_PENDING;
00084     skipRead = false;
00085     rowCount = 0;
00086     batchRead = false;
00087     doneReading = false;
00088     revertToSingletons = false;
00089     ridRuns.clear();
00090     if (!restart) {
00091         pDynamicParamManager->createParam(
00092             insertRowCountParamId, inAccessors[0]->getTupleDesc()[0]);
00093 
00094         // set the rootPageIds of the clusters, if there are dynamic parameters
00095         // corresponding to them
00096         if (parameterIds.size() > 0) {
00097             for (uint i = 0; i < nClusters; i++) {
00098                 if (opaqueToInt(parameterIds[i]) > 0) {
00099                     pClusters[i]->setRootPageId(
00100                         *reinterpret_cast<PageId const *>(
00101                             pDynamicParamManager->getParam(parameterIds[i]).
00102                                 getDatum().pData));
00103                 }
00104             }
00105         }
00106     }
00107 }

ExecStreamResult LbmGeneratorExecStream::execute ( ExecStreamQuantum const & quantum ) [virtual]

Executes this stream.

Parameters:

quantum

governs the maximum amount of execution to perform

Returns:: code indicating reason execution ceased

Implements ExecStream.

Definition at line 145 of file LbmGeneratorExecStream.cpp.

References bitmapTable, createIndex, currRid, doneReading, EXECBUF_EOS, EXECRC_BUF_OVERFLOW, EXECRC_BUF_UNDERFLOW, EXECRC_EOS, EXECRC_QUANTUM_EXPIRED, flushStart, flushTable(), generateMultiKeyBitmaps(), generateSingleKeyBitmaps(), ConfluenceExecStream::inAccessors, initBitmapTable(), inputTuple, insertRowCountParamId, LBM_ENTRYFLUSH_PENDING, LBM_NOFLUSH_PENDING, LBM_TABLEFLUSH_PENDING, MAXU, LcsRowScanBaseExecStream::nClusters, nIdxKeys, LcsRidRun::nRids, numRowsToLoad, outputTuple, LcsRowScanBaseExecStream::pClusters, ExecStream::pDynamicParamManager, SingleOutputExecStream::pOutAccessor, producePending, CircularBuffer< T >::push_back(), LcsRowScanBaseExecStream::ridRuns, rowCount, LcsRidRun::startRid, startRid, and LcsRowScanBaseExecStream::syncColumns().

00147 {
00148     // read the start rid and num of rows to load
00149 
00150     if (inAccessors[0]->getState() == EXECBUF_EOS) {
00151         if (doneReading) {
00152             pOutAccessor->markEOS();
00153             return EXECRC_EOS;
00154         }
00155 
00156     } else {
00157         if (!inAccessors[0]->demandData()) {
00158             return EXECRC_BUF_UNDERFLOW;
00159         }
00160 
00161         inAccessors[0]->unmarshalTuple(inputTuple);
00162 
00163         // in the case of create index, the number of rows affected
00164         // is returned as 0, since the statement is a DDL
00165         LcsRidRun ridRun;
00166         if (createIndex) {
00167             numRowsToLoad = 0;
00168             ridRun.nRids = RecordNum(MAXU);
00169             startRid = LcsRid(0);
00170         } else {
00171             numRowsToLoad =
00172                 *reinterpret_cast<RecordNum const *> (inputTuple[0].pData);
00173             ridRun.nRids = numRowsToLoad;
00174             startRid =
00175                 *reinterpret_cast<LcsRid const *> (inputTuple[1].pData);
00176         }
00177         currRid = startRid;
00178 
00179         // Setup the prefetch rid run to a single run spanning the range
00180         // of rows to be inserted into the index
00181         ridRun.startRid = startRid;
00182         ridRuns.push_back(ridRun);
00183 
00184         // set number of rows to load in a dynamic parameter that
00185         // splicer will later read
00186         pDynamicParamManager->writeParam(insertRowCountParamId, inputTuple[0]);
00187 
00188         inAccessors[0]->consumeTuple();
00189 
00190         // special case where there are no rows -- don't bother reading
00191         // from the table because we may end up reading deleted rows
00192         if (!createIndex && numRowsToLoad == 0) {
00193             doneReading = true;
00194             return EXECRC_BUF_UNDERFLOW;
00195         }
00196 
00197         // position to the starting rid
00198         for (uint iClu = 0; iClu < nClusters; iClu++) {
00199             SharedLcsClusterReader &pScan = pClusters[iClu];
00200             if (!pScan->position(startRid)) {
00201                 // empty table
00202                 doneReading = true;
00203                 return EXECRC_BUF_UNDERFLOW;
00204             }
00205             syncColumns(pScan);
00206         }
00207 
00208         // initialize bitmap table to a single entry, assuming we're
00209         // starting with singleton bitmaps
00210         bool rc = initBitmapTable(1);
00211         assert(rc);
00212     }
00213 
00214     // take care of any pending flushes first
00215 
00216     switch (producePending) {
00217     case LBM_ENTRYFLUSH_PENDING:
00218         // outputTuple already contains the pending tuple to be flushed
00219         if (!pOutAccessor->produceTuple(outputTuple)) {
00220             return EXECRC_BUF_OVERFLOW;
00221         }
00222         bitmapTable[flushStart].inuse = false;
00223         break;
00224     case LBM_TABLEFLUSH_PENDING:
00225         if (!flushTable(flushStart)) {
00226             return EXECRC_BUF_OVERFLOW;
00227         }
00228         break;
00229     default:
00230         break;
00231     }
00232     producePending = LBM_NOFLUSH_PENDING;
00233 
00234     ExecStreamResult rc;
00235 
00236     if (nIdxKeys == 1) {
00237         rc = generateSingleKeyBitmaps(quantum);
00238     } else {
00239         rc = generateMultiKeyBitmaps(quantum);
00240     }
00241 
00242     switch (rc) {
00243     case EXECRC_BUF_OVERFLOW:
00244     case EXECRC_QUANTUM_EXPIRED:
00245         return rc;
00246     case EXECRC_EOS:
00247         // no more rows to process
00248         if (!createIndex) {
00249             // It's possible for the row count to be larger if we're building
00250             // an index on a replaced column.
00251             assert(rowCount >= numRowsToLoad);
00252         }
00253         doneReading = true;
00254         return EXECRC_BUF_UNDERFLOW;
00255     default:
00256         permAssert(false);
00257     }
00258 }

void LbmGeneratorExecStream::getResourceRequirements	(	ExecStreamResourceQuantity &	minQuantity,
		ExecStreamResourceQuantity &	optQuantity,
		ExecStreamResourceSettingType &	optType
	)			`[virtual]`

Determines resource requirements for this stream.

Default implementation declares zero resource requirements.

Parameters:

	minQuantity	receives the minimum resource quantity needed by this stream in order to execute
	optQuantity	receives the resource quantity needed by this stream in order to execute optimally
	optType	Receives the value indicating the accuracy of the optQuantity parameter. This parameter is optional and defaults to EXEC_RESOURCE_ACCURATE if omitted. If the optimum setting is an estimate or no value can be specified (e.g., due to lack of statistics), then this parameter needs to be used to indicate a non-accurate optimum resource setting.

Reimplemented from ExecStream.

Definition at line 109 of file LbmGeneratorExecStream.cpp.

References EXEC_RESOURCE_ACCURATE, EXEC_RESOURCE_ESTIMATE, LcsRowScanBaseExecStream::getResourceRequirements(), ExecStream::getResourceRequirements(), ExecStreamResourceQuantity::nCachePages, nIdxKeys, and numMiscScratchPages.

00113 {
00114     BTreeExecStream::getResourceRequirements(minQuantity, optQuantity);
00115     LcsRowScanBaseExecStream::getResourceRequirements(minQuantity, optQuantity);
00116     numMiscScratchPages = minQuantity.nCachePages;
00117 
00118     // need a minimum of one scratch pages for constructing LbmEntry's
00119     minQuantity.nCachePages += 1;
00120 
00121     // If this is a multi-key index, then we're only creating singleton
00122     // LbmEntry's and therefore, don't need multiple scratch pages.
00123     // Otherwise, we ideally want to set the number of scratch pages
00124     // based on the max number of distinct values in compressed batches.
00125     // Since we don't have that information, we'll use an "estimate" of
00126     // 10 pages.
00127     if (nIdxKeys > 1) {
00128         optQuantity.nCachePages += 1;
00129         optType = EXEC_RESOURCE_ACCURATE;
00130     } else {
00131         optQuantity.nCachePages += 11;
00132         optType = EXEC_RESOURCE_ESTIMATE;
00133     }
00134 }

void LbmGeneratorExecStream::setResourceAllocation ( ExecStreamResourceQuantity & quantity ) [virtual]

Sets current resource allocation for this stream.

If called while the stream is open, this indicates a request for the stream to dynamically adjust its memory usage. If the stream is incapable of honoring the request, it should update quantity with the actual amounts still in use.

Parameters:

quantity

allocated resource quantity

Reimplemented from ExecStream.

Definition at line 136 of file LbmGeneratorExecStream.cpp.

References maxNumScratchPages, ExecStreamResourceQuantity::nCachePages, numMiscScratchPages, and ExecStream::setResourceAllocation().

00138 {
00139     BTreeExecStream::setResourceAllocation(quantity);
00140     LcsRowScanBaseExecStream::setResourceAllocation(quantity);
00141 
00142     maxNumScratchPages = quantity.nCachePages - numMiscScratchPages;
00143 }

void LbmGeneratorExecStream::closeImpl ( ) [virtual]

Implements ClosableObject.

ExecStream implementations may override this to release any resources acquired while open.

Reimplemented from BTreeExecStream.

Definition at line 351 of file LbmGeneratorExecStream.cpp.

References bitmapTable, LcsRowScanBaseExecStream::closeImpl(), BTreeExecStream::closeImpl(), keyCodes, NULL_PAGE_ID, SegmentAccessor::pSegment, BTreeExecStream::scratchAccessor, and scratchPages.

00352 {
00353     BTreeExecStream::closeImpl();
00354     LcsRowScanBaseExecStream::closeImpl();
00355     keyCodes.clear();
00356     bitmapTable.clear();
00357     scratchPages.clear();
00358 
00359     if (scratchAccessor.pSegment) {
00360         scratchAccessor.pSegment->deallocatePageRange(
00361             NULL_PAGE_ID, NULL_PAGE_ID);
00362     }
00363 }

SharedBTreeReader BTreeExecStream::newReader ( ) [protected, virtual, inherited]

Reimplemented in BTreePrefetchSearchExecStream.

Definition at line 67 of file BTreeExecStream.cpp.

References BTreeExecStream::pBTreeAccessBase, BTreeExecStream::pBTreeReader, and BTreeExecStream::treeDescriptor.

Referenced by BTreeReadExecStream::open().

00068 {
00069     SharedBTreeReader pReader = SharedBTreeReader(
00070         new BTreeReader(treeDescriptor));
00071     pBTreeAccessBase = pBTreeReader = pReader;
00072     return pReader;
00073 }

SharedBTreeWriter BTreeExecStream::newWriter ( bool monotonic = false ) [protected, inherited]

Definition at line 75 of file BTreeExecStream.cpp.

References BTreeExecStream::pBTreeAccessBase, BTreeExecStream::pBTreeReader, BTreeExecStream::scratchAccessor, and BTreeExecStream::treeDescriptor.

Referenced by FtrsTableWriter::createIndexWriter(), and BTreeInsertExecStream::open().

00076 {
00077     SharedBTreeWriter pWriter = SharedBTreeWriter(
00078         new BTreeWriter(treeDescriptor,scratchAccessor,monotonic));
00079     pBTreeAccessBase = pBTreeReader = pWriter;
00080     return pWriter;
00081 }

SharedBTreeWriter BTreeExecStream::newWriter ( BTreeExecStreamParams const & params ) [static, inherited]

Definition at line 83 of file BTreeExecStream.cpp.

References BTreeExecStream::copyParamsToDescriptor(), ExecStreamParams::pCacheAccessor, ExecStreamParams::scratchAccessor, and BTreeExecStream::treeDescriptor.

00085 {
00086     BTreeDescriptor treeDescriptor;
00087     copyParamsToDescriptor(treeDescriptor,params,params.pCacheAccessor);
00088     return SharedBTreeWriter(
00089         new BTreeWriter(
00090             treeDescriptor,params.scratchAccessor));
00091 }

void BTreeExecStream::endSearch ( ) [protected, virtual, inherited]

Forgets the current reader or writer's search, releasing any page locks.

Definition at line 107 of file BTreeExecStream.cpp.

References BTreeExecStream::pBTreeReader.

Referenced by BTreeExecStream::closeImpl(), and BTreeExecStream::open().

00108 {
00109     if (pBTreeReader && pBTreeReader->isSingular() == false) {
00110         pBTreeReader->endSearch();
00111     }
00112 }

void BTreeExecStream::prepare ( BTreeExecStreamParams const & params ) [virtual, inherited]

Definition at line 30 of file BTreeExecStream.cpp.

References BTreeExecStream::copyParamsToDescriptor(), ExecStreamParams::pCacheAccessor, SingleOutputExecStream::prepare(), BTreeParams::pRootMap, BTreeExecStream::pRootMap, BTreeParams::rootPageIdParamId, BTreeExecStream::rootPageIdParamId, ExecStreamParams::scratchAccessor, BTreeExecStream::scratchAccessor, and BTreeExecStream::treeDescriptor.

Referenced by LcsClusterAppendExecStream::prepare(), prepare(), BTreeReadExecStream::prepare(), and BTreeInsertExecStream::prepare().

00031 {
00032     SingleOutputExecStream::prepare(params);
00033 
00034     copyParamsToDescriptor(treeDescriptor,params,params.pCacheAccessor);
00035     scratchAccessor = params.scratchAccessor;
00036     pRootMap = params.pRootMap;
00037     rootPageIdParamId = params.rootPageIdParamId;
00038 }

void SingleOutputExecStream::prepare ( SingleOutputExecStreamParams const & params ) [virtual, inherited]

Definition at line 48 of file SingleOutputExecStream.cpp.

References SingleOutputExecStream::getOutputBufProvision(), SingleOutputExecStreamParams::outputTupleDesc, SingleOutputExecStreamParams::outputTupleFormat, SingleOutputExecStream::pOutAccessor, and ExecStream::prepare().

Referenced by BTreeExecStream::prepare(), FlatFileExecStreamImpl::prepare(), ValuesExecStream::prepare(), MockResourceExecStream::prepare(), MockProducerExecStream::prepare(), ConfluenceExecStream::prepare(), and ConduitExecStream::prepare().

00049 {
00050     ExecStream::prepare(params);
00051     assert(pOutAccessor);
00052     assert(pOutAccessor->getProvision() == getOutputBufProvision());
00053     if (pOutAccessor->getTupleDesc().empty()) {
00054         assert(!params.outputTupleDesc.empty());
00055         pOutAccessor->setTupleShape(
00056             params.outputTupleDesc,
00057             params.outputTupleFormat);
00058     }
00059 }

void ExecStream::prepare ( ExecStreamParams const & params ) [virtual, inherited]

Prepares this stream for execution.

A precondition is that input streams must already be defined and prepared. As an effect of this call, the tuple shape should be defined for all output buffers and remain unchanged for the lifetime of the stream. This method is only ever called once, before the first open. Although this method is virtual, derived classes may choose to define an overloaded version instead with a specialized covariant parameter class.

Parameters:

params

instance of stream parameterization class which should be used to prepare this stream

Definition at line 84 of file ExecStream.cpp.

References ExecStreamGraph::getDynamicParamManager(), SegmentAccessor::pCacheAccessor, ExecStreamParams::pCacheAccessor, ExecStream::pDynamicParamManager, ExecStream::pGraph, ExecStream::pQuotaAccessor, ExecStream::pScratchQuotaAccessor, and ExecStreamParams::scratchAccessor.

Referenced by JavaTransformExecStream::prepare(), SingleOutputExecStream::prepare(), and SingleInputExecStream::prepare().

00085 {
00086     if (pGraph) {
00087         pDynamicParamManager = pGraph->getDynamicParamManager();
00088     }
00089     pQuotaAccessor = params.pCacheAccessor;
00090     pScratchQuotaAccessor = params.scratchAccessor.pCacheAccessor;
00091 }

void BTreeExecStream::copyParamsToDescriptor	(	BTreeDescriptor &	,
		BTreeParams const &	,
		SharedCacheAccessor const &
	)			`[static, inherited]`

Definition at line 93 of file BTreeExecStream.cpp.

References BTreeParams::keyProj, BTreeDescriptor::keyProjection, BTreeParams::pageOwnerId, BTreeDescriptor::pageOwnerId, SegmentAccessor::pCacheAccessor, BTreeParams::pSegment, SegmentAccessor::pSegment, BTreeParams::rootPageId, BTreeDescriptor::rootPageId, BTreeDescriptor::segmentAccessor, BTreeParams::segmentId, BTreeDescriptor::segmentId, BTreeExecStream::treeDescriptor, BTreeParams::tupleDesc, and BTreeDescriptor::tupleDescriptor.

Referenced by BTreeExecStream::newWriter(), LbmSplicerExecStream::prepare(), and BTreeExecStream::prepare().

00097 {
00098     treeDescriptor.segmentAccessor.pSegment = params.pSegment;
00099     treeDescriptor.segmentAccessor.pCacheAccessor = pCacheAccessor;
00100     treeDescriptor.tupleDescriptor = params.tupleDesc;
00101     treeDescriptor.keyProjection = params.keyProj;
00102     treeDescriptor.rootPageId = params.rootPageId;
00103     treeDescriptor.segmentId = params.segmentId;
00104     treeDescriptor.pageOwnerId = params.pageOwnerId;
00105 }

void SingleOutputExecStream::setInputBufAccessors ( std::vector< SharedExecStreamBufAccessor > const & inAccessors ) [virtual, inherited]

Initializes the buffer accessors for inputs to this stream.

This method is only ever called once, before prepare.

Parameters:

inAccessors

buffer accessors ordered by input stream

Implements ExecStream.

Reimplemented in ConduitExecStream, and ConfluenceExecStream.

Definition at line 35 of file SingleOutputExecStream.cpp.

00037 {
00038     assert(inAccessors.size() == 0);
00039 }

void SingleOutputExecStream::setOutputBufAccessors ( std::vector< SharedExecStreamBufAccessor > const & outAccessors ) [virtual, inherited]

Initializes the buffer accessors for outputs from this stream.

This method is only ever called once, before prepare.

Parameters:

outAccessors

buffer accessors ordered by output stream

Implements ExecStream.

Reimplemented in ConduitExecStream.

Definition at line 41 of file SingleOutputExecStream.cpp.

References SingleOutputExecStream::pOutAccessor.

Referenced by ConduitExecStream::setOutputBufAccessors().

00043 {
00044     assert(outAccessors.size() == 1);
00045     pOutAccessor = outAccessors[0];
00046 }

ExecStreamBufProvision SingleOutputExecStream::getOutputBufProvision ( ) const [virtual, inherited]

Queries the BufferProvision which this stream is capable of when producing tuples.

Returns:: supported model; default is BUFPROV_NONE

Reimplemented from ExecStream.

Reimplemented in BarrierExecStream, DoubleBufferExecStream, MergeExecStream, MockResourceExecStream, ScratchBufferExecStream, SegBufferExecStream, SegBufferReaderExecStream, ValuesExecStream, FtrsTableWriterExecStream, and LcsClusterAppendExecStream.

Definition at line 69 of file SingleOutputExecStream.cpp.

References BUFPROV_CONSUMER.

Referenced by SingleOutputExecStream::prepare().

00070 {
00071     return BUFPROV_CONSUMER;
00072 }

bool ExecStream::canEarlyClose ( ) [virtual, inherited]

Returns:: true if the stream can be closed early

Reimplemented in SegBufferWriterExecStream.

Definition at line 49 of file ExecStream.cpp.

00050 {
00051     return true;
00052 }

ExecStreamGraph & ExecStream::getGraph ( ) const [inline, inherited]

Returns:: reference to containing graph

Definition at line 293 of file ExecStream.h.

References ExecStream::pGraph.

Referenced by ExternalSortExecStreamImpl::execute(), JavaSinkExecStream::execute(), SegBufferWriterExecStream::execute(), SegBufferExecStream::execute(), BarrierExecStream::execute(), Java_net_sf_farrago_fennel_FennelStorage_tupleStreamFetch(), Java_net_sf_farrago_fennel_FennelStorage_tupleStreamTransformFetch(), JavaSinkExecStream::stuffByteBuffer(), and ExecStreamScheduler::traceStreamBuffers().

00294 {
00295     assert(pGraph);
00296     return *pGraph;
00297 }

ExecStreamId ExecStream::getStreamId ( ) const [inline, inherited]

Returns:: the identifier for this stream within containing graph

Definition at line 288 of file ExecStream.h.

References ExecStream::id.

Referenced by ExternalSortExecStreamImpl::execute(), SegBufferWriterExecStream::execute(), SegBufferExecStream::execute(), CorrelationJoinExecStream::execute(), BarrierExecStream::execute(), Java_net_sf_farrago_fennel_FennelStorage_tupleStreamTransformFetch(), JavaTransformExecStream::open(), SingleInputExecStream::open(), ConfluenceExecStream::open(), CartesianJoinExecStream::prepare(), ParallelExecStreamScheduler::readStream(), DfsTreeExecStreamScheduler::readStream(), LbmMinusExecStream::restartSubtrahends(), ExecStreamScheduler::tracePostExecution(), ExecStreamScheduler::tracePreExecution(), ExecStreamScheduler::traceStreamBuffers(), and ParallelExecStreamScheduler::tryExecuteTask().

00289 {
00290     return id;
00291 }

void ExecStream::getResourceRequirements	(	ExecStreamResourceQuantity &	minQuantity,
		ExecStreamResourceQuantity &	optQuantity
	)			`[virtual, inherited]`

Reimplemented in DoubleBufferExecStream, ScratchBufferExecStream, SegBufferExecStream, SegBufferReaderExecStream, SegBufferWriterExecStream, FlatFileExecStreamImpl, BTreeInsertExecStream, BTreeReadExecStream, FtrsTableWriterExecStream, LbmChopperExecStream, LbmSplicerExecStream, LcsClusterAppendExecStream, LcsClusterReplaceExecStream, LcsRowScanBaseExecStream, and LcsRowScanExecStream.

Definition at line 102 of file ExecStream.cpp.

References ExecStreamResourceQuantity::nCachePages, and ExecStreamResourceQuantity::nThreads.

00105 {
00106     minQuantity.nThreads = 0;
00107     minQuantity.nCachePages = 0;
00108     optQuantity = minQuantity;
00109 }

void ExecStream::setName ( std::string const & ) [virtual, inherited]

Sets unique name of this stream.

Definition at line 157 of file ExecStream.cpp.

References ExecStream::name.

00158 {
00159     name = nameInit;
00160 }

std::string const & ExecStream::getName ( ) const [virtual, inherited]

Returns:: the name of this stream, as known by the optimizer

Definition at line 162 of file ExecStream.cpp.

References ExecStream::name.

Referenced by DfsTreeExecStreamScheduler::findNextConsumer(), ParallelExecStreamScheduler::readStream(), DfsTreeExecStreamScheduler::readStream(), ExecStreamScheduler::tracePostExecution(), and ExecStreamScheduler::tracePreExecution().

00163 {
00164     return name;
00165 }

bool ExecStream::mayBlock ( ) const [virtual, inherited]

Queries whether this stream's implementation may block when execute() is called.

For accurate scheduling, non-blocking implementations are preferred; the scheduler must be aware of the potential for blocking so that it can allocate extra threads accordingly.

Returns:: whether stream may block; default is false

Definition at line 167 of file ExecStream.cpp.

00168 {
00169     return false;
00170 }

void ExecStream::checkAbort ( ) const [virtual, inherited]

Checks whether there is an abort request for this stream's scheduler.

Normally, streams don't need to check this, since the scheduler services abort requests in between quanta. However, streams which enter long-running loops need to check for themselves. If an abort is scheduled, this method will throw an AbortExcn automatically.

Definition at line 72 of file ExecStream.cpp.

References ExecStreamScheduler::checkAbort(), ExecStreamGraph::getScheduler(), and ExecStream::pGraph.

Referenced by LhxJoinExecStream::execute(), LhxAggExecStream::execute(), ExternalSortRunAccessor::fetch(), and ExternalSortMerger::fetch().

00073 {
00074     if (!pGraph) {
00075         return;
00076     }
00077     ExecStreamScheduler *pScheduler = pGraph->getScheduler();
00078     if (!pScheduler) {
00079         return;
00080     }
00081     pScheduler->checkAbort();
00082 }

ExecStreamBufProvision ExecStream::getOutputBufConversion ( ) const [virtual, inherited]

Queries the BufferProvision to which this stream needs its output to be converted, if any.

Returns:: required conversion; default is BUFPROV_NONE

Reimplemented in JavaTransformExecStream.

Definition at line 177 of file ExecStream.cpp.

References BUFPROV_NONE.

00178 {
00179     return BUFPROV_NONE;
00180 }

ExecStreamBufProvision ExecStream::getInputBufProvision ( ) const [virtual, inherited]

Queries the BufferProvision which this stream requires of its inputs when consuming their tuples.

Returns:: required model; default is BUFPROV_NONE

Reimplemented in ConfluenceExecStream, DoubleBufferExecStream, ScratchBufferExecStream, SegBufferExecStream, SegBufferWriterExecStream, SingleInputExecStream, and JavaTransformExecStream.

Definition at line 182 of file ExecStream.cpp.

References BUFPROV_NONE.

00183 {
00184     return BUFPROV_NONE;
00185 }

bool ClosableObject::isClosed ( ) const [inline, inherited]

Returns:: whether the object has been closed

Definition at line 58 of file ClosableObject.h.

00059     {
00060         return !needsClose;
00061     }

void ClosableObject::close ( ) [inherited]

Closes this object, releasing any unallocated resources.

Reimplemented in CollectExecStream, CorrelationJoinExecStream, LcsClusterAppendExecStream, and LcsClusterReplaceExecStream.

Definition at line 39 of file ClosableObject.cpp.

References ClosableObject::closeImpl(), and ClosableObject::needsClose.

Referenced by CacheImpl< PageT, VictimPolicyT >::allocatePages(), LcsRowScanBaseExecStream::closeImpl(), ExecStreamGraphImpl::closeImpl(), FlatFileBuffer::open(), ClosableObjectDestructor::operator()(), and Segment::~Segment().

00040 {
00041     if (!needsClose) {
00042         return;
00043     }
00044     needsClose = false;
00045     closeImpl();
00046 }

void TraceSource::initTraceSource	(	SharedTraceTarget	pTraceTarget,
		std::string	name
	)			`[virtual, inherited]`

For use when initialization has to be deferred until after construction.

Parameters:

	pTraceTarget	the TraceTarget to which messages will be sent
	name	the name of this source

Definition at line 46 of file TraceSource.cpp.

References TraceSource::isTracing(), TraceSource::minimumLevel, TraceSource::name, TraceSource::pTraceTarget, and TRACE_OFF.

Referenced by TestBase::beforeTestCase(), TestBase::TestBase(), and TraceSource::TraceSource().

00049 {
00050     assert(!pTraceTarget.get());
00051 
00052     pTraceTarget = pTraceTargetInit;
00053     name = nameInit;
00054     if (isTracing()) {
00055         minimumLevel = pTraceTarget->getSourceTraceLevel(name);
00056     } else {
00057         minimumLevel = TRACE_OFF;
00058     }
00059 }

void TraceSource::trace	(	TraceLevel	level,
		std::string	message
	)			const `[inherited]`

Records a trace message.

Normally only called via FENNEL_TRACE.

Parameters:

	level	severity level of event being trace
	message	the text of the message

Definition at line 61 of file TraceSource.cpp.

References TraceSource::getTraceTarget(), TraceSource::isTracing(), TraceSource::name, and TraceTarget::notifyTrace().

Referenced by Calculator::exec(), and ExecStreamScheduler::traceStreamBufferContents().

00062 {
00063     if (isTracing()) {
00064         getTraceTarget().notifyTrace(name,level,message);
00065     }
00066 }

bool TraceSource::isTracing ( ) const [inline, inherited]

Returns:: true iff tracing is enabled for this source

Definition at line 88 of file TraceSource.h.

Referenced by TraceSource::initTraceSource(), CalcExecStream::prepare(), and TraceSource::trace().

00089     {
00090         return pTraceTarget.get() ? true : false;
00091     }

bool TraceSource::isTracingLevel ( TraceLevel level ) const [inline, inherited]

Determines whether a particular level is being traced.

Parameters:

level

trace level to test

Returns:: true iff tracing is enabled for the given level

Definition at line 100 of file TraceSource.h.

Referenced by ExecStreamScheduler::addGraph(), SimpleExecStreamGovernor::assignCachePages(), SimpleExecStreamGovernor::distributeCachePages(), Calculator::exec(), ExecStreamScheduler::ExecStreamScheduler(), LcsClusterNodeWriter::getLastClusterPageForWrite(), LcsClusterNodeWriter::moveFromTempToIndex(), JavaSinkExecStream::stuffByteBuffer(), and ExecStreamScheduler::traceStreamBuffers().

00101     {
00102         return level >= minimumLevel;
00103     }

TraceTarget& TraceSource::getTraceTarget ( ) const [inline, inherited]

Returns:: the TraceTarget for this source

Definition at line 108 of file TraceSource.h.

Referenced by TraceSource::trace().

00109     {
00110         assert(isTracing());
00111         return *(pTraceTarget.get());
00112     }

SharedTraceTarget TraceSource::getSharedTraceTarget ( ) const [inline, inherited]

Returns:: the SharedTraceTarget for this source

Definition at line 117 of file TraceSource.h.

Referenced by Database::init(), LcsClusterAppendExecStream::initLoad(), and CalcExecStream::prepare().

00118     {
00119         return pTraceTarget;
00120     }

std::string TraceSource::getTraceSourceName ( ) const [inline, inherited]

Gets the name of this source.

Useful to construct nested names for subcomponents that are also TraceSources.

Returns:: the name

Definition at line 127 of file TraceSource.h.

Referenced by LcsClusterAppendExecStream::initLoad().

00128     {
00129         return name;
00130     }

void TraceSource::setTraceSourceName ( std::string const & n ) [inline, inherited]

Sets the name of this source.

Useful to construct dynamic names for fine-grained filtering.

Definition at line 136 of file TraceSource.h.

00137     {
00138         name = n;
00139     }

TraceLevel TraceSource::getMinimumTraceLevel ( ) const [inline, inherited]

Definition at line 141 of file TraceSource.h.

00142     {
00143         return minimumLevel;
00144     }

void TraceSource::disableTracing ( ) [inherited]

Definition at line 68 of file TraceSource.cpp.

References TraceSource::minimumLevel, TraceSource::pTraceTarget, and TRACE_OFF.

Referenced by TestBase::afterTestCase().

00069 {
00070     pTraceTarget.reset();
00071     minimumLevel = TRACE_OFF;
00072 }

void ErrorSource::initErrorSource	(	SharedErrorTarget	pErrorTarget,
		const std::string &	name
	)			`[virtual, inherited]`

For use when initialization has to be deferred until after construction.

Parameters:

	pErrorTarget	the ErrorTarget to which errors will be posted
	name	the name of this source

Definition at line 47 of file ErrorSource.cpp.

References ErrorSource::name, and ErrorSource::pErrorTarget.

Referenced by ErrorSource::ErrorSource().

00050 {
00051     pErrorTarget = pErrorTargetInit;
00052     name = nameInit;
00053 }

void ErrorSource::postError	(	ErrorLevel	level,
		const std::string &	message,
		void *	address,
		long	capacity,
		int	index
	)			`[inherited]`

Posts an exception, such as a row exception.

See also:: ErrorTarget for a description of the parameters

Definition at line 55 of file ErrorSource.cpp.

References ErrorSource::getErrorTarget(), ErrorSource::hasTarget(), ErrorSource::name, and ErrorTarget::notifyError().

Referenced by FlatFileExecStreamImpl::logError(), ErrorSource::postError(), and LbmSplicerExecStream::postViolation().

00058 {
00059     if (hasTarget()) {
00060         getErrorTarget().notifyError(
00061             name, level, message, address, capacity, index);
00062     }
00063 }

void ErrorSource::postError	(	ErrorLevel	level,
		const std::string &	message,
		const TupleDescriptor &	errorDesc,
		const TupleData &	errorTuple,
		int	index
	)			`[inherited]`

Posts an exception, such as a row exception.

See also:: ErrorTarget for a description of the parameters

Definition at line 65 of file ErrorSource.cpp.

References TupleAccessor::compute(), ErrorSource::errorAccessor, FixedBuffer, TupleAccessor::getByteCount(), TupleAccessor::getMaxByteCount(), ErrorSource::hasTarget(), TupleAccessor::marshal(), ErrorSource::pErrorBuf, and ErrorSource::postError().

00068 {
00069     if (!hasTarget()) {
00070         return;
00071     }
00072 
00073     if (!pErrorBuf) {
00074         errorAccessor.compute(errorDesc);
00075         uint cbMax = errorAccessor.getMaxByteCount();
00076         pErrorBuf.reset(new FixedBuffer[cbMax]);
00077     }
00078 
00079     uint cbTuple = errorAccessor.getByteCount(errorTuple);
00080     errorAccessor.marshal(errorTuple, pErrorBuf.get());
00081     postError(level, message, pErrorBuf.get(), cbTuple, index);
00082 }

bool ErrorSource::hasTarget ( ) const [inline, inherited]

Returns:: true iff an error target has been set

Definition at line 112 of file ErrorSource.h.

Referenced by ErrorSource::postError().

00113     {
00114         return pErrorTarget.get() ? true : false;
00115     }

ErrorTarget& ErrorSource::getErrorTarget ( ) const [inline, inherited]

Returns:: the ErrorTarget for this source

Definition at line 120 of file ErrorSource.h.

Referenced by ErrorSource::postError().

00121     {
00122         assert(hasTarget());
00123         return *(pErrorTarget.get());
00124     }

SharedErrorTarget ErrorSource::getSharedErrorTarget ( ) const [inline, inherited]

Returns:: the SharedErrorTarget for this source

Definition at line 129 of file ErrorSource.h.

00130     {
00131         return pErrorTarget;
00132     }

std::string ErrorSource::getErrorSourceName ( ) const [inline, inherited]

Gets the name of this source.

Useful to construct nested names for subcomponents that are also ErrorSources.

Returns:: the name

Definition at line 139 of file ErrorSource.h.

00140     {
00141         return name;
00142     }

void ErrorSource::setErrorSourceName ( std::string const & n ) [inline, inherited]

Sets the name of this source.

Useful to construct dynamic names for fine-grained filtering.

Definition at line 148 of file ErrorSource.h.

00149     {
00150         name = n;
00151     }

void ErrorSource::disableTarget ( ) [inherited]

Definition at line 84 of file ErrorSource.cpp.

References ErrorSource::pErrorTarget.

00085 {
00086     pErrorTarget.reset();
00087 }

void LcsRowScanBaseExecStream::syncColumns ( SharedLcsClusterReader & pScan ) [protected, inherited]

Positions column readers based on new cluster reader position.

Parameters:

pScan

cluster reader

Definition at line 172 of file LcsRowScanBaseExecStream.cpp.

Referenced by advanceReader(), LcsRowScanExecStream::execute(), execute(), and generateMultiKeyBitmaps().

00173 {
00174     for (uint iCluCol = 0; iCluCol < pScan->nColsToRead; iCluCol++) {
00175         pScan->clusterCols[iCluCol].sync();
00176     }
00177 }

bool LcsRowScanBaseExecStream::readColVals	(	SharedLcsClusterReader &	pScan,
		TupleDataWithBuffer &	tupleData,
		uint	colStart
	)			`[protected, inherited]`

Reads column values based on current position of cluster reader.

Parameters:

	pScan	cluster reader
	tupleData	tupledata where data will be loaded
	colStart	starting column offset where first column will be loaded

Returns:: false if column filters failed; true otherwise

Definition at line 179 of file LcsRowScanBaseExecStream.cpp.

References LcsRowScanBaseExecStream::allSpecial, LcsRowScanBaseExecStream::attrAccessors, LcsRowScanBaseExecStream::projDescriptor, and LcsRowScanBaseExecStream::projMap.

Referenced by LcsRowScanExecStream::execute(), and generateMultiKeyBitmaps().

00183 {
00184     if (!allSpecial) {
00185         for (uint iCluCol = 0; iCluCol < pScan->nColsToRead; iCluCol++) {
00186             // Get value of each column and load it to the appropriate
00187             // tuple datum entry
00188             PBuffer curValue = pScan->clusterCols[iCluCol].getCurrentValue();
00189             uint idx = projMap[colStart + iCluCol];
00190 
00191             attrAccessors[idx].loadValue(tupleData[idx], curValue);
00192             if (pScan->clusterCols[iCluCol].getFilters().hasResidualFilters) {
00193                 if (!pScan->clusterCols[iCluCol].applyFilters(
00194                     projDescriptor,
00195                     tupleData))
00196                 {
00197                     return false;
00198                 }
00199             }
00200         }
00201     }
00202     return true;
00203 }

void LcsRowScanBaseExecStream::buildOutputProj	(	TupleProjection &	outputProj,
		LcsRowScanBaseExecStreamParams const &	params
	)			`[protected, virtual, inherited]`

Builds outputProj from params.

Parameters:

	outputProj	the projection to be built
	params	the LcsRowScanBaseExecStreamParams

Reimplemented in LcsRowScanExecStream.

Definition at line 205 of file LcsRowScanBaseExecStream.cpp.

References LcsRowScanBaseExecStreamParams::outputProj.

Referenced by LcsRowScanBaseExecStream::prepare().

00208 {
00209     /*
00210      * Copy the projection
00211      */
00212     for (uint i = 0;  i < params.outputProj.size(); i++) {
00213         outputProj.push_back(params.outputProj[i]);
00214     }
00215 }

void LcsRowScanBaseExecStream::prepare ( LcsRowScanBaseExecStreamParams const & params ) [virtual, inherited]

Definition at line 33 of file LcsRowScanBaseExecStream.cpp.

Referenced by LcsRowScanExecStream::prepare(), and prepare().

00035 {
00036     ConfluenceExecStream::prepare(params);
00037 
00038     // Copy cluster definition parameters and setup btree readers for each
00039     // cluster.  Also, setup the full output tuple based on the ordered
00040     // list of cluster descriptors.
00041 
00042     nClusters = params.lcsClusterScanDefs.size();
00043     pClusters.reset(new SharedLcsClusterReader[nClusters]);
00044 
00045     uint clusterStart = 0;
00046     uint projCount = 0;
00047     TupleDescriptor allClusterTupleDesc;
00048     TupleProjection newProj, outputProj;
00049 
00050     buildOutputProj(outputProj, params);
00051 
00052     newProj.resize(outputProj.size());
00053 
00054     // if we're projecting non-cluster columns, keep track of them separately;
00055     TupleDescriptor outputTupleDesc = pOutAccessor->getTupleDesc();
00056     for (uint i = 0; i < outputProj.size(); i++) {
00057         if (outputProj[i] == LCS_RID_COLUMN_ID) {
00058             newProj[i] = projCount++;
00059             allClusterTupleDesc.push_back(outputTupleDesc[i]);
00060             nonClusterCols.push_back(params.outputProj[i]);
00061         }
00062     }
00063 
00064     allSpecial = (nonClusterCols.size() == newProj.size());
00065 
00066     for (uint i = 0; i < nClusters; i++) {
00067         SharedLcsClusterReader &pClu = pClusters[i];
00068 
00069         BTreeExecStreamParams const &bTreeParams = params.lcsClusterScanDefs[i];
00070 
00071         BTreeDescriptor treeDescriptor;
00072         treeDescriptor.segmentAccessor.pSegment = bTreeParams.pSegment;
00073         treeDescriptor.segmentAccessor.pCacheAccessor =
00074             bTreeParams.pCacheAccessor;
00075         treeDescriptor.tupleDescriptor = bTreeParams.tupleDesc;
00076         treeDescriptor.keyProjection = bTreeParams.keyProj;
00077         treeDescriptor.rootPageId = bTreeParams.rootPageId;
00078         treeDescriptor.segmentId = bTreeParams.segmentId;
00079         treeDescriptor.pageOwnerId = bTreeParams.pageOwnerId;
00080 
00081         pClu =
00082             SharedLcsClusterReader(
00083                 new LcsClusterReader(treeDescriptor, &ridRuns));
00084 
00085         // setup the cluster and column readers to only read the columns
00086         // that are going to be projected
00087         uint clusterEnd = clusterStart +
00088             params.lcsClusterScanDefs[i].clusterTupleDesc.size() - 1;
00089 
00090         // create a vector of the columns that are projected from
00091         // this cluster and recompute the projection list
00092         // based on the individual cluster projections
00093         TupleProjection clusterProj;
00094         for (uint j = 0; j < newProj.size(); j++) {
00095             if (outputProj[j] >= clusterStart &&
00096                 outputProj[j] <= clusterEnd)
00097             {
00098                 clusterProj.push_back(outputProj[j] - clusterStart);
00099                 newProj[j] = projCount++;
00100             }
00101         }
00102         clusterStart = clusterEnd + 1;
00103 
00104         // need to select at least one column from cluster, except in the
00105         // cases where we're only selecting special columns or when there
00106         // are filter columns; in the former case, we'll just arbitrarily
00107         // read the first column, but not actually project it
00108         if (allSpecial) {
00109            clusterProj.push_back(0);
00110         }
00111         pClu->initColumnReaders(
00112             params.lcsClusterScanDefs[i].clusterTupleDesc.size(),
00113             clusterProj);
00114         if (!allSpecial) {
00115             for (uint j = 0; j < pClu->nColsToRead; j++) {
00116                 allClusterTupleDesc.push_back(
00117                     params.lcsClusterScanDefs[i].
00118                         clusterTupleDesc[clusterProj[j]]);
00119             }
00120         }
00121     }
00122 
00123     // setup projected tuple descriptor, by reshuffling allClusterTupleDesc
00124     // built above, into the correct projection order
00125 
00126     for (uint i = 0; i < newProj.size(); i++) {
00127         projDescriptor.push_back(allClusterTupleDesc[newProj[i]]);
00128     }
00129 
00130     // create a projection map to map cluster data read to the output
00131     // projection
00132     projMap.resize(newProj.size());
00133     for (uint i = 0; i < projMap.size(); i++) {
00134         for (uint j = 0; j < newProj.size(); j++) {
00135             if (newProj[j] == i) {
00136                 projMap[i] = j;
00137             }
00138         }
00139     }
00140 }

void ConfluenceExecStream::prepare ( ConfluenceExecStreamParams const & params ) [virtual, inherited]

Definition at line 37 of file ConfluenceExecStream.cpp.

References ConfluenceExecStream::getInputBufProvision(), ConfluenceExecStream::inAccessors, and SingleOutputExecStream::prepare().

Referenced by LcsRowScanBaseExecStream::prepare(), LbmUnionExecStream::prepare(), LbmChopperExecStream::prepare(), LbmBitOpExecStream::prepare(), LhxJoinExecStream::prepare(), MergeExecStream::prepare(), CorrelationJoinExecStream::prepare(), CartesianJoinExecStream::prepare(), and BarrierExecStream::prepare().

00038 {
00039     SingleOutputExecStream::prepare(params);
00040 
00041     for (uint i = 0; i < inAccessors.size(); ++i) {
00042         assert(inAccessors[i]->getProvision() == getInputBufProvision());
00043     }
00044 }

void LcsRowScanBaseExecStream::getResourceRequirements	(	ExecStreamResourceQuantity &	minQuantity,
		ExecStreamResourceQuantity &	optQuantity
	)			`[virtual, inherited]`

Reimplemented from ExecStream.

Reimplemented in LcsRowScanExecStream.

Definition at line 150 of file LcsRowScanBaseExecStream.cpp.

References ExecStream::getResourceRequirements(), ExecStreamResourceQuantity::nCachePages, and LcsRowScanBaseExecStream::nClusters.

Referenced by LcsRowScanExecStream::getResourceRequirements(), and getResourceRequirements().

00153 {
00154     ConfluenceExecStream::getResourceRequirements(minQuantity, optQuantity);
00155 
00156     // 2 pages per cluster (not taking into account pre-fetches yet)
00157     // - 1 for cluster page
00158     // - 1 for btree page
00159     minQuantity.nCachePages += (nClusters * 2);
00160 
00161     optQuantity = minQuantity;
00162 }

void ConfluenceExecStream::setInputBufAccessors ( std::vector< SharedExecStreamBufAccessor > const & inAccessors ) [virtual, inherited]

Initializes the buffer accessors for inputs to this stream.

This method is only ever called once, before prepare.

Parameters:

inAccessors

buffer accessors ordered by input stream

Reimplemented from SingleOutputExecStream.

Definition at line 31 of file ConfluenceExecStream.cpp.

References ConfluenceExecStream::inAccessors.

00033 {
00034     inAccessors = inAccessorsInit;
00035 }

ExecStreamBufProvision ConfluenceExecStream::getInputBufProvision ( ) const [virtual, inherited]

Queries the BufferProvision which this stream requires of its inputs when consuming their tuples.

Returns:: required model; default is BUFPROV_NONE

Reimplemented from ExecStream.

Definition at line 58 of file ConfluenceExecStream.cpp.

References BUFPROV_PRODUCER.

Referenced by ConfluenceExecStream::prepare().

00059 {
00060     return BUFPROV_PRODUCER;
00061 }

Member Data Documentation

uint LbmGeneratorExecStream::numMiscScratchPages [private]

Number of misc scratch pages needed; excludes pages for generating bitmap entries.

Definition at line 99 of file LbmGeneratorExecStream.h.

Referenced by getResourceRequirements(), and setResourceAllocation().

uint LbmGeneratorExecStream::maxNumScratchPages [private]

Number of scratch pages to allocate for constructing bitmaps as determined by the resource governor.

Definition at line 105 of file LbmGeneratorExecStream.h.

Referenced by initBitmapTable(), and setResourceAllocation().

DynamicParamId LbmGeneratorExecStream::insertRowCountParamId [private]

Dynamic parameter id used to pass along number of rows loaded.

Definition at line 110 of file LbmGeneratorExecStream.h.

Referenced by execute(), open(), and prepare().

bool LbmGeneratorExecStream::createIndex [private]

True if index is being loaded as part of a create index statement.

Definition at line 115 of file LbmGeneratorExecStream.h.

Referenced by execute(), and prepare().

std::vector<DynamicParamId> LbmGeneratorExecStream::parameterIds [private]

Ordered list of dynamic parameter ids for the clusters that provide source data for this stream.

Definition at line 121 of file LbmGeneratorExecStream.h.

Referenced by open(), and prepare().

uint LbmGeneratorExecStream::scratchPageSize [private]

Size of a scratch page.

Definition at line 126 of file LbmGeneratorExecStream.h.

Referenced by initBitmapTable(), and prepare().

uint LbmGeneratorExecStream::entrySize [private]

Size of a bitmap entry buffer.

Definition at line 131 of file LbmGeneratorExecStream.h.

Referenced by addRidToBitmap(), and initBitmapTable().

uint LbmGeneratorExecStream::maxBitmapSize [private]

Max size of a bitmap entry.

Definition at line 136 of file LbmGeneratorExecStream.h.

Referenced by initBitmapTable(), and prepare().

uint LbmGeneratorExecStream::minBitmapSize [private]

Min size of a bitmap entry.

Definition at line 141 of file LbmGeneratorExecStream.h.

Referenced by initBitmapTable(), and prepare().

ClusterPageLock LbmGeneratorExecStream::scratchLock [private]

Lock for scratch accessor.

Definition at line 146 of file LbmGeneratorExecStream.h.

Referenced by initBitmapTable(), and prepare().

TupleData LbmGeneratorExecStream::inputTuple [private]

Input tuple data.

Definition at line 151 of file LbmGeneratorExecStream.h.

Referenced by execute(), and prepare().

RecordNum LbmGeneratorExecStream::numRowsToLoad [private]

Number of rows to load.

Definition at line 156 of file LbmGeneratorExecStream.h.

Referenced by execute().

RecordNum LbmGeneratorExecStream::rowCount [private]

Running count of number of rows read.

Definition at line 161 of file LbmGeneratorExecStream.h.

Referenced by createSingletonBitmapEntry(), execute(), generateBitmaps(), and open().

LcsRid LbmGeneratorExecStream::startRid [private]

Starting rid;.

Definition at line 166 of file LbmGeneratorExecStream.h.

Referenced by execute().

LcsRid LbmGeneratorExecStream::currRid [private]

Current rid being loaded.

Definition at line 171 of file LbmGeneratorExecStream.h.

Referenced by createSingletonBitmapEntry(), execute(), generateBitmaps(), and generateMultiKeyBitmaps().

TupleDataWithBuffer LbmGeneratorExecStream::bitmapTuple [private]

Tuple data with buffer for the bitmap tuple.

Definition at line 176 of file LbmGeneratorExecStream.h.

Referenced by createSingletonBitmapEntry(), generateBitmaps(), generateMultiKeyBitmaps(), generateSingletons(), initRidAndBitmap(), and prepare().

TupleDescriptor LbmGeneratorExecStream::bitmapTupleDesc [private]

Tuple descriptor representing bitmap tuple.

Definition at line 181 of file LbmGeneratorExecStream.h.

Referenced by addRidToBitmap(), and prepare().

TupleData LbmGeneratorExecStream::outputTuple [private]

Pointer to generated tuple data.

Definition at line 186 of file LbmGeneratorExecStream.h.

Referenced by execute(), and flushEntry().

uint LbmGeneratorExecStream::nIdxKeys [private]

Number of keys in the bitmap index, excluding the starting rid.

Definition at line 191 of file LbmGeneratorExecStream.h.

Referenced by execute(), getResourceRequirements(), initRidAndBitmap(), and prepare().

bool LbmGeneratorExecStream::batchRead [private]

True if current batch has been read.

Definition at line 196 of file LbmGeneratorExecStream.h.

Referenced by generateBitmaps(), generateSingleKeyBitmaps(), generateSingletons(), and open().

uint LbmGeneratorExecStream::currBatch [private]

Current batch entry being processed.

Definition at line 201 of file LbmGeneratorExecStream.h.

Referenced by generateBitmaps().

std::vector<uint16_t> LbmGeneratorExecStream::keyCodes [private]

Keycodes read from a batch.

Definition at line 206 of file LbmGeneratorExecStream.h.

Referenced by closeImpl(), and generateBitmaps().

std::vector<LbmEntryInfo> LbmGeneratorExecStream::bitmapTable [private]

Table of bitmap entries under construction.

Definition at line 211 of file LbmGeneratorExecStream.h.

Referenced by addRidToBitmap(), closeImpl(), execute(), flushBuffer(), flushEntry(), flushTable(), and initBitmapTable().

uint LbmGeneratorExecStream::nBitmapEntries [private]

Number of entries in the bitmap table.

Definition at line 216 of file LbmGeneratorExecStream.h.

Referenced by addRidToBitmap(), flushBuffer(), flushTable(), initBitmapTable(), and open().

uint LbmGeneratorExecStream::flushIdx [private]

Index of buffer entry to flush.

Definition at line 221 of file LbmGeneratorExecStream.h.

Referenced by flushBuffer(), initBitmapTable(), and open().

uint LbmGeneratorExecStream::nBitmapBuffers [private]

Number of entries in the bitmap buffer table.

Definition at line 226 of file LbmGeneratorExecStream.h.

Referenced by open().

uint LbmGeneratorExecStream::nScratchPagesAllocated [private]

Number of scratch pages allocated.

Definition at line 231 of file LbmGeneratorExecStream.h.

Referenced by initBitmapTable(), and open().

std::vector<PBuffer> LbmGeneratorExecStream::scratchPages [private]

Vector of pointers to scratch pages allocated.

Definition at line 236 of file LbmGeneratorExecStream.h.

Referenced by closeImpl(), and initBitmapTable().

LbmPendingProduceType LbmGeneratorExecStream::producePending [private]

Produce of one or more output tuples pending: LBM_TABLEFLUSH_PENDING, LBM_ENTRYFLUSH_PENDING, LBM_FINALFLUSH_PENDING, LBM_NOFLUSH_PENDING.

Definition at line 242 of file LbmGeneratorExecStream.h.

Referenced by execute(), flushEntry(), flushTable(), and open().

uint LbmGeneratorExecStream::flushStart [private]

Index into bitmap table from which to start a pending table flush or the single entry currently being flushed.

Definition at line 248 of file LbmGeneratorExecStream.h.

Referenced by execute(), and flushEntry().

bool LbmGeneratorExecStream::skipRead [private]

If true, skip the initial read the next time generator is called since we haven't finished processing the current rowid.

Definition at line 254 of file LbmGeneratorExecStream.h.

Referenced by createSingletonBitmapEntry(), generateBitmaps(), generateMultiKeyBitmaps(), generateSingletons(), and open().

bool LbmGeneratorExecStream::doneReading [private]

If true, all rows from the column store table have been read.

Definition at line 259 of file LbmGeneratorExecStream.h.

Referenced by execute(), and open().

bool LbmGeneratorExecStream::revertToSingletons [private]

If true, even though the current batch is compressed and the index has a single key, generate singleton bitmaps for the batch due to lack of buffer space.

Definition at line 266 of file LbmGeneratorExecStream.h.

Referenced by generateBitmaps(), generateSingleKeyBitmaps(), and open().

BTreeDescriptor BTreeExecStream::treeDescriptor [protected, inherited]

Definition at line 113 of file BTreeExecStream.h.

Referenced by BTreeInsertExecStream::buildTree(), BTreeExecStream::closeImpl(), BTreeExecStream::copyParamsToDescriptor(), LcsClusterReplaceExecStream::getTupleForLoad(), LcsClusterAppendExecStream::initLoad(), LcsClusterReplaceExecStream::initTupleLoadParams(), BTreePrefetchSearchExecStream::innerSearchLoop(), BTreePrefetchSearchExecStream::newReader(), BTreeExecStream::newReader(), BTreeExecStream::newWriter(), LcsClusterReplaceExecStream::open(), BTreeSearchExecStream::open(), BTreePrefetchSearchExecStream::open(), BTreeInsertExecStream::open(), BTreeExecStream::open(), LcsClusterReplaceExecStream::prepare(), LcsClusterAppendExecStream::prepare(), LbmSearchExecStream::prepare(), prepare(), BTreeSearchExecStream::prepare(), BTreeInsertExecStream::prepare(), BTreeExecStream::prepare(), and BTreeInsertExecStream::truncateTree().

SegmentAccessor BTreeExecStream::scratchAccessor [protected, inherited]

Reimplemented in LcsClusterAppendExecStream.

Definition at line 114 of file BTreeExecStream.h.

Referenced by closeImpl(), BTreePrefetchSearchExecStream::closeImpl(), BTreeExecStream::newWriter(), prepare(), BTreePrefetchSearchExecStream::prepare(), and BTreeExecStream::prepare().

BTreeOwnerRootMap* BTreeExecStream::pRootMap [protected, inherited]

Definition at line 115 of file BTreeExecStream.h.

Referenced by BTreeExecStream::closeImpl(), BTreeExecStream::open(), and BTreeExecStream::prepare().

SharedBTreeAccessBase BTreeExecStream::pBTreeAccessBase [protected, inherited]

Definition at line 116 of file BTreeExecStream.h.

Referenced by BTreeInsertExecStream::closeImpl(), BTreeExecStream::closeImpl(), BTreePrefetchSearchExecStream::newReader(), BTreeExecStream::newReader(), BTreeExecStream::newWriter(), and BTreeExecStream::open().

SharedBTreeReader BTreeExecStream::pBTreeReader [protected, inherited]

Definition at line 117 of file BTreeExecStream.h.

Referenced by BTreeExecStream::endSearch(), BTreePrefetchSearchExecStream::newReader(), BTreeExecStream::newReader(), and BTreeExecStream::newWriter().

DynamicParamId BTreeExecStream::rootPageIdParamId [protected, inherited]

Definition at line 118 of file BTreeExecStream.h.

Referenced by BTreeSearchExecStream::open(), BTreeInsertExecStream::open(), BTreeInsertExecStream::prepare(), and BTreeExecStream::prepare().

SharedExecStreamBufAccessor SingleOutputExecStream::pOutAccessor [protected, inherited]

Definition at line 56 of file SingleOutputExecStream.h.

bool ExecStream::isOpen [protected, inherited]

Whether this stream is currently open.

Note that this is not quite the opposite of the inherited ClosableObject.needsClose, since a stream needs to be closed before destruction if it has been prepared but never opened.

Definition at line 61 of file ExecStream.h.

Referenced by ExecStream::closeImpl(), ExecStream::ExecStream(), and ExecStream::open().

ExecStreamGraph* ExecStream::pGraph [protected, inherited]

Dataflow graph containing this stream.

Note that we don't use a weak_ptr for this because it needs to be accessed frequently during execution, and the extra locking overhead would be frivolous.

Definition at line 68 of file ExecStream.h.

Referenced by ExecStream::checkAbort(), ExecStream::ExecStream(), CorrelationJoinExecStream::execute(), ExecStream::getGraph(), JavaTransformExecStream::open(), SingleInputExecStream::open(), MergeExecStream::open(), ExecStream::open(), CorrelationJoinExecStream::open(), ConfluenceExecStream::open(), ExecStream::prepare(), CartesianJoinExecStream::prepare(), and LbmMinusExecStream::restartSubtrahends().

ExecStreamId ExecStream::id [protected, inherited]

Identifier for this stream; local to its containing graph.

Definition at line 73 of file ExecStream.h.

Referenced by ExecStream::getStreamId().

std::string ExecStream::name [protected, inherited]

Name of stream, as known by optimizer.

Reimplemented from TraceSource.

Definition at line 78 of file ExecStream.h.

Referenced by ExecStream::ExecStream(), ExecStream::getName(), FlatFileExecStreamImpl::open(), and ExecStream::setName().

SharedDynamicParamManager ExecStream::pDynamicParamManager [protected, inherited]

The dynamic parameter manager available to this stream.

(Obtained at prepare() time. Keep a shared pointer in case the stream is reassigned to another graph for execution; cf ExecStreamGraph::mergeFrom())

Definition at line 85 of file ExecStream.h.

Referenced by CorrelationJoinExecStream::close(), SegBufferReaderExecStream::closeImpl(), LbmUnionExecStream::execute(), LbmSplicerExecStream::execute(), execute(), LbmChopperExecStream::execute(), CorrelationJoinExecStream::execute(), BarrierExecStream::execute(), LcsClusterReplaceExecStream::open(), LbmUnionExecStream::open(), LbmSplicerExecStream::open(), open(), LbmBitOpExecStream::open(), BTreeSearchExecStream::open(), BTreeInsertExecStream::open(), SegBufferWriterExecStream::open(), SegBufferReaderExecStream::open(), NestedLoopJoinExecStream::open(), CorrelationJoinExecStream::open(), ExecStream::prepare(), CalcExecStream::prepare(), NestedLoopJoinExecStream::processLeftInput(), LbmSearchExecStream::reachedTupleLimit(), ReshapeExecStream::readDynamicParams(), SegBufferWriterExecStream::readReaderRefCount(), BTreeSearchExecStream::readSearchKey(), BTreeSearchExecStream::readUpperBoundKey(), LbmSearchExecStream::setAdditionalKeys(), and LbmBitOpExecStream::writeStartRidParamValue().

SharedLogicalTxn ExecStream::pTxn [protected, inherited]

The transaction embracing the stream.

Obtained at open() time; but not released at close() time, to allow TableWriters to replay a txn. Keep a shared pointer in case the stream is reassigned to another graph for execution; cf ExecStreamGraph::mergeFrom())

Definition at line 94 of file ExecStream.h.

Referenced by FtrsTableWriterExecStream::commitSavepoint(), FtrsTableWriterExecStream::createSavepoint(), FtrsTableWriterExecStream::open(), ExecStream::open(), and FtrsTableWriterExecStream::rollbackSavepoint().