LbmSplicerExecStream Class Reference

LbmSplicerExecStream takes as input a stream of bitmap entries. More...

#include <LbmSplicerExecStream.h>

Inheritance diagram for LbmSplicerExecStream:


Public Member Functions
virtual void	prepare (LbmSplicerExecStreamParams const &params)
virtual void	open (bool restart)
	Opens this stream, acquiring any resources needed in order to be able to fetch data.
virtual ExecStreamResult	execute (ExecStreamQuantum const &quantum)
	Executes this stream.
virtual void	getResourceRequirements (ExecStreamResourceQuantity &minQuantity, ExecStreamResourceQuantity &optQuantity)
virtual void	closeImpl ()
	Implements ClosableObject.
virtual void	prepare (DiffluenceExecStreamParams const &params)
virtual void	prepare (SingleInputExecStreamParams const &params)
virtual void	prepare (ExecStreamParams const &params)
	Prepares this stream for execution.
virtual void	setOutputBufAccessors (std::vector< SharedExecStreamBufAccessor > const &outAccessors)
	Initializes the buffer accessors for outputs from this stream.
virtual ExecStreamBufProvision	getOutputBufProvision () const
	Indicate to the consumer if the buffer is provided by this exec stream which is the producer.
virtual void	setInputBufAccessors (std::vector< SharedExecStreamBufAccessor > const &inAccessors)
	Initializes the buffer accessors for inputs to this stream.
virtual ExecStreamBufProvision	getInputBufProvision () const
	Queries the BufferProvision which this stream requires of its inputs when consuming their tuples.
virtual bool	canEarlyClose ()
	Returns: true if the stream can be closed early
ExecStreamGraph &	getGraph () const
	Returns: reference to containing graph
ExecStreamId	getStreamId () const
	Returns: the identifier for this stream within containing graph
virtual void	getResourceRequirements (ExecStreamResourceQuantity &minQuantity, ExecStreamResourceQuantity &optQuantity, ExecStreamResourceSettingType &optType)
	Determines resource requirements for this stream.
virtual void	setResourceAllocation (ExecStreamResourceQuantity &quantity)
	Sets current resource allocation for this stream.
virtual void	setName (std::string const &)
	Sets unique name of this stream.
virtual std::string const &	getName () const
	Returns: the name of this stream, as known by the optimizer
virtual bool	mayBlock () const
	Queries whether this stream's implementation may block when execute() is called.
virtual void	checkAbort () const
	Checks whether there is an abort request for this stream's scheduler.
virtual ExecStreamBufProvision	getOutputBufConversion () const
	Queries the BufferProvision to which this stream needs its output to be converted, if any.
bool	isClosed () const
	Returns: whether the object has been closed
void	close ()
	Closes this object, releasing any unallocated resources.
virtual void	initTraceSource (SharedTraceTarget pTraceTarget, std::string name)
	For use when initialization has to be deferred until after construction.
void	trace (TraceLevel level, std::string message) const
	Records a trace message.
bool	isTracing () const
	Returns: true iff tracing is enabled for this source
bool	isTracingLevel (TraceLevel level) const
	Determines whether a particular level is being traced.
TraceTarget &	getTraceTarget () const
	Returns: the TraceTarget for this source
SharedTraceTarget	getSharedTraceTarget () const
	Returns: the SharedTraceTarget for this source
std::string	getTraceSourceName () const
	Gets the name of this source.
void	setTraceSourceName (std::string const &n)
	Sets the name of this source.
TraceLevel	getMinimumTraceLevel () const
void	disableTracing ()
virtual void	initErrorSource (SharedErrorTarget pErrorTarget, const std::string &name)
	For use when initialization has to be deferred until after construction.
void	postError (ErrorLevel level, const std::string &message, void *address, long capacity, int index)
	Posts an exception, such as a row exception.
void	postError (ErrorLevel level, const std::string &message, const TupleDescriptor &errorDesc, const TupleData &errorTuple, int index)
	Posts an exception, such as a row exception.
bool	hasTarget () const
	Returns: true iff an error target has been set
ErrorTarget &	getErrorTarget () const
	Returns: the ErrorTarget for this source
SharedErrorTarget	getSharedErrorTarget () const
	Returns: the SharedErrorTarget for this source
std::string	getErrorSourceName () const
	Gets the name of this source.
void	setErrorSourceName (std::string const &n)
	Sets the name of this source.
void	disableTarget ()
Protected Attributes
std::vector< SharedExecStreamBufAccessor >	outAccessors
	List of output buffer accessors.
TupleDescriptor	outputTupleDesc
	Output tuple descriptor.
SharedExecStreamBufAccessor	pInAccessor
bool	isOpen
	Whether this stream is currently open.
ExecStreamGraph *	pGraph
	Dataflow graph containing this stream.
ExecStreamId	id
	Identifier for this stream; local to its containing graph.
std::string	name
	Name of stream, as known by optimizer.
SharedDynamicParamManager	pDynamicParamManager
	The dynamic parameter manager available to this stream.
SharedLogicalTxn	pTxn
	The transaction embracing the stream.
ExecStreamResourceQuantity	resourceAllocation
	Resource quantities currently allocated to this stream.
SharedCacheAccessor	pQuotaAccessor
	CacheAccessor used for quota tracking.
SharedCacheAccessor	pScratchQuotaAccessor
	CacheAccessor used for scratch page quota tracking.
bool	needsClose
Private Member Functions
bool	isEmpty ()
	Determines, and remembers whether the bitmap being updated is empty.
bool	existingEntry (TupleData const &bitmapEntry)
	Determines whether a bitmap entry already exists in the bitmap index, based on the index key values.
bool	findMatchingBTreeEntry (TupleData const &bitmapEntry, TupleData &bTreeTupleData, bool leastUpper)
	Searches the btree, looking for the btree record matching the index keys and startRid of a specified bitmap entry.
bool	findBTreeEntry (TupleData const &bitmapEntry, TupleData &bTreeTupleData)
	Searches the btree, looking for the first btree record which overlaps with a given bitmap entry (or the insertion point for a new one if no existing match exists).
bool	ridOverlaps (LcsRid rid, TupleData &bitmapTupleData, bool firstByte)
	Determines whether a rid intersects the rid range spanned by a bitmap entry.
void	findBetterEntry (TupleData const &bitmapEntry)
	Determines if there exists a better entry in the btree, corresponding to the bitmap entry passed in, as compared to whatever is the current bitmap entry.
void	spliceEntry (TupleData &bitmapEntry)
	Splices a bitmap entry to the current entry under construction.
void	insertBitmapEntry ()
	Inserts current bitmap entry under construction into the bitmap index.
void	createNewBitmapEntry (TupleData const &bitmapEntry)
	Creates a new current bitmap entry for construction.
void	upsertSingleton (TupleData const &bitmapEntry)
	Efficient update or insert of tuples into unique indexes.
ExecStreamResult	getValidatedTuple ()
	Reads input tuples and filters out Rids that cause key violations.
bool	uniqueRequired (const TupleData &tuple)
	Determines whether a tuple key must be unique.
uint	countKeyRows (const TupleData &tuple)
	Counts the number of rows in the index with a particular key value, prior to modification.
void	setUpsertRid (LcsRid rid)
	For key values that can only have one valid rid, remembers the value of a single rid that was chosen to be for insert or update.
const LcsRid *	getUpsertRidPtr () const
	For key values that can only have one valid rid, gets a pointer to a rid to be inserted into the index.
void	postViolation (const TupleData &input, const TupleData &violation)
	Generates an error record and posts it to an ErrorTarget.
Private Attributes
SegmentAccessor	scratchAccessor
	Scratch accessor.
BTreeDescriptor	writeBTreeDesc
	Descriptor corresponding to the btree that splicer writes to.
BTreeDescriptor	deletionBTreeDesc
	Descriptor corresponding to the deletion index btree.
DynamicParamId	insertRowCountParamId
	Parameter id of dynamic parameter containing final insert row count.
bool	createNewIndex
	If true, create a new index that the splicer will be writing.
bool	newIndexCreated
	True if a new index was created.
DynamicParamId	writeRowCountParamId
	Parameter id of the dynamic parameter used to write the row count affected by this stream that will be read downstream.
uint	maxEntrySize
	Maximum size of an LbmEntry.
boost::scoped_array< FixedBuffer >	bitmapBuffer
	Buffer for LbmEntry.
boost::scoped_array< FixedBuffer >	mergeBuffer
	Buffer for merges on LbmEntry.
SharedLbmEntry	pCurrentEntry
	Current bitmap entry under construction.
bool	currEntry
	true if current entry is under construction
bool	currExistingEntry
	True if current bitmap entry under construction refers to an already existing entry in the bitmap index.
LcsRid	currBTreeStartRid
	Start rid of the current existing entry in the bitmap index.
bool	emptyTableUnknown
	True if we need to determine emptyTable.
bool	emptyTable
	True if table that the bitmap is being constructed on was empty to begin with.
SharedBTreeWriter	bTreeWriter
	Writes btree index corresponding to bitmap.
bool	bTreeWriterMoved
	Whether btree writer position was moved for unique constraint validation.
bool	isDone
	True if output has been produced.
TupleData	inputTuple
	Input tuple.
TupleData	singletonTuple
	Input tuple corresponding to singleton entries.
TupleData	outputTuple
	Output tuple containing rowcount.
RecordNum	numRowsLoaded
	Number of rows loaded into bitmap index.
TupleData	bTreeTupleData
	Tuple data for reading bitmaps from btree index.
TupleData	tempBTreeTupleData
TupleDescriptor	bitmapTupleDesc
	Tuple descriptor representing bitmap tuple.
uint	nIdxKeys
	Number of keys in the bitmap index, excluding the starting rid.
bool	computeRowCount
	True if splicer needs to compute the row count result rather than reading it from a dynamic parameter.
bool	uniqueKey
	Whether the index being updated is a unique key.
bool	currValidation
	Whether an input tuple is currently being validated.
bool	firstValidation
	Whether any tuple has been validated yet.
TupleDataWithBuffer	currUniqueKey
	The current unique key value being validated.
uint	nKeyRows
	Number of rows for the current key value (after taking into account the deletion index).
LbmDeletionIndexReader	deletionReader
	Reads rids from the deletion index.
TupleData	deletionTuple
	Current tuple for deletion index rid reader.
LbmTupleRidReader	inputRidReader
	Reads rids from an input tuple.
bool	nullUpsertRid
	True if no RID has been accepted as the update/insert for the current key value.
LcsRid	upsertRid
	If a RID has been accepted as the update/insert for the current key value, this contains the value of the accepted RID.
SharedExecStreamBufAccessor	violationAccessor
	Accessor for the violation output stream.
TupleData	violationTuple
	Violation data tuple.
std::string	errorMsg
	Error message for constraint violations.
TupleDescriptor	errorDesc
	TupleDescriptor for error records.
TupleData	errorTuple
	TupleData used to build error records.
SnapshotRandomAllocationSegment *	pSnapshotSegment
	Underlying snapshot segment corresponding to the index that will be written.
PageId	origRootPageId
	The original root pageId of the index that will be written.

Detailed Description

LbmSplicerExecStream takes as input a stream of bitmap entries.

If possible, it splices the entries into larger entries before writing them into a btree. The bitmap entries from the input can correspond to new entries, or they may need to be spliced into existing entries in the btree.

As output, LbmSplicerExecStream writes out a count corresponding to the number of new row values inserted into the btree. This value is either directly computed by LbmSplicerExecStream (provided the bitmap entry inputs all correspond to singleton rids), or it is passed into LbmSplicerExecStream via a dynamic parameter.

Optionally, LbmSplicerExecStream may also write out the row count into a dynamic parameter that is read downstream by another exec stream.

If the index being updated is a primary key or a unique key, then multiple entries mapping to the same key value are rejected. When a new key value is encountered, undeleted entries are counted for that key value. If an entry already exists, then all inputs with the key value are rejected. Otherwise, if no entry exists, a single input RID is accepted as the valid insert (or update). (We deviate from the SQL standard.) In order to count existing entries, we read from the deletion index. Violations are written to a second output. Unique keys differ from primary keys in that multiple entries are allowed for null keys.

Author:: Zelaine Fong

Version:

Id: //open/dev/fennel/lucidera/bitmap/LbmSplicerExecStream.h#16

Definition at line 78 of file LbmSplicerExecStream.h.

Member Function Documentation

bool LbmSplicerExecStream::isEmpty ( ) [private]

Determines, and remembers whether the bitmap being updated is empty.

Should only be called when the bitmap is actually ready for access.

Definition at line 176 of file LbmSplicerExecStream.cpp.

References bTreeWriter, emptyTable, emptyTableUnknown, scratchAccessor, and writeBTreeDesc.

Referenced by countKeyRows(), existingEntry(), findBetterEntry(), and upsertSingleton().

00177 {
00178     if (emptyTableUnknown) {
00179         if (bTreeWriter->searchFirst() == false) {
00180             bTreeWriter->endSearch();
00181             emptyTable = true;
00182             // switch writer to monotonic now that we know the table
00183             // is empty
00184             bTreeWriter.reset();
00185             bTreeWriter = SharedBTreeWriter(
00186                 new BTreeWriter(writeBTreeDesc, scratchAccessor, true));
00187         } else {
00188             emptyTable = false;
00189         }
00190         emptyTableUnknown = false;
00191     }
00192     return emptyTable;
00193 }

bool LbmSplicerExecStream::existingEntry ( TupleData const & bitmapEntry ) [private]

Determines whether a bitmap entry already exists in the bitmap index, based on the index key values.

If multiple entries have the same key value, locates the last entry with that key value, i.e., the one with the largest starting rid value. Avoids index lookup if table was empty at the start of the load. Sets current bitmap entry either to the existing btree entry (if it exists). Otherwise, sets it to the bitmap entry passed in.

Parameters:

bitmapEntry

tupledata corresponding to bitmap entry being checked

Returns:: true if entry already exists in bitmap index; always return false if table was empty at the start of the load

Definition at line 302 of file LbmSplicerExecStream.cpp.

References bTreeTupleData, bTreeWriterMoved, createNewBitmapEntry(), currExistingEntry, findBTreeEntry(), and isEmpty().

Referenced by execute().

00303 {
00304     if (!isEmpty()) {
00305         // if entry already exists in the btree, then the current bitmap
00306         // entry becomes that existing btree entry
00307         if (findBTreeEntry(bitmapEntry, bTreeTupleData)) {
00308             currExistingEntry = true;
00309             createNewBitmapEntry(bTreeTupleData);
00310             bTreeWriterMoved = false;
00311             return true;
00312         }
00313     }
00314 
00315     // set current bitmap entry to new entry
00316     currExistingEntry = false;
00317     createNewBitmapEntry(bitmapEntry);
00318     return false;
00319 }

bool LbmSplicerExecStream::findMatchingBTreeEntry	(	TupleData const &	bitmapEntry,
		TupleData &	bTreeTupleData,
		bool	leastUpper
	)			`[private]`

Searches the btree, looking for the btree record matching the index keys and startRid of a specified bitmap entry.

Parameters:

	bitmapEntry	entry for which we are trying to match
	bTreeTupleData	tuple data where the btree record will be returned if a matching entry is found; if a non-match is found, the tuple data contains the greatest lower bound btree entry found
	leastUpper	if true, use a least upper bound search to locate the btree record

Returns:: true if a matching entry is found in the btree; false otherwise; (in this case, bTreeWriter is positioned at the location of the greatest lower bound btree entry corresponding to the bitmap entry)

Definition at line 321 of file LbmSplicerExecStream.cpp.

References bTreeTupleData, bTreeWriter, and DUP_SEEK_BEGIN.

Referenced by findBTreeEntry(), and insertBitmapEntry().

00325 {
00326     bool match =
00327         bTreeWriter->searchForKey(
00328             bitmapEntry,
00329             DUP_SEEK_BEGIN,
00330             leastUpper);
00331     bTreeWriter->getTupleAccessorForRead().unmarshal(bTreeTupleData);
00332     return match;
00333 }

bool LbmSplicerExecStream::findBTreeEntry	(	TupleData const &	bitmapEntry,
		TupleData &	bTreeTupleData
	)			`[private]`

Searches the btree, looking for the first btree record which overlaps with a given bitmap entry (or the insertion point for a new one if no existing match exists).

Parameters:

	bitmapEntry	entry for which to find an overlap match
	bTreeTupleData	tuple data where the btree record will be returned if a matching entry is found (otherwise invalid data references are returned here after search)

Returns:: true if entry found in btree; false if no entry found (in this case, bTreeWriter is positioned to correct location for inserting new entry)

Definition at line 335 of file LbmSplicerExecStream.cpp.

References bitmapTupleDesc, bTreeTupleData, bTreeWriter, TupleDescriptor::compareTuplesKey(), DUP_SEEK_BEGIN, findMatchingBTreeEntry(), nIdxKeys, ridOverlaps(), and LbmSegment::roundToByteBoundary().

Referenced by countKeyRows(), existingEntry(), findBetterEntry(), and upsertSingleton().

00337 {
00338     // First do a greatest lower bound lookup into the btree, searching on
00339     // both the actual key index values and the startRid
00340     bool match =
00341         findMatchingBTreeEntry(bitmapEntry, bTreeTupleData, (nIdxKeys > 0));
00342 
00343     if (match == false) {
00344         if (nIdxKeys == 0) {
00345             // If there are no index keys, then we are splicing individual
00346             // rids.  In that case, we should always be splicing into the
00347             // best btree entry available.  First see if the greatest lower
00348             // bound entry overlaps the rid we're looking for.  If it doesn't,
00349             // try the next entry.  If that doesn't overlap, go back to the
00350             // greatest lower bound entry so we can splice the new rid to
00351             // the end of that entry.
00352             LcsRid newRid = *reinterpret_cast<LcsRid const *>
00353                 (bitmapEntry[0].pData);
00354             if (!ridOverlaps(newRid, bTreeTupleData, false)) {
00355                 match = bTreeWriter->searchNext();
00356                 if (match) {
00357                     bTreeWriter->getTupleAccessorForRead().unmarshal(
00358                         bTreeTupleData);
00359                     if (!ridOverlaps(newRid, bTreeTupleData, true)) {
00360                         match = bTreeWriter->searchForKey(
00361                             bitmapEntry, DUP_SEEK_BEGIN, false);
00362                         assert(match == false);
00363                         bTreeWriter->getTupleAccessorForRead().unmarshal(
00364                             bTreeTupleData);
00365                     }
00366                 }
00367             }
00368             match = true;
00369 
00370         } else {
00371             // In the case where we have actual index keys, we've done a
00372             // least upper bound search to locate the entry.  See if
00373             // the keys without the startRid match.  If they do, then we've
00374             // located a singleton rid that overlaps with the entry we're
00375             // trying to splice.  If so, that is the entry we want to splice
00376             // into.  Otherwise, the desired entry may be in front of the
00377             // one we've located.  Therefore, we need to do a greatest lower
00378             // bound search to locate that previous entry (since we don't have
00379             // a BTreeReader::searchPrev method), and then compare the keys
00380             // to see if we have a match.
00381             if (!bTreeWriter->isSingular()) {
00382                 int keyComp =
00383                     bitmapTupleDesc.compareTuplesKey(
00384                         bTreeTupleData,
00385                         bitmapEntry,
00386                         nIdxKeys);
00387                 if (keyComp == 0) {
00388                     assert(
00389                         LbmSegment::roundToByteBoundary(
00390                             *reinterpret_cast<LcsRid const *>(
00391                                 bTreeTupleData[nIdxKeys].pData)) ==
00392                         LbmSegment::roundToByteBoundary(
00393                             *reinterpret_cast<LcsRid const *>(
00394                                 bitmapEntry[nIdxKeys].pData)));
00395                     return true;
00396                 }
00397             }
00398 
00399             // Position to the previous entry by doing a glb search
00400             match =
00401                 bTreeWriter->searchForKey(bitmapEntry, DUP_SEEK_BEGIN, false);
00402             assert(match == false);
00403             bTreeWriter->getTupleAccessorForRead().unmarshal(bTreeTupleData);
00404             int keyComp =
00405                 bitmapTupleDesc.compareTuplesKey(
00406                     bTreeTupleData,
00407                     bitmapEntry,
00408                     nIdxKeys);
00409             if (keyComp == 0) {
00410                 match = true;
00411             }
00412         }
00413     }
00414     return match;
00415 }

bool LbmSplicerExecStream::ridOverlaps	(	LcsRid	rid,
		TupleData &	bitmapTupleData,
		bool	firstByte
	)			`[private]`

Determines whether a rid intersects the rid range spanned by a bitmap entry.

If the bitmap is a singleton, then the byte occupied by the singleton rid is used in determining intersection.

Parameters:

	rid	the rid
	bitmapTupleData	tupleData representing a bitmap entry
	firstByte	if true, only consider overlap in the first byte of the bitmap

Returns:: true if the rid overlaps the bitmap entry

Definition at line 417 of file LbmSplicerExecStream.cpp.

References LbmEntry::getRowCount(), LbmSegment::LbmOneByteSize, and LbmSegment::roundToByteBoundary().

Referenced by findBTreeEntry().

00421 {
00422     // Convert singletons to the rid range representing all bits in the byte
00423     // corresponding to the singleton rid
00424     LcsRid startRid =
00425         LbmSegment::roundToByteBoundary(
00426             *reinterpret_cast<LcsRid const *>(bitmapTupleData[0].pData));
00427     uint rowCount;
00428     if (firstByte) {
00429        rowCount = LbmSegment::LbmOneByteSize;
00430     } else {
00431         rowCount = LbmEntry::getRowCount(bitmapTupleData);
00432         if (rowCount == 1) {
00433             rowCount = LbmSegment::LbmOneByteSize;
00434         }
00435     }
00436     if (rid >= startRid && rid < startRid + rowCount) {
00437         return true;
00438     } else {
00439         return false;
00440     }
00441 }

void LbmSplicerExecStream::findBetterEntry ( TupleData const & bitmapEntry ) [private]

Determines if there exists a better entry in the btree, corresponding to the bitmap entry passed in, as compared to whatever is the current bitmap entry.

If there is, the current entry is written to the btree, and the current entry is set to the btree entry found.

Parameters:

bitmapEntry

tupleData corresponding to the entry to be spliced

Definition at line 443 of file LbmSplicerExecStream.cpp.

References bTreeTupleData, computeRowCount, createNewBitmapEntry(), currExistingEntry, findBTreeEntry(), insertBitmapEntry(), isEmpty(), LbmSegment::LbmOneByteSize, pCurrentEntry, and LbmSegment::roundToByteBoundary().

Referenced by execute().

00444 {
00445     // If there is a better btree entry, write out the current entry and set
00446     // the current entry to the btree entry found.  The btree entry is "better"
00447     // if it's the entry that we should be splicing the new rid into.
00448     //
00449     // In other words, one of the following conditions must be true:
00450     //
00451     // 1) bTreeStartRid <= newRid < currentStartRid
00452     // 2) currentStartRid < bTreeStartRid <= newRid
00453     // 3) newRid <= bTreeStartRid < currentStartRid
00454     //
00455     // NOTE - condition 1 occurs when the current bitmap entry is split, and
00456     // the current entry becomes the right portion of that bitmap entry.
00457     // Also, conditions 1 and 3 can be combined into:
00458     //
00459     // currentStartRid > newRid && currentStartRid > bTreeStartRid
00460 
00461     assert(computeRowCount);
00462     if (!isEmpty()) {
00463         if (findBTreeEntry(bitmapEntry, bTreeTupleData)) {
00464             LcsRid bTreeRid =
00465                 LbmSegment::roundToByteBoundary(
00466                     *reinterpret_cast<LcsRid const *> (
00467                         bTreeTupleData[0].pData));
00468             LcsRid newRid = *reinterpret_cast<LcsRid const *>
00469                 (bitmapEntry[0].pData);
00470             LcsRid currRid =
00471                 LbmSegment::roundToByteBoundary(pCurrentEntry->getStartRID());
00472 
00473             if ((currRid > newRid && currRid > bTreeRid) ||
00474                 (newRid >= bTreeRid && bTreeRid > currRid))
00475             {
00476                 // If the current entry is a superset of the btree entry found,
00477                 // then ignore the btree entry, and continuing splicing into
00478                 // the current entry
00479                 uint rowCount = pCurrentEntry->getRowCount();
00480                 if (rowCount == 1) {
00481                     rowCount = LbmSegment::LbmOneByteSize;
00482                 }
00483                 if ((bTreeRid >= currRid) && (bTreeRid < currRid + rowCount)) {
00484                     return;
00485                 }
00486 
00487                 // Write out the current entry before we switch over to the
00488                 // new one
00489                 insertBitmapEntry();
00490                 currExistingEntry = true;
00491                 createNewBitmapEntry(bTreeTupleData);
00492             }
00493         }
00494     }
00495 }

void LbmSplicerExecStream::spliceEntry ( TupleData & bitmapEntry ) [private]

Splices a bitmap entry to the current entry under construction.

If the combined size of the two entries exceeds the max size of a bitmap entry, the current entry will be inserted into the btree and the new entry becomes the current entry under construction.

Parameters:

bitmapEntry

tupledata corresponding to the entry to be spliced

Definition at line 497 of file LbmSplicerExecStream.cpp.

References createNewBitmapEntry(), insertBitmapEntry(), pCurrentEntry, LbmEntry::toString(), and TRACE_FINE.

Referenced by execute().

00498 {
00499     FENNEL_TRACE(TRACE_FINE, "splice two entries");
00500     FENNEL_TRACE(TRACE_FINE, pCurrentEntry->toString());
00501     FENNEL_TRACE(TRACE_FINE, LbmEntry::toString(bitmapEntry));
00502 
00503     if (!pCurrentEntry->mergeEntry(bitmapEntry)) {
00504         insertBitmapEntry();
00505         createNewBitmapEntry(bitmapEntry);
00506     }
00507 }

void LbmSplicerExecStream::insertBitmapEntry ( ) [private]

Inserts current bitmap entry under construction into the bitmap index.

Definition at line 509 of file LbmSplicerExecStream.cpp.

References bTreeTupleData, bTreeWriter, bTreeWriterMoved, computeRowCount, currBTreeStartRid, currExistingEntry, DUP_FAIL, findMatchingBTreeEntry(), nIdxKeys, pCurrentEntry, tempBTreeTupleData, LbmEntry::toString(), and TRACE_FINE.

Referenced by execute(), findBetterEntry(), and spliceEntry().

00510 {
00511     TupleData const &indexTuple = pCurrentEntry->produceEntryTuple();
00512 
00513     // implement btree updates as deletes followed by inserts
00514     if (currExistingEntry) {
00515         // when we're inserting in random singleton mode, we may have
00516         // repositioned in the btree, trying to find a better btree entry,
00517         // so we need to position back to the original btree entry before
00518         // we delete it. the btree may also reposition for validation
00519         if (bTreeWriterMoved) {
00520             for (uint i = 0; i < nIdxKeys; i++) {
00521                 tempBTreeTupleData[i] = indexTuple[i];
00522             }
00523         }
00524         if (computeRowCount || bTreeWriterMoved) {
00525             tempBTreeTupleData[nIdxKeys].pData =
00526                 (PConstBuffer) &currBTreeStartRid;
00527             bool match =
00528                 findMatchingBTreeEntry(
00529                     tempBTreeTupleData,
00530                     tempBTreeTupleData,
00531                     false);
00532             permAssert(match);
00533         }
00534         FENNEL_TRACE(TRACE_FINE, "delete Tuple from BTree");
00535         FENNEL_TRACE(TRACE_FINE, LbmEntry::toString(bTreeTupleData));
00536 
00537         bTreeWriter->deleteCurrent();
00538         currExistingEntry = false;
00539     }
00540 
00541     FENNEL_TRACE(TRACE_FINE, "insert Tuple into BTree");
00542     FENNEL_TRACE(TRACE_FINE, LbmEntry::toString(indexTuple));
00543 
00544     bTreeWriter->insertTupleData(indexTuple, DUP_FAIL);
00545 }

void LbmSplicerExecStream::createNewBitmapEntry ( TupleData const & bitmapEntry ) [private]

Creates a new current bitmap entry for construction.

Parameters:

bitmapEntry

tupledata corresponding to initial value for bitmap entry

Definition at line 547 of file LbmSplicerExecStream.cpp.

References currBTreeStartRid, currEntry, nIdxKeys, and pCurrentEntry.

Referenced by existingEntry(), findBetterEntry(), and spliceEntry().

00548 {
00549     pCurrentEntry->setEntryTuple(bitmapEntry);
00550     currBTreeStartRid = *reinterpret_cast<LcsRid const *>
00551         (bitmapEntry[nIdxKeys].pData);
00552     currEntry = true;
00553 }

void LbmSplicerExecStream::upsertSingleton ( TupleData const & bitmapEntry ) [private]

Efficient update or insert of tuples into unique indexes.

Parameters:

bitmapEntry

tupledata corresponding to the entry to be upserted

Definition at line 555 of file LbmSplicerExecStream.cpp.

References bTreeTupleData, bTreeWriter, DUP_FAIL, findBTreeEntry(), isEmpty(), and LbmEntry::isSingleton().

Referenced by execute().

00556 {
00557     if (!isEmpty()) {
00558         if (findBTreeEntry(bitmapEntry, bTreeTupleData)) {
00559             assert(LbmEntry::isSingleton(bTreeTupleData));
00560             bTreeWriter->deleteCurrent();
00561         }
00562     }
00563     bTreeWriter->insertTupleData(bitmapEntry, DUP_FAIL);
00564 }

ExecStreamResult LbmSplicerExecStream::getValidatedTuple ( ) [private]

Reads input tuples and filters out Rids that cause key violations.

Key violations are posted to an ErrorTarget for logging while Rids are output to a violation stream for deletion. This method completes when a valid tuple has been generated. Sets the inputTuple field.

Returns:: EXECRC_YIELD on success, or other status codes for input stream underflow, end of input stream, or violation stream overflow

Definition at line 566 of file LbmSplicerExecStream.cpp.

References LbmIterableRidReader::advance(), bitmapTupleDesc, bTreeWriter, TupleDescriptor::compareTuplesKey(), computeRowCount, countKeyRows(), BTreeBuilder::createEmptyRoot(), createNewIndex, currUniqueKey, currValidation, deletionReader, emptyTable, emptyTableUnknown, EXECRC_BUF_OVERFLOW, EXECRC_BUF_UNDERFLOW, EXECRC_YIELD, firstValidation, LbmIterableRidReader::getNext(), BTreeAccessBase::getRootPageId(), getUpsertRidPtr(), LbmIterableRidReader::hasNext(), LbmTupleRidReader::init(), inputRidReader, inputTuple, newIndexCreated, nIdxKeys, nKeyRows, NULL_PAGE_ID, nullUpsertRid, numRowsLoaded, LbmIterableRidReader::peek(), SingleInputExecStream::pInAccessor, postViolation(), SegmentAccessor::pSegment, TupleDataWithBuffer::resetBuffer(), BTreeDescriptor::rootPageId, scratchAccessor, LbmDeletionIndexReader::searchForRid(), BTreeDescriptor::segmentAccessor, setUpsertRid(), singletonTuple, LbmEntry::toString(), TRACE_FINE, uniqueRequired(), violationAccessor, violationTuple, and writeBTreeDesc.

Referenced by execute().

00567 {
00568     while (true) {
00569         if (!currValidation) {
00570             if (!pInAccessor->demandData()) {
00571                 return EXECRC_BUF_UNDERFLOW;
00572             }
00573 
00574             if (computeRowCount) {
00575                 pInAccessor->unmarshalTuple(singletonTuple);
00576                 inputTuple[0] = singletonTuple[0];
00577                 inputTuple[1].pData = NULL;
00578                 inputTuple[1].cbData = 0;
00579                 inputTuple[2].pData = NULL;
00580                 inputTuple[2].cbData = 0;
00581                 numRowsLoaded++;
00582             } else {
00583                 pInAccessor->unmarshalTuple(inputTuple);
00584             }
00585 
00586             FENNEL_TRACE(TRACE_FINE, "input Tuple from sorter");
00587             FENNEL_TRACE(TRACE_FINE, LbmEntry::toString(inputTuple));
00588 
00589             // If we're creating a new index, we need to defer creating it until
00590             // we know that there are new input tuples.  Otherwise, there's no
00591             // point in creating it.
00592             if (createNewIndex && !newIndexCreated) {
00593                 newIndexCreated = true;
00594                 writeBTreeDesc.rootPageId = NULL_PAGE_ID;
00595                 BTreeBuilder builder(
00596                     writeBTreeDesc,
00597                     writeBTreeDesc.segmentAccessor.pSegment);
00598                 builder.createEmptyRoot();
00599                 writeBTreeDesc.rootPageId = builder.getRootPageId();
00600                 emptyTable = true;
00601                 emptyTableUnknown = false;
00602                 bTreeWriter = SharedBTreeWriter(
00603                     new BTreeWriter(
00604                         writeBTreeDesc,
00605                         scratchAccessor,
00606                         emptyTable));
00607             }
00608 
00609             if (!uniqueRequired(inputTuple)) {
00610                 return EXECRC_YIELD;
00611             }
00612 
00613             // count existing entries for key, if the key has not been seen yet
00614             if (firstValidation
00615                 || bitmapTupleDesc.compareTuplesKey(
00616                     inputTuple, currUniqueKey, nIdxKeys) != 0)
00617             {
00618                 firstValidation = false;
00619                 currUniqueKey.resetBuffer();
00620                 for (uint i = 0; i < nIdxKeys; i++) {
00621                     currUniqueKey[i].memCopyFrom(inputTuple[i]);
00622                 }
00623                 nKeyRows = countKeyRows(inputTuple);
00624             }
00625 
00626             // prepare to emit rids for key violations
00627             inputRidReader.init(inputTuple);
00628             nullUpsertRid = true;
00629             currValidation = true;
00630         }
00631 
00632         // if there were no undeleted values for the current key, we can
00633         // insert/update a single rid
00634         if (nKeyRows == 0) {
00635             assert(getUpsertRidPtr() == NULL);
00636             if (!createNewIndex) {
00637                 setUpsertRid(inputRidReader.getNext());
00638                 nKeyRows++;
00639             } else {
00640                 // Loop until we find a non-deleted rid.  Deleted rids only
00641                 // occur when rebuilding an existing index.
00642                 do {
00643                     LcsRid rid = inputRidReader.getNext();
00644                     if (!deletionReader.searchForRid(rid)) {
00645                         setUpsertRid(rid);
00646                         nKeyRows++;
00647                         break;
00648                     }
00649                 } while (inputRidReader.hasNext());
00650             }
00651         }
00652 
00653         // all other rids are rejected as duplicate keys, unless they're deleted
00654         // rids
00655         while (inputRidReader.hasNext()) {
00656             if (!violationTuple.size()) {
00657                 // if there is a possibility of violations, the splicer should
00658                 // have been initialized with a second output
00659                 permAssert(false);
00660             }
00661             LcsRid rid = inputRidReader.peek();
00662             if (createNewIndex && deletionReader.searchForRid(rid)) {
00663                 inputRidReader.advance();
00664                 continue;
00665             }
00666             violationTuple[0].pData = reinterpret_cast<PConstBuffer>(&rid);
00667             violationTuple[0].cbData = 8;
00668             if (!violationAccessor->produceTuple(violationTuple)) {
00669                 return EXECRC_BUF_OVERFLOW;
00670             }
00671             postViolation(inputTuple, violationTuple);
00672             inputRidReader.advance();
00673         }
00674         currValidation = false;
00675 
00676         if (getUpsertRidPtr() != NULL) {
00677             // since a rid was accepted, return it as a validated tuple
00678             inputTuple[nIdxKeys].pData =
00679                 reinterpret_cast<PConstBuffer>(getUpsertRidPtr());
00680             inputTuple[nIdxKeys].cbData = 8;
00681             inputTuple[nIdxKeys + 1].pData = NULL;
00682             inputTuple[nIdxKeys + 1].cbData = 0;
00683             inputTuple[nIdxKeys + 2].pData = NULL;
00684             inputTuple[nIdxKeys + 2].cbData = 0;
00685             return EXECRC_YIELD;
00686         }
00687 
00688         // every rid in the current tuple was either rejected or already
00689         // deleted, so try the next tuple
00690         pInAccessor->consumeTuple();
00691     }
00692 }

bool LbmSplicerExecStream::uniqueRequired ( const TupleData & tuple ) [private]

Determines whether a tuple key must be unique.

A tuple key must be unique if the index has a uniqueness constraint and the tuple key does not contain any nulls.

Parameters:

tuple

input tuple to be checked for uniqueness requirments

Returns:: true if the tuple key must be unique

Definition at line 694 of file LbmSplicerExecStream.cpp.

References nIdxKeys, and uniqueKey.

Referenced by execute(), and getValidatedTuple().

00695 {
00696     if (uniqueKey) {
00697         for (uint i = 0; i < nIdxKeys; i++) {
00698             if (tuple[i].isNull()) {
00699                 return false;
00700             }
00701         }
00702         return true;
00703     }
00704     return false;
00705 }

uint LbmSplicerExecStream::countKeyRows ( const TupleData & tuple ) [private]

Counts the number of rows in the index with a particular key value, prior to modification.

This count factors in the deletion index.

Parameters:

tuple

tupledata containing the key value to search the index for

Returns:: number of rows with a key value, less the deleted rows

Definition at line 707 of file LbmSplicerExecStream.cpp.

References bTreeTupleData, deletionReader, findBTreeEntry(), LbmEntry::getStartRid(), inputTuple, isEmpty(), LbmEntry::isSingleton(), LbmDeletionIndexReader::searchForRid(), and uniqueKey.

Referenced by getValidatedTuple().

00708 {
00709     assert(uniqueKey);
00710     if (isEmpty()) {
00711         return 0;
00712     }
00713 
00714     if (!findBTreeEntry(inputTuple, bTreeTupleData)) {
00715         return 0;
00716     }
00717     assert(LbmEntry::isSingleton(bTreeTupleData));
00718     LcsRid rid = LbmEntry::getStartRid(bTreeTupleData);
00719 
00720     if (deletionReader.searchForRid(rid)) {
00721         return 0;
00722     }
00723     return 1;
00724 }

void LbmSplicerExecStream::setUpsertRid ( LcsRid rid ) [inline, private]

For key values that can only have one valid rid, remembers the value of a single rid that was chosen to be for insert or update.

Parameters:

rid

a rid to be inserted into the index

Definition at line 508 of file LbmSplicerExecStream.h.

References nullUpsertRid, and upsertRid.

Referenced by getValidatedTuple().

00509 {
00510     nullUpsertRid = false;
00511     upsertRid = rid;
00512 }

const LcsRid * LbmSplicerExecStream::getUpsertRidPtr ( ) const [inline, private]

For key values that can only have one valid rid, gets a pointer to a rid to be inserted into the index.

Returns:: a pointer to a rid to be inserted, or NULL if none was set

Definition at line 514 of file LbmSplicerExecStream.h.

References nullUpsertRid, and upsertRid.

Referenced by getValidatedTuple().

00515 {
00516     return (nullUpsertRid ? NULL : &upsertRid);
00517 }

void LbmSplicerExecStream::postViolation	(	const TupleData &	input,
		const TupleData &	violation
	)			`[private]`

Generates an error record and posts it to an ErrorTarget.

Parameters:

	input	the input tuple causing the error
	violation	the violation tuple created for the error

Definition at line 726 of file LbmSplicerExecStream.cpp.

References bitmapTupleDesc, TupleData::compute(), errorDesc, errorMsg, errorTuple, nIdxKeys, ErrorSource::postError(), and ROW_ERROR.

Referenced by getValidatedTuple().

00728 {
00729     if (!errorTuple.size()) {
00730         for (uint i = 0; i < nIdxKeys + 1; i++) {
00731             errorDesc.push_back(bitmapTupleDesc[i]);
00732         }
00733         errorTuple.compute(errorDesc);
00734         errorMsg = FennelResource::instance().uniqueConstraintViolated();
00735     }
00736 
00737     for (uint i = 0; i < nIdxKeys; i++) {
00738         errorTuple[i] = input[i];
00739     }
00740     errorTuple[nIdxKeys] = violation[0];
00741 
00742     postError(ROW_ERROR, errorMsg, errorDesc, errorTuple, -1);
00743 }

void LbmSplicerExecStream::prepare ( LbmSplicerExecStreamParams const & params ) [virtual]

Definition at line 32 of file LbmSplicerExecStream.cpp.

References bitmapTupleDesc, LbmSplicerExecStreamParams::bTreeParams, bTreeTupleData, TupleData::compute(), TupleDataWithBuffer::computeAndAllocate(), computeRowCount, BTreeExecStream::copyParamsToDescriptor(), LbmSplicerExecStreamParams::createNewIndex, createNewIndex, currUniqueKey, deletionBTreeDesc, deletionTuple, LbmEntry::getSizeBounds(), inputTuple, LbmSplicerExecStreamParams::insertRowCountParamId, insertRowCountParamId, BTreeDescriptor::keyProjection, maxEntrySize, nIdxKeys, numRowsLoaded, opaqueToInt(), origRootPageId, DiffluenceExecStream::outAccessors, outputTuple, SingleInputExecStream::pInAccessor, DiffluenceExecStream::prepare(), SegmentAccessor::pSegment, BTreeDescriptor::rootPageId, scratchAccessor, BTreeDescriptor::segmentAccessor, singletonTuple, tempBTreeTupleData, BTreeDescriptor::tupleDescriptor, uniqueKey, violationAccessor, violationTuple, writeBTreeDesc, LbmSplicerExecStreamParams::writeRowCountParamId, and writeRowCountParamId.

00033 {
00034     DiffluenceExecStream::prepare(params);
00035     scratchAccessor = params.scratchAccessor;
00036 
00037     // Setup btree accessed by splicer
00038     assert(params.bTreeParams.size() <= 2);
00039     assert(params.bTreeParams[0].pRootMap == NULL);
00040     BTreeExecStream::copyParamsToDescriptor(
00041         writeBTreeDesc,
00042         params.bTreeParams[0],
00043         params.pCacheAccessor);
00044 
00045     insertRowCountParamId = params.insertRowCountParamId;
00046     computeRowCount = (opaqueToInt(insertRowCountParamId) == 0);
00047     writeRowCountParamId = params.writeRowCountParamId;
00048 
00049     bitmapTupleDesc = writeBTreeDesc.tupleDescriptor;
00050     bTreeTupleData.compute(bitmapTupleDesc);
00051     tempBTreeTupleData.compute(bitmapTupleDesc);
00052     inputTuple.compute(bitmapTupleDesc);
00053     nIdxKeys = writeBTreeDesc.keyProjection.size() - 1;
00054 
00055     // if the rowcount needs to be computed, then the input contains singleton
00056     // rids; so setup a special tupleData to receive that input
00057     if (computeRowCount) {
00058         assert(nIdxKeys == 0);
00059         assert(pInAccessor->getTupleDesc().size() == 1);
00060         singletonTuple.compute(pInAccessor->getTupleDesc());
00061     } else {
00062         assert(
00063             writeBTreeDesc.tupleDescriptor == pInAccessor->getTupleDesc());
00064     }
00065 
00066     uint minEntrySize;
00067     LbmEntry::getSizeBounds(
00068         bitmapTupleDesc,
00069         writeBTreeDesc.segmentAccessor.pSegment->getUsablePageSize(),
00070         minEntrySize,
00071         maxEntrySize);
00072 
00073     // setup output tuple
00074     outputTuple.compute(outAccessors[0]->getTupleDesc());
00075     outputTuple[0].pData = (PConstBuffer) &numRowsLoaded;
00076     assert(outputTuple.size() == 1);
00077 
00078     // constraint checking
00079     uniqueKey = false;
00080     if (params.bTreeParams.size() >= 2) {
00081         uniqueKey = true;
00082         BTreeExecStream::copyParamsToDescriptor(
00083             deletionBTreeDesc,
00084             params.bTreeParams[1],
00085             params.pCacheAccessor);
00086         deletionTuple.compute(deletionBTreeDesc.tupleDescriptor);
00087 
00088         TupleDescriptor currUniqueKeyDesc;
00089         for (uint i = 0; i < nIdxKeys; i++) {
00090             currUniqueKeyDesc.push_back(bitmapTupleDesc[i]);
00091         }
00092         currUniqueKey.computeAndAllocate(currUniqueKeyDesc);
00093 
00094         // setup violation output
00095         if (outAccessors.size() > 1) {
00096             violationAccessor = outAccessors[1];
00097             violationTuple.compute(violationAccessor->getTupleDesc());
00098         }
00099     }
00100 
00101     createNewIndex = params.createNewIndex;
00102     // If the index is going to be dynamically created, save the original
00103     // root pageId of the index so we can use it later to version the
00104     // index root page.
00105     if (createNewIndex) {
00106         origRootPageId = writeBTreeDesc.rootPageId;
00107     }
00108 }

void LbmSplicerExecStream::open ( bool restart ) [virtual]

Opens this stream, acquiring any resources needed in order to be able to fetch data.

A precondition is that input streams must already be opened. A stream can be closed and reopened.

Parameters:

restart

if true, the stream must be already open, and should reset itself to start from the beginning of its result set

Reimplemented from DiffluenceExecStream.

Definition at line 110 of file LbmSplicerExecStream.cpp.

References bitmapBuffer, bitmapTupleDesc, bTreeWriter, bTreeWriterMoved, createNewIndex, currEntry, currExistingEntry, currValidation, deletionBTreeDesc, deletionReader, deletionTuple, emptyTable, emptyTableUnknown, firstValidation, FixedBuffer, SegmentFactory::getSnapshotSegment(), LbmDeletionIndexReader::init(), isDone, maxEntrySize, mergeBuffer, newIndexCreated, numRowsLoaded, opaqueToInt(), DiffluenceExecStream::open(), DiffluenceExecStream::outAccessors, pCurrentEntry, ExecStream::pDynamicParamManager, pSnapshotSegment, scratchAccessor, uniqueKey, writeBTreeDesc, and writeRowCountParamId.

00111 {
00112     DiffluenceExecStream::open(restart);
00113 
00114     if (!restart) {
00115         bitmapBuffer.reset(new FixedBuffer[maxEntrySize]);
00116         mergeBuffer.reset(new FixedBuffer[maxEntrySize]);
00117         pCurrentEntry = SharedLbmEntry(new LbmEntry());
00118         pCurrentEntry->init(
00119             bitmapBuffer.get(), mergeBuffer.get(), maxEntrySize,
00120             bitmapTupleDesc);
00121 
00122         newIndexCreated = false;
00123         emptyTable = false;
00124         emptyTableUnknown = true;
00125         bTreeWriter = SharedBTreeWriter(
00126             new BTreeWriter(writeBTreeDesc, scratchAccessor, emptyTable));
00127         bTreeWriterMoved = true;
00128 
00129         if (opaqueToInt(writeRowCountParamId) > 0) {
00130             pDynamicParamManager->createParam(
00131                 writeRowCountParamId,
00132                 outAccessors[0]->getTupleDesc()[0]);
00133         }
00134 
00135         if (uniqueKey) {
00136             SharedBTreeReader deletionBTreeReader = SharedBTreeReader(
00137                 new BTreeReader(deletionBTreeDesc));
00138             deletionReader.init(deletionBTreeReader, deletionTuple);
00139         }
00140 
00141         // If the index is going to be dynamically created, the underlying
00142         // segment associated with the index needs to be a snapshot segment.
00143         // Retrieve the snapshot segment.  This needs to be done at open time
00144         // because the segment changes across transaction boundaries.
00145         if (createNewIndex) {
00146             pSnapshotSegment =
00147                 SegmentFactory::getSnapshotSegment(
00148                     writeBTreeDesc.segmentAccessor.pSegment);
00149             assert(pSnapshotSegment != NULL);
00150         }
00151     }
00152     isDone = false;
00153     currEntry = false;
00154     currExistingEntry = false;
00155     numRowsLoaded = 0;
00156 
00157     currValidation = false;
00158     firstValidation = true;
00159 }

ExecStreamResult LbmSplicerExecStream::execute ( ExecStreamQuantum const & quantum ) [virtual]

Executes this stream.

Parameters:

quantum

governs the maximum amount of execution to perform

Returns:: code indicating reason execution ceased

Implements ExecStream.

Definition at line 195 of file LbmSplicerExecStream.cpp.

References bitmapTupleDesc, TupleDatum::cbData, computeRowCount, currEntry, EXECBUF_EOS, EXECRC_BUF_OVERFLOW, EXECRC_EOS, EXECRC_QUANTUM_EXPIRED, EXECRC_YIELD, existingEntry(), findBetterEntry(), getValidatedTuple(), inputTuple, insertBitmapEntry(), insertRowCountParamId, isDone, newIndexCreated, ExecStreamQuantum::nTuplesMax, numRowsLoaded, opaqueToInt(), origRootPageId, DiffluenceExecStream::outAccessors, outputTuple, pCurrentEntry, TupleDatum::pData, ExecStream::pDynamicParamManager, SingleInputExecStream::pInAccessor, pSnapshotSegment, BTreeDescriptor::rootPageId, spliceEntry(), uniqueRequired(), upsertSingleton(), SnapshotRandomAllocationSegment::versionPage(), writeBTreeDesc, and writeRowCountParamId.

00196 {
00197     if (isDone) {
00198         // Version the index roots if the index was dynamically created
00199         if (newIndexCreated) {
00200             pSnapshotSegment->versionPage(
00201                 origRootPageId,
00202                 writeBTreeDesc.rootPageId);
00203         }
00204 
00205         for (uint i = 0; i < outAccessors.size(); i++) {
00206             outAccessors[i]->markEOS();
00207         }
00208         return EXECRC_EOS;
00209     }
00210 
00211     // no more input; write out last bitmap entry and produce final row count
00212     // which is either stored in a dynamic parameter set upstream or is
00213     // computed by splicer
00214 
00215     if (pInAccessor->getState() == EXECBUF_EOS) {
00216         if (currEntry) {
00217             insertBitmapEntry();
00218         }
00219         if (!computeRowCount) {
00220             numRowsLoaded = *reinterpret_cast<RecordNum const *>(
00221                 pDynamicParamManager->getParam(
00222                     insertRowCountParamId).getDatum().pData);
00223         }
00224         if (opaqueToInt(writeRowCountParamId) > 0) {
00225             TupleDatum rowCountDatum;
00226             rowCountDatum.pData = (PConstBuffer) &numRowsLoaded;
00227             rowCountDatum.cbData = sizeof(numRowsLoaded);
00228             pDynamicParamManager->writeParam(
00229                 writeRowCountParamId,
00230                 rowCountDatum);
00231         }
00232         bool rc = outAccessors[0]->produceTuple(outputTuple);
00233         assert(rc);
00234         isDone = true;
00235         return EXECRC_BUF_OVERFLOW;
00236     }
00237 
00238     for (uint i = 0; i < quantum.nTuplesMax; i++) {
00239         ExecStreamResult rc = getValidatedTuple();
00240         if (rc != EXECRC_YIELD) {
00241             return rc;
00242         }
00243 
00244         if (uniqueRequired(inputTuple)) {
00245             if (currEntry) {
00246                 // Write out the current entry before we insert the unique
00247                 // key.
00248                 insertBitmapEntry();
00249                 currEntry = false;
00250             }
00251             upsertSingleton(inputTuple);
00252         } else if (!currEntry) {
00253             // if the key already exists in the index, splice the
00254             // entry just read to the existing btree entry
00255             if (existingEntry(inputTuple)) {
00256                 spliceEntry(inputTuple);
00257             }
00258         } else {
00259             // Compare the key values of the currentEntry with the
00260             // input tuple.  If they're the same, try splicing with
00261             // currentEntry.  Otherwise, write out currentEntry and
00262             // set currentEntry to the new input
00263             int keyComp = pCurrentEntry->compareEntry(
00264                 inputTuple, bitmapTupleDesc);
00265             assert(keyComp <= 0);
00266             if (keyComp == 0) {
00267                 // If we're in the mode where we're splicing in random
00268                 // singleton entries, first make sure there isn't a "better"
00269                 // entry in the btree for the entry we're trying to splice in.
00270                 // A better entry is one whose startRID is closer to the
00271                 // singleton rid we're trying to splice in.
00272                 if (computeRowCount) {
00273                     findBetterEntry(inputTuple);
00274                 }
00275                 spliceEntry(inputTuple);
00276             } else {
00277                 insertBitmapEntry();
00278                 if (existingEntry(inputTuple)) {
00279                     spliceEntry(inputTuple);
00280                 }
00281             }
00282         }
00283         pInAccessor->consumeTuple();
00284     }
00285 
00286     return EXECRC_QUANTUM_EXPIRED;
00287 }

void LbmSplicerExecStream::getResourceRequirements	(	ExecStreamResourceQuantity &	minQuantity,
		ExecStreamResourceQuantity &	optQuantity
	)			`[virtual]`

Reimplemented from ExecStream.

Definition at line 161 of file LbmSplicerExecStream.cpp.

References ExecStream::getResourceRequirements(), ExecStreamResourceQuantity::nCachePages, and uniqueKey.

00164 {
00165     DiffluenceExecStream::getResourceRequirements(minQuantity, optQuantity);
00166 
00167     // btree pages
00168     minQuantity.nCachePages += 5;
00169     if (uniqueKey) {
00170         minQuantity.nCachePages += 5;
00171     }
00172 
00173     optQuantity = minQuantity;
00174 }

void LbmSplicerExecStream::closeImpl ( ) [virtual]

Implements ClosableObject.

ExecStream implementations may override this to release any resources acquired while open.

Reimplemented from ExecStream.

Definition at line 289 of file LbmSplicerExecStream.cpp.

References bitmapBuffer, bTreeWriter, ExecStream::closeImpl(), deletionReader, LbmDeletionIndexReader::endSearch(), mergeBuffer, and pCurrentEntry.

00290 {
00291     if (bTreeWriter) {
00292         bTreeWriter->endSearch();
00293     }
00294     deletionReader.endSearch();
00295     DiffluenceExecStream::closeImpl();
00296     bitmapBuffer.reset();
00297     mergeBuffer.reset();
00298     pCurrentEntry.reset();
00299     bTreeWriter.reset();
00300 }

void DiffluenceExecStream::prepare ( DiffluenceExecStreamParams const & params ) [virtual, inherited]

Definition at line 42 of file DiffluenceExecStream.cpp.

References DiffluenceExecStream::getOutputBufProvision(), DiffluenceExecStream::outAccessors, DiffluenceExecStreamParams::outputTupleDesc, DiffluenceExecStreamParams::outputTupleFormat, SingleInputExecStream::pInAccessor, and SingleInputExecStream::prepare().

Referenced by prepare(), and SegBufferWriterExecStream::prepare().

00043 {
00044     SingleInputExecStream::prepare(params);
00045 
00046     // By default, shape for all outputs is the same as the input if the
00047     // outputTupleDesc wasn't explicitly set.
00048     TupleDescriptor tupleDesc;
00049     TupleFormat tupleFormat;
00050     if (params.outputTupleDesc.empty()) {
00051         tupleDesc = pInAccessor->getTupleDesc();
00052         tupleFormat = pInAccessor->getTupleFormat();
00053     } else {
00054         tupleDesc = params.outputTupleDesc;
00055         tupleFormat = params.outputTupleFormat;
00056     }
00057     for (uint i = 0; i < outAccessors.size(); ++i) {
00058         assert(outAccessors[i]->getProvision() == getOutputBufProvision());
00059         outAccessors[i]->setTupleShape(tupleDesc, tupleFormat);
00060     }
00061 }

void SingleInputExecStream::prepare ( SingleInputExecStreamParams const & params ) [virtual, inherited]

Definition at line 44 of file SingleInputExecStream.cpp.

References SingleInputExecStream::getInputBufProvision(), SingleInputExecStream::pInAccessor, and ExecStream::prepare().

Referenced by JavaSinkExecStream::prepare(), MockConsumerExecStream::prepare(), DiffluenceExecStream::prepare(), and ConduitExecStream::prepare().

00045 {
00046     ExecStream::prepare(params);
00047 
00048     assert(pInAccessor);
00049     assert(pInAccessor->getProvision() == getInputBufProvision());
00050 }

void ExecStream::prepare ( ExecStreamParams const & params ) [virtual, inherited]

Prepares this stream for execution.

A precondition is that input streams must already be defined and prepared. As an effect of this call, the tuple shape should be defined for all output buffers and remain unchanged for the lifetime of the stream. This method is only ever called once, before the first open. Although this method is virtual, derived classes may choose to define an overloaded version instead with a specialized covariant parameter class.

Parameters:

params

instance of stream parameterization class which should be used to prepare this stream

Definition at line 84 of file ExecStream.cpp.

References ExecStreamGraph::getDynamicParamManager(), SegmentAccessor::pCacheAccessor, ExecStreamParams::pCacheAccessor, ExecStream::pDynamicParamManager, ExecStream::pGraph, ExecStream::pQuotaAccessor, ExecStream::pScratchQuotaAccessor, and ExecStreamParams::scratchAccessor.

Referenced by JavaTransformExecStream::prepare(), SingleOutputExecStream::prepare(), and SingleInputExecStream::prepare().

00085 {
00086     if (pGraph) {
00087         pDynamicParamManager = pGraph->getDynamicParamManager();
00088     }
00089     pQuotaAccessor = params.pCacheAccessor;
00090     pScratchQuotaAccessor = params.scratchAccessor.pCacheAccessor;
00091 }

void DiffluenceExecStream::setOutputBufAccessors ( std::vector< SharedExecStreamBufAccessor > const & outAccessors ) [virtual, inherited]

Initializes the buffer accessors for outputs from this stream.

This method is only ever called once, before prepare.

Parameters:

outAccessors

buffer accessors ordered by output stream

Reimplemented from SingleInputExecStream.

Definition at line 36 of file DiffluenceExecStream.cpp.

References DiffluenceExecStream::outAccessors.

00038 {
00039     outAccessors = outAccessorsInit;
00040 }

ExecStreamBufProvision DiffluenceExecStream::getOutputBufProvision ( ) const [virtual, inherited]

Indicate to the consumer if the buffer is provided by this exec stream which is the producer.

Reimplemented from ExecStream.

Reimplemented in SegBufferWriterExecStream, and SplitterExecStream.

Definition at line 75 of file DiffluenceExecStream.cpp.

References BUFPROV_CONSUMER.

Referenced by DiffluenceExecStream::prepare().

00076 {
00077     /*
00078      * Indicate to the consumer that buffer should be provided by the consumer.
00079      * By default, DiffluenceExecStream does not have any associated buffers.
00080      */
00081     return BUFPROV_CONSUMER;
00082 }

void SingleInputExecStream::setInputBufAccessors ( std::vector< SharedExecStreamBufAccessor > const & inAccessors ) [virtual, inherited]

Initializes the buffer accessors for inputs to this stream.

This method is only ever called once, before prepare.

Parameters:

inAccessors

buffer accessors ordered by input stream

Implements ExecStream.

Reimplemented in ConduitExecStream.

Definition at line 37 of file SingleInputExecStream.cpp.

References SingleInputExecStream::pInAccessor.

Referenced by ConduitExecStream::setInputBufAccessors().

00039 {
00040     assert(inAccessors.size() == 1);
00041     pInAccessor = inAccessors[0];
00042 }

ExecStreamBufProvision SingleInputExecStream::getInputBufProvision ( ) const [virtual, inherited]

Queries the BufferProvision which this stream requires of its inputs when consuming their tuples.

Returns:: required model; default is BUFPROV_NONE

Reimplemented from ExecStream.

Reimplemented in DoubleBufferExecStream, ScratchBufferExecStream, SegBufferExecStream, and SegBufferWriterExecStream.

Definition at line 62 of file SingleInputExecStream.cpp.

References BUFPROV_PRODUCER.

Referenced by SingleInputExecStream::prepare().

00063 {
00064     return BUFPROV_PRODUCER;
00065 }

bool ExecStream::canEarlyClose ( ) [virtual, inherited]

Returns:: true if the stream can be closed early

Reimplemented in SegBufferWriterExecStream.

Definition at line 49 of file ExecStream.cpp.

00050 {
00051     return true;
00052 }

ExecStreamGraph & ExecStream::getGraph ( ) const [inline, inherited]

Returns:: reference to containing graph

Definition at line 293 of file ExecStream.h.

References ExecStream::pGraph.

Referenced by ExternalSortExecStreamImpl::execute(), JavaSinkExecStream::execute(), SegBufferWriterExecStream::execute(), SegBufferExecStream::execute(), BarrierExecStream::execute(), Java_net_sf_farrago_fennel_FennelStorage_tupleStreamFetch(), Java_net_sf_farrago_fennel_FennelStorage_tupleStreamTransformFetch(), JavaSinkExecStream::stuffByteBuffer(), and ExecStreamScheduler::traceStreamBuffers().

00294 {
00295     assert(pGraph);
00296     return *pGraph;
00297 }

ExecStreamId ExecStream::getStreamId ( ) const [inline, inherited]

Returns:: the identifier for this stream within containing graph

Definition at line 288 of file ExecStream.h.

References ExecStream::id.

Referenced by ExternalSortExecStreamImpl::execute(), SegBufferWriterExecStream::execute(), SegBufferExecStream::execute(), CorrelationJoinExecStream::execute(), BarrierExecStream::execute(), Java_net_sf_farrago_fennel_FennelStorage_tupleStreamTransformFetch(), JavaTransformExecStream::open(), SingleInputExecStream::open(), ConfluenceExecStream::open(), CartesianJoinExecStream::prepare(), ParallelExecStreamScheduler::readStream(), DfsTreeExecStreamScheduler::readStream(), LbmMinusExecStream::restartSubtrahends(), ExecStreamScheduler::tracePostExecution(), ExecStreamScheduler::tracePreExecution(), ExecStreamScheduler::traceStreamBuffers(), and ParallelExecStreamScheduler::tryExecuteTask().

00289 {
00290     return id;
00291 }

void ExecStream::getResourceRequirements	(	ExecStreamResourceQuantity &	minQuantity,
		ExecStreamResourceQuantity &	optQuantity,
		ExecStreamResourceSettingType &	optType
	)			`[virtual, inherited]`

Determines resource requirements for this stream.

Default implementation declares zero resource requirements.

Parameters:

	minQuantity	receives the minimum resource quantity needed by this stream in order to execute
	optQuantity	receives the resource quantity needed by this stream in order to execute optimally
	optType	Receives the value indicating the accuracy of the optQuantity parameter. This parameter is optional and defaults to EXEC_RESOURCE_ACCURATE if omitted. If the optimum setting is an estimate or no value can be specified (e.g., due to lack of statistics), then this parameter needs to be used to indicate a non-accurate optimum resource setting.

Reimplemented in MockResourceExecStream, BTreePrefetchSearchExecStream, LhxAggExecStream, LhxJoinExecStream, LbmGeneratorExecStream, LbmUnionExecStream, and ExternalSortExecStreamImpl.

Definition at line 93 of file ExecStream.cpp.

References EXEC_RESOURCE_ACCURATE.

00097 {
00098     getResourceRequirements(minQuantity, optQuantity);
00099     optType = EXEC_RESOURCE_ACCURATE;
00100 }

void ExecStream::setResourceAllocation ( ExecStreamResourceQuantity & quantity ) [virtual, inherited]

Sets current resource allocation for this stream.

If called while the stream is open, this indicates a request for the stream to dynamically adjust its memory usage. If the stream is incapable of honoring the request, it should update quantity with the actual amounts still in use.

Parameters:

quantity

allocated resource quantity

Reimplemented in MockResourceExecStream, BTreePrefetchSearchExecStream, LhxAggExecStream, LhxJoinExecStream, LbmGeneratorExecStream, LbmUnionExecStream, and ExternalSortExecStreamImpl.

Definition at line 111 of file ExecStream.cpp.

References ExecStreamResourceQuantity::nCachePages, ExecStream::pQuotaAccessor, ExecStream::pScratchQuotaAccessor, and ExecStream::resourceAllocation.

Referenced by ExternalSortExecStreamImpl::setResourceAllocation(), LbmUnionExecStream::setResourceAllocation(), LbmGeneratorExecStream::setResourceAllocation(), LhxJoinExecStream::setResourceAllocation(), LhxAggExecStream::setResourceAllocation(), and BTreePrefetchSearchExecStream::setResourceAllocation().

00113 {
00114     resourceAllocation = quantity;
00115     if (pQuotaAccessor) {
00116         pQuotaAccessor->setMaxLockedPages(quantity.nCachePages);
00117     }
00118     if (pScratchQuotaAccessor) {
00119         pScratchQuotaAccessor->setMaxLockedPages(quantity.nCachePages);
00120     }
00121 }

void ExecStream::setName ( std::string const & ) [virtual, inherited]

Sets unique name of this stream.

Definition at line 157 of file ExecStream.cpp.

References ExecStream::name.

00158 {
00159     name = nameInit;
00160 }

std::string const & ExecStream::getName ( ) const [virtual, inherited]

Returns:: the name of this stream, as known by the optimizer

Definition at line 162 of file ExecStream.cpp.

References ExecStream::name.

Referenced by DfsTreeExecStreamScheduler::findNextConsumer(), ParallelExecStreamScheduler::readStream(), DfsTreeExecStreamScheduler::readStream(), ExecStreamScheduler::tracePostExecution(), and ExecStreamScheduler::tracePreExecution().

00163 {
00164     return name;
00165 }

bool ExecStream::mayBlock ( ) const [virtual, inherited]

Queries whether this stream's implementation may block when execute() is called.

For accurate scheduling, non-blocking implementations are preferred; the scheduler must be aware of the potential for blocking so that it can allocate extra threads accordingly.

Returns:: whether stream may block; default is false

Definition at line 167 of file ExecStream.cpp.

00168 {
00169     return false;
00170 }

void ExecStream::checkAbort ( ) const [virtual, inherited]

Checks whether there is an abort request for this stream's scheduler.

Normally, streams don't need to check this, since the scheduler services abort requests in between quanta. However, streams which enter long-running loops need to check for themselves. If an abort is scheduled, this method will throw an AbortExcn automatically.

Definition at line 72 of file ExecStream.cpp.

References ExecStreamScheduler::checkAbort(), ExecStreamGraph::getScheduler(), and ExecStream::pGraph.

Referenced by LhxJoinExecStream::execute(), LhxAggExecStream::execute(), ExternalSortRunAccessor::fetch(), and ExternalSortMerger::fetch().

00073 {
00074     if (!pGraph) {
00075         return;
00076     }
00077     ExecStreamScheduler *pScheduler = pGraph->getScheduler();
00078     if (!pScheduler) {
00079         return;
00080     }
00081     pScheduler->checkAbort();
00082 }

ExecStreamBufProvision ExecStream::getOutputBufConversion ( ) const [virtual, inherited]

Queries the BufferProvision to which this stream needs its output to be converted, if any.

Returns:: required conversion; default is BUFPROV_NONE

Reimplemented in JavaTransformExecStream.

Definition at line 177 of file ExecStream.cpp.

References BUFPROV_NONE.

00178 {
00179     return BUFPROV_NONE;
00180 }

bool ClosableObject::isClosed ( ) const [inline, inherited]

Returns:: whether the object has been closed

Definition at line 58 of file ClosableObject.h.

00059     {
00060         return !needsClose;
00061     }

void ClosableObject::close ( ) [inherited]

Closes this object, releasing any unallocated resources.

Reimplemented in CollectExecStream, CorrelationJoinExecStream, LcsClusterAppendExecStream, and LcsClusterReplaceExecStream.

Definition at line 39 of file ClosableObject.cpp.

References ClosableObject::closeImpl(), and ClosableObject::needsClose.

Referenced by CacheImpl< PageT, VictimPolicyT >::allocatePages(), LcsRowScanBaseExecStream::closeImpl(), ExecStreamGraphImpl::closeImpl(), FlatFileBuffer::open(), ClosableObjectDestructor::operator()(), and Segment::~Segment().

00040 {
00041     if (!needsClose) {
00042         return;
00043     }
00044     needsClose = false;
00045     closeImpl();
00046 }

void TraceSource::initTraceSource	(	SharedTraceTarget	pTraceTarget,
		std::string	name
	)			`[virtual, inherited]`

For use when initialization has to be deferred until after construction.

Parameters:

	pTraceTarget	the TraceTarget to which messages will be sent
	name	the name of this source

Definition at line 46 of file TraceSource.cpp.

References TraceSource::isTracing(), TraceSource::minimumLevel, TraceSource::name, TraceSource::pTraceTarget, and TRACE_OFF.

Referenced by TestBase::beforeTestCase(), TestBase::TestBase(), and TraceSource::TraceSource().

00049 {
00050     assert(!pTraceTarget.get());
00051 
00052     pTraceTarget = pTraceTargetInit;
00053     name = nameInit;
00054     if (isTracing()) {
00055         minimumLevel = pTraceTarget->getSourceTraceLevel(name);
00056     } else {
00057         minimumLevel = TRACE_OFF;
00058     }
00059 }

void TraceSource::trace	(	TraceLevel	level,
		std::string	message
	)			const `[inherited]`

Records a trace message.

Normally only called via FENNEL_TRACE.

Parameters:

	level	severity level of event being trace
	message	the text of the message

Definition at line 61 of file TraceSource.cpp.

References TraceSource::getTraceTarget(), TraceSource::isTracing(), TraceSource::name, and TraceTarget::notifyTrace().

Referenced by Calculator::exec(), and ExecStreamScheduler::traceStreamBufferContents().

00062 {
00063     if (isTracing()) {
00064         getTraceTarget().notifyTrace(name,level,message);
00065     }
00066 }

bool TraceSource::isTracing ( ) const [inline, inherited]

Returns:: true iff tracing is enabled for this source

Definition at line 88 of file TraceSource.h.

Referenced by TraceSource::initTraceSource(), CalcExecStream::prepare(), and TraceSource::trace().

00089     {
00090         return pTraceTarget.get() ? true : false;
00091     }

bool TraceSource::isTracingLevel ( TraceLevel level ) const [inline, inherited]

Determines whether a particular level is being traced.

Parameters:

level

trace level to test

Returns:: true iff tracing is enabled for the given level

Definition at line 100 of file TraceSource.h.

Referenced by ExecStreamScheduler::addGraph(), SimpleExecStreamGovernor::assignCachePages(), SimpleExecStreamGovernor::distributeCachePages(), Calculator::exec(), ExecStreamScheduler::ExecStreamScheduler(), LcsClusterNodeWriter::getLastClusterPageForWrite(), LcsClusterNodeWriter::moveFromTempToIndex(), JavaSinkExecStream::stuffByteBuffer(), and ExecStreamScheduler::traceStreamBuffers().

00101     {
00102         return level >= minimumLevel;
00103     }

TraceTarget& TraceSource::getTraceTarget ( ) const [inline, inherited]

Returns:: the TraceTarget for this source

Definition at line 108 of file TraceSource.h.

Referenced by TraceSource::trace().

00109     {
00110         assert(isTracing());
00111         return *(pTraceTarget.get());
00112     }

SharedTraceTarget TraceSource::getSharedTraceTarget ( ) const [inline, inherited]

Returns:: the SharedTraceTarget for this source

Definition at line 117 of file TraceSource.h.

Referenced by Database::init(), LcsClusterAppendExecStream::initLoad(), and CalcExecStream::prepare().

00118     {
00119         return pTraceTarget;
00120     }

std::string TraceSource::getTraceSourceName ( ) const [inline, inherited]

Gets the name of this source.

Useful to construct nested names for subcomponents that are also TraceSources.

Returns:: the name

Definition at line 127 of file TraceSource.h.

Referenced by LcsClusterAppendExecStream::initLoad().

00128     {
00129         return name;
00130     }

void TraceSource::setTraceSourceName ( std::string const & n ) [inline, inherited]

Sets the name of this source.

Useful to construct dynamic names for fine-grained filtering.

Definition at line 136 of file TraceSource.h.

00137     {
00138         name = n;
00139     }

TraceLevel TraceSource::getMinimumTraceLevel ( ) const [inline, inherited]

Definition at line 141 of file TraceSource.h.

00142     {
00143         return minimumLevel;
00144     }

void TraceSource::disableTracing ( ) [inherited]

Definition at line 68 of file TraceSource.cpp.

References TraceSource::minimumLevel, TraceSource::pTraceTarget, and TRACE_OFF.

Referenced by TestBase::afterTestCase().

00069 {
00070     pTraceTarget.reset();
00071     minimumLevel = TRACE_OFF;
00072 }

void ErrorSource::initErrorSource	(	SharedErrorTarget	pErrorTarget,
		const std::string &	name
	)			`[virtual, inherited]`

For use when initialization has to be deferred until after construction.

Parameters:

	pErrorTarget	the ErrorTarget to which errors will be posted
	name	the name of this source

Definition at line 47 of file ErrorSource.cpp.

References ErrorSource::name, and ErrorSource::pErrorTarget.

Referenced by ErrorSource::ErrorSource().

00050 {
00051     pErrorTarget = pErrorTargetInit;
00052     name = nameInit;
00053 }

void ErrorSource::postError	(	ErrorLevel	level,
		const std::string &	message,
		void *	address,
		long	capacity,
		int	index
	)			`[inherited]`

Posts an exception, such as a row exception.

See also:: ErrorTarget for a description of the parameters

Definition at line 55 of file ErrorSource.cpp.

References ErrorSource::getErrorTarget(), ErrorSource::hasTarget(), ErrorSource::name, and ErrorTarget::notifyError().

Referenced by FlatFileExecStreamImpl::logError(), ErrorSource::postError(), and postViolation().

00058 {
00059     if (hasTarget()) {
00060         getErrorTarget().notifyError(
00061             name, level, message, address, capacity, index);
00062     }
00063 }

void ErrorSource::postError	(	ErrorLevel	level,
		const std::string &	message,
		const TupleDescriptor &	errorDesc,
		const TupleData &	errorTuple,
		int	index
	)			`[inherited]`

Posts an exception, such as a row exception.

See also:: ErrorTarget for a description of the parameters

Definition at line 65 of file ErrorSource.cpp.

References TupleAccessor::compute(), ErrorSource::errorAccessor, FixedBuffer, TupleAccessor::getByteCount(), TupleAccessor::getMaxByteCount(), ErrorSource::hasTarget(), TupleAccessor::marshal(), ErrorSource::pErrorBuf, and ErrorSource::postError().

00068 {
00069     if (!hasTarget()) {
00070         return;
00071     }
00072 
00073     if (!pErrorBuf) {
00074         errorAccessor.compute(errorDesc);
00075         uint cbMax = errorAccessor.getMaxByteCount();
00076         pErrorBuf.reset(new FixedBuffer[cbMax]);
00077     }
00078 
00079     uint cbTuple = errorAccessor.getByteCount(errorTuple);
00080     errorAccessor.marshal(errorTuple, pErrorBuf.get());
00081     postError(level, message, pErrorBuf.get(), cbTuple, index);
00082 }

bool ErrorSource::hasTarget ( ) const [inline, inherited]

Returns:: true iff an error target has been set

Definition at line 112 of file ErrorSource.h.

Referenced by ErrorSource::postError().

00113     {
00114         return pErrorTarget.get() ? true : false;
00115     }

ErrorTarget& ErrorSource::getErrorTarget ( ) const [inline, inherited]

Returns:: the ErrorTarget for this source

Definition at line 120 of file ErrorSource.h.

Referenced by ErrorSource::postError().

00121     {
00122         assert(hasTarget());
00123         return *(pErrorTarget.get());
00124     }

SharedErrorTarget ErrorSource::getSharedErrorTarget ( ) const [inline, inherited]

Returns:: the SharedErrorTarget for this source

Definition at line 129 of file ErrorSource.h.

00130     {
00131         return pErrorTarget;
00132     }

std::string ErrorSource::getErrorSourceName ( ) const [inline, inherited]

Gets the name of this source.

Useful to construct nested names for subcomponents that are also ErrorSources.

Returns:: the name

Definition at line 139 of file ErrorSource.h.

00140     {
00141         return name;
00142     }

void ErrorSource::setErrorSourceName ( std::string const & n ) [inline, inherited]

Sets the name of this source.

Useful to construct dynamic names for fine-grained filtering.

Definition at line 148 of file ErrorSource.h.

00149     {
00150         name = n;
00151     }

void ErrorSource::disableTarget ( ) [inherited]

Definition at line 84 of file ErrorSource.cpp.

References ErrorSource::pErrorTarget.

00085 {
00086     pErrorTarget.reset();
00087 }

Member Data Documentation

SegmentAccessor LbmSplicerExecStream::scratchAccessor [private]

Scratch accessor.

Definition at line 84 of file LbmSplicerExecStream.h.

Referenced by getValidatedTuple(), isEmpty(), open(), and prepare().

BTreeDescriptor LbmSplicerExecStream::writeBTreeDesc [private]

Descriptor corresponding to the btree that splicer writes to.

Definition at line 89 of file LbmSplicerExecStream.h.

Referenced by execute(), getValidatedTuple(), isEmpty(), open(), and prepare().

BTreeDescriptor LbmSplicerExecStream::deletionBTreeDesc [private]

Descriptor corresponding to the deletion index btree.

Definition at line 94 of file LbmSplicerExecStream.h.

Referenced by open(), and prepare().

DynamicParamId LbmSplicerExecStream::insertRowCountParamId [private]

Parameter id of dynamic parameter containing final insert row count.

Definition at line 99 of file LbmSplicerExecStream.h.

Referenced by execute(), and prepare().

bool LbmSplicerExecStream::createNewIndex [private]

If true, create a new index that the splicer will be writing.

Definition at line 104 of file LbmSplicerExecStream.h.

Referenced by getValidatedTuple(), open(), and prepare().

bool LbmSplicerExecStream::newIndexCreated [private]

True if a new index was created.

This will only be true if createNewIndex is true and there is at least one input tuple, indicating that the existing index needs to be recreated and the new one versioned off of the original.

Definition at line 112 of file LbmSplicerExecStream.h.

Referenced by execute(), getValidatedTuple(), and open().

DynamicParamId LbmSplicerExecStream::writeRowCountParamId [private]

Parameter id of the dynamic parameter used to write the row count affected by this stream that will be read downstream.

Definition at line 118 of file LbmSplicerExecStream.h.

Referenced by execute(), open(), and prepare().

uint LbmSplicerExecStream::maxEntrySize [private]

Maximum size of an LbmEntry.

Definition at line 123 of file LbmSplicerExecStream.h.

Referenced by open(), and prepare().

boost::scoped_array<FixedBuffer> LbmSplicerExecStream::bitmapBuffer [private]

Buffer for LbmEntry.

Definition at line 128 of file LbmSplicerExecStream.h.

Referenced by closeImpl(), and open().

boost::scoped_array<FixedBuffer> LbmSplicerExecStream::mergeBuffer [private]

Buffer for merges on LbmEntry.

Definition at line 133 of file LbmSplicerExecStream.h.

Referenced by closeImpl(), and open().

SharedLbmEntry LbmSplicerExecStream::pCurrentEntry [private]

Current bitmap entry under construction.

Definition at line 138 of file LbmSplicerExecStream.h.

Referenced by closeImpl(), createNewBitmapEntry(), execute(), findBetterEntry(), insertBitmapEntry(), open(), and spliceEntry().

bool LbmSplicerExecStream::currEntry [private]

true if current entry is under construction

Definition at line 143 of file LbmSplicerExecStream.h.

Referenced by createNewBitmapEntry(), execute(), and open().

bool LbmSplicerExecStream::currExistingEntry [private]

True if current bitmap entry under construction refers to an already existing entry in the bitmap index.

Definition at line 149 of file LbmSplicerExecStream.h.

Referenced by existingEntry(), findBetterEntry(), insertBitmapEntry(), and open().

LcsRid LbmSplicerExecStream::currBTreeStartRid [private]

Start rid of the current existing entry in the bitmap index.

Definition at line 154 of file LbmSplicerExecStream.h.

Referenced by createNewBitmapEntry(), and insertBitmapEntry().

bool LbmSplicerExecStream::emptyTableUnknown [private]

True if we need to determine emptyTable.

We can't do this in init() because upstream ExecStream's may insert into the index being updated. An example of early update is when the deletion phase of a merge statement appends the deletion index. The same index may be later is appended for merge violations.

Definition at line 163 of file LbmSplicerExecStream.h.

Referenced by getValidatedTuple(), isEmpty(), and open().

bool LbmSplicerExecStream::emptyTable [private]

True if table that the bitmap is being constructed on was empty to begin with.

Definition at line 169 of file LbmSplicerExecStream.h.

Referenced by getValidatedTuple(), isEmpty(), and open().

SharedBTreeWriter LbmSplicerExecStream::bTreeWriter [private]

Writes btree index corresponding to bitmap.

Definition at line 174 of file LbmSplicerExecStream.h.

Referenced by closeImpl(), findBTreeEntry(), findMatchingBTreeEntry(), getValidatedTuple(), insertBitmapEntry(), isEmpty(), open(), and upsertSingleton().

bool LbmSplicerExecStream::bTreeWriterMoved [private]

Whether btree writer position was moved for unique constraint validation.

If currExistingEntry is true, then the btree writer is expected to stay positioned at the existing entry. However, constraint validation may run side searches and can reposition the btree writer.

Definition at line 182 of file LbmSplicerExecStream.h.

Referenced by existingEntry(), insertBitmapEntry(), and open().

bool LbmSplicerExecStream::isDone [private]

True if output has been produced.

Definition at line 187 of file LbmSplicerExecStream.h.

Referenced by execute(), and open().

TupleData LbmSplicerExecStream::inputTuple [private]

Input tuple.

Definition at line 192 of file LbmSplicerExecStream.h.

Referenced by countKeyRows(), execute(), getValidatedTuple(), and prepare().

TupleData LbmSplicerExecStream::singletonTuple [private]

Input tuple corresponding to singleton entries.

Used only in the case where the rowcount needs to be computed by splicer.

Definition at line 198 of file LbmSplicerExecStream.h.

Referenced by getValidatedTuple(), and prepare().

TupleData LbmSplicerExecStream::outputTuple [private]

Output tuple containing rowcount.

Definition at line 203 of file LbmSplicerExecStream.h.

Referenced by execute(), and prepare().

RecordNum LbmSplicerExecStream::numRowsLoaded [private]

Number of rows loaded into bitmap index.

Definition at line 208 of file LbmSplicerExecStream.h.

Referenced by execute(), getValidatedTuple(), open(), and prepare().

TupleData LbmSplicerExecStream::bTreeTupleData [private]

Tuple data for reading bitmaps from btree index.

Definition at line 213 of file LbmSplicerExecStream.h.

Referenced by countKeyRows(), existingEntry(), findBetterEntry(), findBTreeEntry(), findMatchingBTreeEntry(), insertBitmapEntry(), prepare(), and upsertSingleton().

TupleData LbmSplicerExecStream::tempBTreeTupleData [private]

Definition at line 214 of file LbmSplicerExecStream.h.

Referenced by insertBitmapEntry(), and prepare().

TupleDescriptor LbmSplicerExecStream::bitmapTupleDesc [private]

Tuple descriptor representing bitmap tuple.

Definition at line 219 of file LbmSplicerExecStream.h.

Referenced by execute(), findBTreeEntry(), getValidatedTuple(), open(), postViolation(), and prepare().

uint LbmSplicerExecStream::nIdxKeys [private]

Number of keys in the bitmap index, excluding the starting rid.

Definition at line 224 of file LbmSplicerExecStream.h.

Referenced by createNewBitmapEntry(), findBTreeEntry(), getValidatedTuple(), insertBitmapEntry(), postViolation(), prepare(), and uniqueRequired().

bool LbmSplicerExecStream::computeRowCount [private]

True if splicer needs to compute the row count result rather than reading it from a dynamic parameter.

Definition at line 230 of file LbmSplicerExecStream.h.

Referenced by execute(), findBetterEntry(), getValidatedTuple(), insertBitmapEntry(), and prepare().

bool LbmSplicerExecStream::uniqueKey [private]

Whether the index being updated is a unique key.

Definition at line 235 of file LbmSplicerExecStream.h.

Referenced by countKeyRows(), getResourceRequirements(), open(), prepare(), and uniqueRequired().

bool LbmSplicerExecStream::currValidation [private]

Whether an input tuple is currently being validated.

Definition at line 240 of file LbmSplicerExecStream.h.

Referenced by getValidatedTuple(), and open().

bool LbmSplicerExecStream::firstValidation [private]

Whether any tuple has been validated yet.

Definition at line 245 of file LbmSplicerExecStream.h.

Referenced by getValidatedTuple(), and open().

TupleDataWithBuffer LbmSplicerExecStream::currUniqueKey [private]

The current unique key value being validated.

Definition at line 250 of file LbmSplicerExecStream.h.

Referenced by getValidatedTuple(), and prepare().

uint LbmSplicerExecStream::nKeyRows [private]

Number of rows for the current key value (after taking into account the deletion index).

Definition at line 256 of file LbmSplicerExecStream.h.

Referenced by getValidatedTuple().

LbmDeletionIndexReader LbmSplicerExecStream::deletionReader [private]

Reads rids from the deletion index.

Definition at line 261 of file LbmSplicerExecStream.h.

Referenced by closeImpl(), countKeyRows(), getValidatedTuple(), and open().

TupleData LbmSplicerExecStream::deletionTuple [private]

Current tuple for deletion index rid reader.

Definition at line 266 of file LbmSplicerExecStream.h.

Referenced by open(), and prepare().

LbmTupleRidReader LbmSplicerExecStream::inputRidReader [private]

Reads rids from an input tuple.

Definition at line 271 of file LbmSplicerExecStream.h.

Referenced by getValidatedTuple().

bool LbmSplicerExecStream::nullUpsertRid [private]

True if no RID has been accepted as the update/insert for the current key value.

Definition at line 277 of file LbmSplicerExecStream.h.

Referenced by getUpsertRidPtr(), getValidatedTuple(), and setUpsertRid().

LcsRid LbmSplicerExecStream::upsertRid [private]

If a RID has been accepted as the update/insert for the current key value, this contains the value of the accepted RID.

Definition at line 283 of file LbmSplicerExecStream.h.

Referenced by getUpsertRidPtr(), and setUpsertRid().

SharedExecStreamBufAccessor LbmSplicerExecStream::violationAccessor [private]

Accessor for the violation output stream.

Definition at line 288 of file LbmSplicerExecStream.h.

Referenced by getValidatedTuple(), and prepare().

TupleData LbmSplicerExecStream::violationTuple [private]

Violation data tuple.

Definition at line 293 of file LbmSplicerExecStream.h.

Referenced by getValidatedTuple(), and prepare().

std::string LbmSplicerExecStream::errorMsg [private]

Error message for constraint violations.

Definition at line 298 of file LbmSplicerExecStream.h.

Referenced by postViolation().

TupleDescriptor LbmSplicerExecStream::errorDesc [private]

TupleDescriptor for error records.

Definition at line 303 of file LbmSplicerExecStream.h.

Referenced by postViolation().

TupleData LbmSplicerExecStream::errorTuple [private]

TupleData used to build error records.

Definition at line 308 of file LbmSplicerExecStream.h.

Referenced by postViolation().

SnapshotRandomAllocationSegment* LbmSplicerExecStream::pSnapshotSegment [private]

Underlying snapshot segment corresponding to the index that will be written.

Only used when a new index is dynamically created.

Definition at line 314 of file LbmSplicerExecStream.h.

Referenced by execute(), and open().

PageId LbmSplicerExecStream::origRootPageId [private]

The original root pageId of the index that will be written.

Only used when a new index is dynamically created.

Definition at line 320 of file LbmSplicerExecStream.h.

Referenced by execute(), and prepare().

std::vector<SharedExecStreamBufAccessor> DiffluenceExecStream::outAccessors [protected, inherited]

List of output buffer accessors.

Definition at line 63 of file DiffluenceExecStream.h.

Referenced by execute(), SplitterExecStream::execute(), SegBufferWriterExecStream::execute(), open(), DiffluenceExecStream::open(), prepare(), SegBufferWriterExecStream::prepare(), DiffluenceExecStream::prepare(), and DiffluenceExecStream::setOutputBufAccessors().

TupleDescriptor DiffluenceExecStream::outputTupleDesc [protected, inherited]

Output tuple descriptor.

Currently, all outputs must have the same descriptor.

Definition at line 69 of file DiffluenceExecStream.h.

SharedExecStreamBufAccessor SingleInputExecStream::pInAccessor [protected, inherited]

Definition at line 51 of file SingleInputExecStream.h.

bool ExecStream::isOpen [protected, inherited]

Whether this stream is currently open.

Note that this is not quite the opposite of the inherited ClosableObject.needsClose, since a stream needs to be closed before destruction if it has been prepared but never opened.

Definition at line 61 of file ExecStream.h.

Referenced by ExecStream::closeImpl(), ExecStream::ExecStream(), and ExecStream::open().

ExecStreamGraph* ExecStream::pGraph [protected, inherited]

Dataflow graph containing this stream.

Note that we don't use a weak_ptr for this because it needs to be accessed frequently during execution, and the extra locking overhead would be frivolous.

Definition at line 68 of file ExecStream.h.

Referenced by ExecStream::checkAbort(), ExecStream::ExecStream(), CorrelationJoinExecStream::execute(), ExecStream::getGraph(), JavaTransformExecStream::open(), SingleInputExecStream::open(), MergeExecStream::open(), ExecStream::open(), CorrelationJoinExecStream::open(), ConfluenceExecStream::open(), ExecStream::prepare(), CartesianJoinExecStream::prepare(), and LbmMinusExecStream::restartSubtrahends().

ExecStreamId ExecStream::id [protected, inherited]

Identifier for this stream; local to its containing graph.

Definition at line 73 of file ExecStream.h.

Referenced by ExecStream::getStreamId().

std::string ExecStream::name [protected, inherited]

Name of stream, as known by optimizer.

Reimplemented from TraceSource.

Definition at line 78 of file ExecStream.h.

Referenced by ExecStream::ExecStream(), ExecStream::getName(), FlatFileExecStreamImpl::open(), and ExecStream::setName().

SharedDynamicParamManager ExecStream::pDynamicParamManager [protected, inherited]

The dynamic parameter manager available to this stream.

(Obtained at prepare() time. Keep a shared pointer in case the stream is reassigned to another graph for execution; cf ExecStreamGraph::mergeFrom())

Definition at line 85 of file ExecStream.h.

Referenced by CorrelationJoinExecStream::close(), SegBufferReaderExecStream::closeImpl(), LbmUnionExecStream::execute(), execute(), LbmGeneratorExecStream::execute(), LbmChopperExecStream::execute(), CorrelationJoinExecStream::execute(), BarrierExecStream::execute(), LcsClusterReplaceExecStream::open(), LbmUnionExecStream::open(), open(), LbmGeneratorExecStream::open(), LbmBitOpExecStream::open(), BTreeSearchExecStream::open(), BTreeInsertExecStream::open(), SegBufferWriterExecStream::open(), SegBufferReaderExecStream::open(), NestedLoopJoinExecStream::open(), CorrelationJoinExecStream::open(), ExecStream::prepare(), CalcExecStream::prepare(), NestedLoopJoinExecStream::processLeftInput(), LbmSearchExecStream::reachedTupleLimit(), ReshapeExecStream::readDynamicParams(), SegBufferWriterExecStream::readReaderRefCount(), BTreeSearchExecStream::readSearchKey(), BTreeSearchExecStream::readUpperBoundKey(), LbmSearchExecStream::setAdditionalKeys(), and LbmBitOpExecStream::writeStartRidParamValue().

SharedLogicalTxn ExecStream::pTxn [protected, inherited]

The transaction embracing the stream.

Obtained at open() time; but not released at close() time, to allow TableWriters to replay a txn. Keep a shared pointer in case the stream is reassigned to another graph for execution; cf ExecStreamGraph::mergeFrom())

Definition at line 94 of file ExecStream.h.

Referenced by FtrsTableWriterExecStream::commitSavepoint(), FtrsTableWriterExecStream::createSavepoint(), FtrsTableWriterExecStream::open(), ExecStream::open(), and FtrsTableWriterExecStream::rollbackSavepoint().