Fennel: LbmIntersectExecStream Class Reference

bool LbmIntersectExecStream::intersectSegments ( uint len ) [private]

Performs intersect operation on all segments.

Parameters:

len

length of intersecting segments

Returns:: false if buffer overflow occurred writing out a segment

Definition at line 100 of file LbmIntersectExecStream.cpp.

References LbmBitOpExecStream::addByteSeg, LbmBitOpExecStream::addLen, LbmBitOpExecStream::addRid, LbmBitOpExecStream::addSegments(), LbmSegment::LbmOneByteSize, LbmBitOpExecStream::nInputs, LbmBitOpExecStream::pByteSegBuf, LbmBitOpExecStream::segmentReaders, LbmBitOpExecStream::startRid, and LbmBitOpExecStream::writeStartRidParamValue().

Referenced by execute().

00101 {
00102     LcsRid currRid;
00103     PBuffer currByteSeg;
00104     uint currLen;
00105 
00106     // initialize temporary buffer with all 1's
00107     for (uint i = 0; i < len; i++) {
00108         pByteSegBuf[i] = 0xff;
00109     }
00110 
00111     // retrieve each current segment and perform the AND operation
00112     for (uint i = 0; i < nInputs; i++) {
00113         segmentReaders[i].readCurrentByteSegment(
00114             currRid, currByteSeg, currLen);
00115         if (i == 0) {
00116             addRid = currRid;
00117         } else {
00118             permAssert(addRid == currRid);
00119         }
00120         // byte segments are stored in reverse order, so currByteSeg points
00121         // to the end of the buffer whereas pByteSegBuf points to the
00122         // beginning but needs to be filled in backwards
00123         for (int j = 0; j < len; j++) {
00124             pByteSegBuf[len - j - 1] &= currByteSeg[-j];
00125         }
00126     }
00127 
00128     // the next set of segments to read will start past the end of
00129     // the overlapping segments just read
00130     startRid = addRid + len * LbmSegment::LbmOneByteSize;
00131     writeStartRidParamValue();
00132 
00133     // add the AND'd segment to the segment under construction;
00134     // if the segment is full and the output buffer fills up writing
00135     // out the segment, return
00136     assert(len > 0);
00137     addLen = len;
00138     addByteSeg = pByteSegBuf;
00139     if (!addSegments()) {
00140         return false;
00141     }
00142 
00143     return true;
00144 }

void LbmIntersectExecStream::prepare ( LbmIntersectExecStreamParams const & params ) [virtual]

Definition at line 28 of file LbmIntersectExecStream.cpp.

References LbmBitOpExecStream::prepare().

00029 {
00030     LbmBitOpExecStream::prepare(params);
00031 }

void LbmIntersectExecStream::open ( bool restart ) [virtual]

Opens this stream, acquiring any resources needed in order to be able to fetch data.

A precondition is that input streams must already be opened. A stream can be closed and reopened.

Parameters:

restart

if true, the stream must be already open, and should reset itself to start from the beginning of its result set

Reimplemented from LbmBitOpExecStream.

Definition at line 33 of file LbmIntersectExecStream.cpp.

References LbmBitOpExecStream::iInput, minLen, nMatches, and LbmBitOpExecStream::open().

00034 {
00035     LbmBitOpExecStream::open(restart);
00036     iInput = 0;
00037     nMatches = 0;
00038     minLen = 0;
00039 }

ExecStreamResult LbmIntersectExecStream::execute ( ExecStreamQuantum const & quantum ) [virtual]

Executes this stream.

Parameters:

quantum

governs the maximum amount of execution to perform

Returns:: code indicating reason execution ceased

Implements ExecStream.

Definition at line 41 of file LbmIntersectExecStream.cpp.

References EXECRC_BUF_OVERFLOW, EXECRC_QUANTUM_EXPIRED, EXECRC_YIELD, LbmBitOpExecStream::iInput, intersectSegments(), minLen, LbmBitOpExecStream::nInputs, nMatches, ExecStreamQuantum::nTuplesMax, LbmBitOpExecStream::producePending, LbmBitOpExecStream::producePendingOutput(), LbmBitOpExecStream::readInput(), LbmBitOpExecStream::startRid, and LbmBitOpExecStream::writeStartRidParamValue().

00043 {
00044     if (producePending) {
00045         ExecStreamResult rc = producePendingOutput(iInput);
00046         if (rc != EXECRC_YIELD) {
00047             return rc;
00048         }
00049         nMatches = 0;
00050     }
00051 
00052     for (uint i = 0; i < quantum.nTuplesMax; i++) {
00053         while (nMatches < nInputs) {
00054             // get the first segment from the input with at least a starting
00055             // rid of startRid
00056             LcsRid currRid;
00057             PBuffer currByteSeg;
00058             uint currLen;
00059 
00060             ExecStreamResult rc = readInput(
00061                 iInput, currRid, currByteSeg, currLen);
00062             if (rc != EXECRC_YIELD) {
00063                 return rc;
00064             }
00065 
00066             // if the starting rid of this current segment that has just
00067             // been read matches the desired starting rid, indicate that
00068             // we have a match; otherwise, reset the starting rid to the
00069             // new, larger rid value and start all over again, looking
00070             // for matches from all other input streams
00071             assert(currRid >= startRid);
00072             if (currRid > startRid) {
00073                 startRid = currRid;
00074                 writeStartRidParamValue();
00075                 nMatches = 1;
00076                 minLen = currLen;
00077             } else {
00078                 // the shortest segment indicates where the overlapping ends
00079                 if (nMatches == 0 || currLen < minLen) {
00080                     minLen = currLen;
00081                 }
00082                 nMatches++;
00083             }
00084 
00085             // now try reading segments from the other streams
00086             iInput = ++iInput % nInputs;
00087         }
00088 
00089         // intersect the overlapping segments
00090         if (!intersectSegments(minLen)) {
00091             return EXECRC_BUF_OVERFLOW;
00092         }
00093 
00094         nMatches = 0;
00095     }
00096 
00097     return EXECRC_QUANTUM_EXPIRED;
00098 }

void LbmIntersectExecStream::closeImpl ( ) [virtual]

Implements ClosableObject.

ExecStream implementations may override this to release any resources acquired while open.

Reimplemented from LbmBitOpExecStream.

Definition at line 146 of file LbmIntersectExecStream.cpp.

References LbmBitOpExecStream::closeImpl().

00147 {
00148     LbmBitOpExecStream::closeImpl();
00149 }

ExecStreamResult LbmBitOpExecStream::producePendingOutput ( uint iInput ) [protected, inherited]

Produces output tuple that previously failed due to buffer overflow.

Writes out the remaining segments that could not fit in the previous buffer. Determines if input has reached EOS.

Parameters:

iInput

input to read to determine if EOS reached

Returns:: EXECRC_YIELD if successful

Definition at line 110 of file LbmBitOpExecStream.cpp.

References LbmBitOpExecStream::addSegments(), EXECBUF_EOS, EXECRC_BUF_OVERFLOW, EXECRC_EOS, EXECRC_YIELD, ConfluenceExecStream::inAccessors, LbmSegmentWriter::isEmpty(), LbmBitOpExecStream::outputTuple, SingleOutputExecStream::pOutAccessor, LbmBitOpExecStream::producePending, LbmBitOpExecStream::produceTuple(), LbmSegmentWriter::reset(), and LbmBitOpExecStream::segmentWriter.

Referenced by LbmMinusExecStream::execute(), and execute().

00111 {
00112     if (!produceTuple(outputTuple)) {
00113         return EXECRC_BUF_OVERFLOW;
00114     }
00115     // in the middle of adding segments when buffer overflow occurred;
00116     // go back and add the remaining segments
00117     if (!segmentWriter.isEmpty()) {
00118         segmentWriter.reset();
00119         if (!addSegments()) {
00120             return EXECRC_BUF_OVERFLOW;
00121         }
00122     }
00123     producePending = false;
00124     if (inAccessors[iInput]->getState() == EXECBUF_EOS) {
00125         pOutAccessor->markEOS();
00126         return EXECRC_EOS;
00127     }
00128 
00129     return EXECRC_YIELD;
00130 }

ExecStreamResult LbmBitOpExecStream::readInput	(	uint	iInput,
		LcsRid &	currRid,
		PBuffer &	currByteSeg,
		uint &	currLen
	)			`[protected, inherited]`

Reads a byte segment from a specific input stream.

Parameters:

	iInput	input to read
	currRid	startRid of the segment read
	currByteSeg	byte segment read; points to the beginning of segment that is stored backwards
	currLen	length of the byte segment read

Returns:: EXECRC_YIELD if read was successful

Definition at line 132 of file LbmBitOpExecStream.cpp.

References LbmBitOpExecStream::bitmapBufSize, EXECRC_BUF_OVERFLOW, EXECRC_EOS, EXECRC_YIELD, LbmBitOpExecStream::flush(), SingleOutputExecStream::pOutAccessor, LbmBitOpExecStream::segmentReaders, and LbmBitOpExecStream::startRid.

Referenced by execute(), and LbmMinusExecStream::readMinuendInputAndFlush().

00134 {
00135     ExecStreamResult rc = segmentReaders[iInput].advanceToRid(startRid);
00136 
00137     if (rc == EXECRC_EOS) {
00138         // write out the last pending segment
00139         if (! flush()) {
00140             return EXECRC_BUF_OVERFLOW;
00141         }
00142         pOutAccessor->markEOS();
00143         return EXECRC_EOS;
00144     } else if (rc != EXECRC_YIELD) {
00145         return rc;
00146     }
00147 
00148     segmentReaders[iInput].readCurrentByteSegment(
00149         currRid, currByteSeg, currLen);
00150     // segment read should never be larger than space available
00151     // for segments
00152     assert(currLen <= bitmapBufSize);
00153 
00154     return EXECRC_YIELD;
00155 }

bool LbmBitOpExecStream::flush ( ) [protected, inherited]

Flushes the segment writer if it has any data.

The data is transferred to a bitmap tuple and the segment writer is reset. Finally, the tuple is produced to the output stream. If the tuple cannot be written, then it becomes a pending tuple.

Returns:: false if buffer overflow occured while producing the tuple

Definition at line 157 of file LbmBitOpExecStream.cpp.

References LbmSegmentWriter::isEmpty(), LbmBitOpExecStream::outputTuple, LbmBitOpExecStream::producePending, LbmSegmentWriter::produceSegmentTuple(), LbmBitOpExecStream::produceTuple(), LbmSegmentWriter::reset(), and LbmBitOpExecStream::segmentWriter.

Referenced by LbmBitOpExecStream::readInput(), LbmMinusExecStream::readMinuendInput(), and LbmMinusExecStream::readMinuendInputAndFlush().

00158 {
00159     assert (!producePending);
00160 
00161     if (!segmentWriter.isEmpty()) {
00162         outputTuple = segmentWriter.produceSegmentTuple();
00163         segmentWriter.reset();
00164         if (!produceTuple(outputTuple)) {
00165             producePending = true;
00166         }
00167     }
00168     return !producePending;
00169  }

bool LbmBitOpExecStream::addSegments ( ) [protected, inherited]

Adds the processed segments to the segment under construction.

If the segment fills up, writes it to the output buffer and continues constructing the rest of the segment. Leading, trailing, and intermediate zeros in the segment are removed.

Returns:: false if buffer overflow occurred writing out a segment

Definition at line 171 of file LbmBitOpExecStream.cpp.

References LbmBitOpExecStream::addByteSeg, LbmBitOpExecStream::addLen, LbmBitOpExecStream::addRid, LbmSegmentWriter::addSegment(), LbmBitOpExecStream::outputTuple, LbmBitOpExecStream::producePending, LbmSegmentWriter::produceSegmentTuple(), LbmBitOpExecStream::produceTuple(), LbmSegmentWriter::reset(), and LbmBitOpExecStream::segmentWriter.

Referenced by LbmMinusExecStream::execute(), intersectSegments(), and LbmBitOpExecStream::producePendingOutput().

00172 {
00173     while (addLen > 0) {
00174         if (segmentWriter.addSegment(addRid, addByteSeg, addLen)) {
00175             break;
00176         }
00177 
00178         outputTuple = segmentWriter.produceSegmentTuple();
00179         if (!produceTuple(outputTuple)) {
00180             producePending = true;
00181             return false;
00182         }
00183 
00184         // loop back and start creating a new segment for the remainder of
00185         // the segments that wouldn't fit
00186         segmentWriter.reset();
00187     }
00188 
00189     return true;
00190 }

void LbmBitOpExecStream::writeStartRidParamValue ( ) [protected, inherited]

Writes the startRid value to the startRid dynamic parameter, if one exists.

Definition at line 205 of file LbmBitOpExecStream.cpp.

References opaqueToInt(), ExecStream::pDynamicParamManager, LbmBitOpExecStream::startRidDatum, and LbmBitOpExecStream::startRidParamId.

Referenced by execute(), intersectSegments(), LbmBitOpExecStream::open(), and LbmMinusExecStream::readMinuendInputAndFlush().

00206 {
00207     if (opaqueToInt(startRidParamId) > 0) {
00208         pDynamicParamManager->writeParam(startRidParamId, startRidDatum);
00209     }
00210 }

bool LbmBitOpExecStream::produceTuple ( TupleData bitmapTuple ) [protected, virtual, inherited]

Produces a tuple to the output stream, based on a bitmap.

Returns:: false if buffer overflow occured while producing the tuple

Reimplemented in LbmMinusExecStream.

Definition at line 192 of file LbmBitOpExecStream.cpp.

References SingleOutputExecStream::pOutAccessor.

Referenced by LbmBitOpExecStream::addSegments(), LbmBitOpExecStream::flush(), and LbmBitOpExecStream::producePendingOutput().

00193 {
00194     assert(pOutAccessor->getTupleDesc().size() == bitmapTuple.size());
00195     return pOutAccessor->produceTuple(bitmapTuple);
00196 }

void LbmBitOpExecStream::prepare ( LbmBitOpExecStreamParams const & params ) [virtual, inherited]

Definition at line 28 of file LbmBitOpExecStream.cpp.

Referenced by LbmMinusExecStream::prepare(), and prepare().

00029 {
00030     ConfluenceExecStream::prepare(params);
00031 
00032     // set dynanmic parameter ids
00033     rowLimitParamId = params.rowLimitParamId;
00034     startRidParamId = params.startRidParamId;
00035 
00036     // setup tupledatums for writing dynamic parameter values
00037     rowLimitDatum.pData = (PConstBuffer) &rowLimit;
00038     rowLimitDatum.cbData = sizeof(rowLimit);
00039     startRidDatum.pData = (PConstBuffer) &startRid;
00040     startRidDatum.cbData = sizeof(startRid);
00041 
00042     // initialize segment readers for reading bitmaps from input stream
00043     nInputs = inAccessors.size();
00044     segmentReaders.reset(new LbmSegmentReader[nInputs]);
00045     bitmapSegTuples.reset(new TupleData[nInputs]);
00046     for (uint i = 0; i < nInputs; i++) {
00047         bitmapSegTuples[i].compute(inAccessors[i]->getTupleDesc());
00048     }
00049 
00050     nFields = inAccessors[0]->getTupleDesc().size() - 3;
00051 }

void ConfluenceExecStream::prepare ( ConfluenceExecStreamParams const & params ) [virtual, inherited]

Definition at line 37 of file ConfluenceExecStream.cpp.

References ConfluenceExecStream::getInputBufProvision(), ConfluenceExecStream::inAccessors, and SingleOutputExecStream::prepare().

Referenced by LcsRowScanBaseExecStream::prepare(), LbmUnionExecStream::prepare(), LbmChopperExecStream::prepare(), LbmBitOpExecStream::prepare(), LhxJoinExecStream::prepare(), MergeExecStream::prepare(), CorrelationJoinExecStream::prepare(), CartesianJoinExecStream::prepare(), and BarrierExecStream::prepare().

00038 {
00039     SingleOutputExecStream::prepare(params);
00040 
00041     for (uint i = 0; i < inAccessors.size(); ++i) {
00042         assert(inAccessors[i]->getProvision() == getInputBufProvision());
00043     }
00044 }

void SingleOutputExecStream::prepare ( SingleOutputExecStreamParams const & params ) [virtual, inherited]

Definition at line 48 of file SingleOutputExecStream.cpp.

References SingleOutputExecStream::getOutputBufProvision(), SingleOutputExecStreamParams::outputTupleDesc, SingleOutputExecStreamParams::outputTupleFormat, SingleOutputExecStream::pOutAccessor, and ExecStream::prepare().

Referenced by BTreeExecStream::prepare(), FlatFileExecStreamImpl::prepare(), ValuesExecStream::prepare(), MockResourceExecStream::prepare(), MockProducerExecStream::prepare(), ConfluenceExecStream::prepare(), and ConduitExecStream::prepare().

00049 {
00050     ExecStream::prepare(params);
00051     assert(pOutAccessor);
00052     assert(pOutAccessor->getProvision() == getOutputBufProvision());
00053     if (pOutAccessor->getTupleDesc().empty()) {
00054         assert(!params.outputTupleDesc.empty());
00055         pOutAccessor->setTupleShape(
00056             params.outputTupleDesc,
00057             params.outputTupleFormat);
00058     }
00059 }

void ExecStream::prepare ( ExecStreamParams const & params ) [virtual, inherited]

Prepares this stream for execution.

A precondition is that input streams must already be defined and prepared. As an effect of this call, the tuple shape should be defined for all output buffers and remain unchanged for the lifetime of the stream. This method is only ever called once, before the first open. Although this method is virtual, derived classes may choose to define an overloaded version instead with a specialized covariant parameter class.

Parameters:

params

instance of stream parameterization class which should be used to prepare this stream

Definition at line 84 of file ExecStream.cpp.

References ExecStreamGraph::getDynamicParamManager(), SegmentAccessor::pCacheAccessor, ExecStreamParams::pCacheAccessor, ExecStream::pDynamicParamManager, ExecStream::pGraph, ExecStream::pQuotaAccessor, ExecStream::pScratchQuotaAccessor, and ExecStreamParams::scratchAccessor.

Referenced by JavaTransformExecStream::prepare(), SingleOutputExecStream::prepare(), and SingleInputExecStream::prepare().

00085 {
00086     if (pGraph) {
00087         pDynamicParamManager = pGraph->getDynamicParamManager();
00088     }
00089     pQuotaAccessor = params.pCacheAccessor;
00090     pScratchQuotaAccessor = params.scratchAccessor.pCacheAccessor;
00091 }

void ConfluenceExecStream::setInputBufAccessors ( std::vector< SharedExecStreamBufAccessor > const & inAccessors ) [virtual, inherited]

Initializes the buffer accessors for inputs to this stream.

This method is only ever called once, before prepare.

Parameters:

inAccessors

buffer accessors ordered by input stream

Reimplemented from SingleOutputExecStream.

Definition at line 31 of file ConfluenceExecStream.cpp.

References ConfluenceExecStream::inAccessors.

00033 {
00034     inAccessors = inAccessorsInit;
00035 }

ExecStreamBufProvision ConfluenceExecStream::getInputBufProvision ( ) const [virtual, inherited]

Queries the BufferProvision which this stream requires of its inputs when consuming their tuples.

Returns:: required model; default is BUFPROV_NONE

Reimplemented from ExecStream.

Definition at line 58 of file ConfluenceExecStream.cpp.

References BUFPROV_PRODUCER.

Referenced by ConfluenceExecStream::prepare().

00059 {
00060     return BUFPROV_PRODUCER;
00061 }

void SingleOutputExecStream::setOutputBufAccessors ( std::vector< SharedExecStreamBufAccessor > const & outAccessors ) [virtual, inherited]

Initializes the buffer accessors for outputs from this stream.

This method is only ever called once, before prepare.

Parameters:

outAccessors

buffer accessors ordered by output stream

Implements ExecStream.

Reimplemented in ConduitExecStream.

Definition at line 41 of file SingleOutputExecStream.cpp.

References SingleOutputExecStream::pOutAccessor.

Referenced by ConduitExecStream::setOutputBufAccessors().

00043 {
00044     assert(outAccessors.size() == 1);
00045     pOutAccessor = outAccessors[0];
00046 }

ExecStreamBufProvision SingleOutputExecStream::getOutputBufProvision ( ) const [virtual, inherited]

Queries the BufferProvision which this stream is capable of when producing tuples.

Returns:: supported model; default is BUFPROV_NONE

Reimplemented from ExecStream.

Reimplemented in BarrierExecStream, DoubleBufferExecStream, MergeExecStream, MockResourceExecStream, ScratchBufferExecStream, SegBufferExecStream, SegBufferReaderExecStream, ValuesExecStream, FtrsTableWriterExecStream, and LcsClusterAppendExecStream.

Definition at line 69 of file SingleOutputExecStream.cpp.

References BUFPROV_CONSUMER.

Referenced by SingleOutputExecStream::prepare().

00070 {
00071     return BUFPROV_CONSUMER;
00072 }

bool ExecStream::canEarlyClose ( ) [virtual, inherited]

Returns:: true if the stream can be closed early

Reimplemented in SegBufferWriterExecStream.

Definition at line 49 of file ExecStream.cpp.

00050 {
00051     return true;
00052 }

ExecStreamGraph & ExecStream::getGraph ( ) const [inline, inherited]

Returns:: reference to containing graph

Definition at line 293 of file ExecStream.h.

References ExecStream::pGraph.

Referenced by ExternalSortExecStreamImpl::execute(), JavaSinkExecStream::execute(), SegBufferWriterExecStream::execute(), SegBufferExecStream::execute(), BarrierExecStream::execute(), Java_net_sf_farrago_fennel_FennelStorage_tupleStreamFetch(), Java_net_sf_farrago_fennel_FennelStorage_tupleStreamTransformFetch(), JavaSinkExecStream::stuffByteBuffer(), and ExecStreamScheduler::traceStreamBuffers().

00294 {
00295     assert(pGraph);
00296     return *pGraph;
00297 }

ExecStreamId ExecStream::getStreamId ( ) const [inline, inherited]

Returns:: the identifier for this stream within containing graph

Definition at line 288 of file ExecStream.h.

References ExecStream::id.

Referenced by ExternalSortExecStreamImpl::execute(), SegBufferWriterExecStream::execute(), SegBufferExecStream::execute(), CorrelationJoinExecStream::execute(), BarrierExecStream::execute(), Java_net_sf_farrago_fennel_FennelStorage_tupleStreamTransformFetch(), JavaTransformExecStream::open(), SingleInputExecStream::open(), ConfluenceExecStream::open(), CartesianJoinExecStream::prepare(), ParallelExecStreamScheduler::readStream(), DfsTreeExecStreamScheduler::readStream(), LbmMinusExecStream::restartSubtrahends(), ExecStreamScheduler::tracePostExecution(), ExecStreamScheduler::tracePreExecution(), ExecStreamScheduler::traceStreamBuffers(), and ParallelExecStreamScheduler::tryExecuteTask().

00289 {
00290     return id;
00291 }

void ExecStream::getResourceRequirements	(	ExecStreamResourceQuantity &	minQuantity,
		ExecStreamResourceQuantity &	optQuantity,
		ExecStreamResourceSettingType &	optType
	)			`[virtual, inherited]`

Determines resource requirements for this stream.

Default implementation declares zero resource requirements.

Parameters:

	minQuantity	receives the minimum resource quantity needed by this stream in order to execute
	optQuantity	receives the resource quantity needed by this stream in order to execute optimally
	optType	Receives the value indicating the accuracy of the optQuantity parameter. This parameter is optional and defaults to EXEC_RESOURCE_ACCURATE if omitted. If the optimum setting is an estimate or no value can be specified (e.g., due to lack of statistics), then this parameter needs to be used to indicate a non-accurate optimum resource setting.

Reimplemented in MockResourceExecStream, BTreePrefetchSearchExecStream, LhxAggExecStream, LhxJoinExecStream, LbmGeneratorExecStream, LbmUnionExecStream, and ExternalSortExecStreamImpl.

Definition at line 93 of file ExecStream.cpp.

References EXEC_RESOURCE_ACCURATE.

00097 {
00098     getResourceRequirements(minQuantity, optQuantity);
00099     optType = EXEC_RESOURCE_ACCURATE;
00100 }

void ExecStream::getResourceRequirements	(	ExecStreamResourceQuantity &	minQuantity,
		ExecStreamResourceQuantity &	optQuantity
	)			`[virtual, inherited]`

Reimplemented in DoubleBufferExecStream, ScratchBufferExecStream, SegBufferExecStream, SegBufferReaderExecStream, SegBufferWriterExecStream, FlatFileExecStreamImpl, BTreeInsertExecStream, BTreeReadExecStream, FtrsTableWriterExecStream, LbmChopperExecStream, LbmSplicerExecStream, LcsClusterAppendExecStream, LcsClusterReplaceExecStream, LcsRowScanBaseExecStream, and LcsRowScanExecStream.

Definition at line 102 of file ExecStream.cpp.

References ExecStreamResourceQuantity::nCachePages, and ExecStreamResourceQuantity::nThreads.

00105 {
00106     minQuantity.nThreads = 0;
00107     minQuantity.nCachePages = 0;
00108     optQuantity = minQuantity;
00109 }

void ExecStream::setResourceAllocation ( ExecStreamResourceQuantity & quantity ) [virtual, inherited]

Sets current resource allocation for this stream.

If called while the stream is open, this indicates a request for the stream to dynamically adjust its memory usage. If the stream is incapable of honoring the request, it should update quantity with the actual amounts still in use.

Parameters:

quantity

allocated resource quantity

Reimplemented in MockResourceExecStream, BTreePrefetchSearchExecStream, LhxAggExecStream, LhxJoinExecStream, LbmGeneratorExecStream, LbmUnionExecStream, and ExternalSortExecStreamImpl.

Definition at line 111 of file ExecStream.cpp.

References ExecStreamResourceQuantity::nCachePages, ExecStream::pQuotaAccessor, ExecStream::pScratchQuotaAccessor, and ExecStream::resourceAllocation.

Referenced by ExternalSortExecStreamImpl::setResourceAllocation(), LbmUnionExecStream::setResourceAllocation(), LbmGeneratorExecStream::setResourceAllocation(), LhxJoinExecStream::setResourceAllocation(), LhxAggExecStream::setResourceAllocation(), and BTreePrefetchSearchExecStream::setResourceAllocation().

00113 {
00114     resourceAllocation = quantity;
00115     if (pQuotaAccessor) {
00116         pQuotaAccessor->setMaxLockedPages(quantity.nCachePages);
00117     }
00118     if (pScratchQuotaAccessor) {
00119         pScratchQuotaAccessor->setMaxLockedPages(quantity.nCachePages);
00120     }
00121 }

void ExecStream::setName ( std::string const & ) [virtual, inherited]

Sets unique name of this stream.

Definition at line 157 of file ExecStream.cpp.

References ExecStream::name.

00158 {
00159     name = nameInit;
00160 }

std::string const & ExecStream::getName ( ) const [virtual, inherited]

Returns:: the name of this stream, as known by the optimizer

Definition at line 162 of file ExecStream.cpp.

References ExecStream::name.

Referenced by DfsTreeExecStreamScheduler::findNextConsumer(), ParallelExecStreamScheduler::readStream(), DfsTreeExecStreamScheduler::readStream(), ExecStreamScheduler::tracePostExecution(), and ExecStreamScheduler::tracePreExecution().

00163 {
00164     return name;
00165 }

bool ExecStream::mayBlock ( ) const [virtual, inherited]

Queries whether this stream's implementation may block when execute() is called.

For accurate scheduling, non-blocking implementations are preferred; the scheduler must be aware of the potential for blocking so that it can allocate extra threads accordingly.

Returns:: whether stream may block; default is false

Definition at line 167 of file ExecStream.cpp.

00168 {
00169     return false;
00170 }

void ExecStream::checkAbort ( ) const [virtual, inherited]

Checks whether there is an abort request for this stream's scheduler.

Normally, streams don't need to check this, since the scheduler services abort requests in between quanta. However, streams which enter long-running loops need to check for themselves. If an abort is scheduled, this method will throw an AbortExcn automatically.

Definition at line 72 of file ExecStream.cpp.

References ExecStreamScheduler::checkAbort(), ExecStreamGraph::getScheduler(), and ExecStream::pGraph.

Referenced by LhxJoinExecStream::execute(), LhxAggExecStream::execute(), ExternalSortRunAccessor::fetch(), and ExternalSortMerger::fetch().

00073 {
00074     if (!pGraph) {
00075         return;
00076     }
00077     ExecStreamScheduler *pScheduler = pGraph->getScheduler();
00078     if (!pScheduler) {
00079         return;
00080     }
00081     pScheduler->checkAbort();
00082 }

ExecStreamBufProvision ExecStream::getOutputBufConversion ( ) const [virtual, inherited]

Queries the BufferProvision to which this stream needs its output to be converted, if any.

Returns:: required conversion; default is BUFPROV_NONE

Reimplemented in JavaTransformExecStream.

Definition at line 177 of file ExecStream.cpp.

References BUFPROV_NONE.

00178 {
00179     return BUFPROV_NONE;
00180 }

bool ClosableObject::isClosed ( ) const [inline, inherited]

Returns:: whether the object has been closed

Definition at line 58 of file ClosableObject.h.

00059     {
00060         return !needsClose;
00061     }

void ClosableObject::close ( ) [inherited]

Closes this object, releasing any unallocated resources.

Reimplemented in CollectExecStream, CorrelationJoinExecStream, LcsClusterAppendExecStream, and LcsClusterReplaceExecStream.

Definition at line 39 of file ClosableObject.cpp.

References ClosableObject::closeImpl(), and ClosableObject::needsClose.

Referenced by CacheImpl< PageT, VictimPolicyT >::allocatePages(), LcsRowScanBaseExecStream::closeImpl(), ExecStreamGraphImpl::closeImpl(), FlatFileBuffer::open(), ClosableObjectDestructor::operator()(), and Segment::~Segment().

00040 {
00041     if (!needsClose) {
00042         return;
00043     }
00044     needsClose = false;
00045     closeImpl();
00046 }

void TraceSource::initTraceSource	(	SharedTraceTarget	pTraceTarget,
		std::string	name
	)			`[virtual, inherited]`

For use when initialization has to be deferred until after construction.

Parameters:

	pTraceTarget	the TraceTarget to which messages will be sent
	name	the name of this source

Definition at line 46 of file TraceSource.cpp.

References TraceSource::isTracing(), TraceSource::minimumLevel, TraceSource::name, TraceSource::pTraceTarget, and TRACE_OFF.

Referenced by TestBase::beforeTestCase(), TestBase::TestBase(), and TraceSource::TraceSource().

00049 {
00050     assert(!pTraceTarget.get());
00051 
00052     pTraceTarget = pTraceTargetInit;
00053     name = nameInit;
00054     if (isTracing()) {
00055         minimumLevel = pTraceTarget->getSourceTraceLevel(name);
00056     } else {
00057         minimumLevel = TRACE_OFF;
00058     }
00059 }

void TraceSource::trace	(	TraceLevel	level,
		std::string	message
	)			const `[inherited]`

Records a trace message.

Normally only called via FENNEL_TRACE.

Parameters:

	level	severity level of event being trace
	message	the text of the message

Definition at line 61 of file TraceSource.cpp.

References TraceSource::getTraceTarget(), TraceSource::isTracing(), TraceSource::name, and TraceTarget::notifyTrace().

Referenced by Calculator::exec(), and ExecStreamScheduler::traceStreamBufferContents().

00062 {
00063     if (isTracing()) {
00064         getTraceTarget().notifyTrace(name,level,message);
00065     }
00066 }

bool TraceSource::isTracing ( ) const [inline, inherited]

Returns:: true iff tracing is enabled for this source

Definition at line 88 of file TraceSource.h.

Referenced by TraceSource::initTraceSource(), CalcExecStream::prepare(), and TraceSource::trace().

00089     {
00090         return pTraceTarget.get() ? true : false;
00091     }

bool TraceSource::isTracingLevel ( TraceLevel level ) const [inline, inherited]

Determines whether a particular level is being traced.

Parameters:

level

trace level to test

Returns:: true iff tracing is enabled for the given level

Definition at line 100 of file TraceSource.h.

Referenced by ExecStreamScheduler::addGraph(), SimpleExecStreamGovernor::assignCachePages(), SimpleExecStreamGovernor::distributeCachePages(), Calculator::exec(), ExecStreamScheduler::ExecStreamScheduler(), LcsClusterNodeWriter::getLastClusterPageForWrite(), LcsClusterNodeWriter::moveFromTempToIndex(), JavaSinkExecStream::stuffByteBuffer(), and ExecStreamScheduler::traceStreamBuffers().

00101     {
00102         return level >= minimumLevel;
00103     }

TraceTarget& TraceSource::getTraceTarget ( ) const [inline, inherited]

Returns:: the TraceTarget for this source

Definition at line 108 of file TraceSource.h.

Referenced by TraceSource::trace().

00109     {
00110         assert(isTracing());
00111         return *(pTraceTarget.get());
00112     }

SharedTraceTarget TraceSource::getSharedTraceTarget ( ) const [inline, inherited]

Returns:: the SharedTraceTarget for this source

Definition at line 117 of file TraceSource.h.

Referenced by Database::init(), LcsClusterAppendExecStream::initLoad(), and CalcExecStream::prepare().

00118     {
00119         return pTraceTarget;
00120     }

std::string TraceSource::getTraceSourceName ( ) const [inline, inherited]

Gets the name of this source.

Useful to construct nested names for subcomponents that are also TraceSources.

Returns:: the name

Definition at line 127 of file TraceSource.h.

Referenced by LcsClusterAppendExecStream::initLoad().

00128     {
00129         return name;
00130     }

void TraceSource::setTraceSourceName ( std::string const & n ) [inline, inherited]

Sets the name of this source.

Useful to construct dynamic names for fine-grained filtering.

Definition at line 136 of file TraceSource.h.

00137     {
00138         name = n;
00139     }

TraceLevel TraceSource::getMinimumTraceLevel ( ) const [inline, inherited]

Definition at line 141 of file TraceSource.h.

00142     {
00143         return minimumLevel;
00144     }

void TraceSource::disableTracing ( ) [inherited]

Definition at line 68 of file TraceSource.cpp.

References TraceSource::minimumLevel, TraceSource::pTraceTarget, and TRACE_OFF.

Referenced by TestBase::afterTestCase().

00069 {
00070     pTraceTarget.reset();
00071     minimumLevel = TRACE_OFF;
00072 }

void ErrorSource::initErrorSource	(	SharedErrorTarget	pErrorTarget,
		const std::string &	name
	)			`[virtual, inherited]`

For use when initialization has to be deferred until after construction.

Parameters:

	pErrorTarget	the ErrorTarget to which errors will be posted
	name	the name of this source

Definition at line 47 of file ErrorSource.cpp.

References ErrorSource::name, and ErrorSource::pErrorTarget.

Referenced by ErrorSource::ErrorSource().

00050 {
00051     pErrorTarget = pErrorTargetInit;
00052     name = nameInit;
00053 }

void ErrorSource::postError	(	ErrorLevel	level,
		const std::string &	message,
		void *	address,
		long	capacity,
		int	index
	)			`[inherited]`

Posts an exception, such as a row exception.

See also:: ErrorTarget for a description of the parameters

Definition at line 55 of file ErrorSource.cpp.

References ErrorSource::getErrorTarget(), ErrorSource::hasTarget(), ErrorSource::name, and ErrorTarget::notifyError().

Referenced by FlatFileExecStreamImpl::logError(), ErrorSource::postError(), and LbmSplicerExecStream::postViolation().

00058 {
00059     if (hasTarget()) {
00060         getErrorTarget().notifyError(
00061             name, level, message, address, capacity, index);
00062     }
00063 }

void ErrorSource::postError	(	ErrorLevel	level,
		const std::string &	message,
		const TupleDescriptor &	errorDesc,
		const TupleData &	errorTuple,
		int	index
	)			`[inherited]`

Posts an exception, such as a row exception.

See also:: ErrorTarget for a description of the parameters

Definition at line 65 of file ErrorSource.cpp.

References TupleAccessor::compute(), ErrorSource::errorAccessor, FixedBuffer, TupleAccessor::getByteCount(), TupleAccessor::getMaxByteCount(), ErrorSource::hasTarget(), TupleAccessor::marshal(), ErrorSource::pErrorBuf, and ErrorSource::postError().

00068 {
00069     if (!hasTarget()) {
00070         return;
00071     }
00072 
00073     if (!pErrorBuf) {
00074         errorAccessor.compute(errorDesc);
00075         uint cbMax = errorAccessor.getMaxByteCount();
00076         pErrorBuf.reset(new FixedBuffer[cbMax]);
00077     }
00078 
00079     uint cbTuple = errorAccessor.getByteCount(errorTuple);
00080     errorAccessor.marshal(errorTuple, pErrorBuf.get());
00081     postError(level, message, pErrorBuf.get(), cbTuple, index);
00082 }

bool ErrorSource::hasTarget ( ) const [inline, inherited]

Returns:: true iff an error target has been set

Definition at line 112 of file ErrorSource.h.

Referenced by ErrorSource::postError().

00113     {
00114         return pErrorTarget.get() ? true : false;
00115     }

ErrorTarget& ErrorSource::getErrorTarget ( ) const [inline, inherited]

Returns:: the ErrorTarget for this source

Definition at line 120 of file ErrorSource.h.

Referenced by ErrorSource::postError().

00121     {
00122         assert(hasTarget());
00123         return *(pErrorTarget.get());
00124     }

SharedErrorTarget ErrorSource::getSharedErrorTarget ( ) const [inline, inherited]

Returns:: the SharedErrorTarget for this source

Definition at line 129 of file ErrorSource.h.

00130     {
00131         return pErrorTarget;
00132     }

std::string ErrorSource::getErrorSourceName ( ) const [inline, inherited]

Gets the name of this source.

Useful to construct nested names for subcomponents that are also ErrorSources.

Returns:: the name

Definition at line 139 of file ErrorSource.h.

00140     {
00141         return name;
00142     }

void ErrorSource::setErrorSourceName ( std::string const & n ) [inline, inherited]

Sets the name of this source.

Useful to construct dynamic names for fine-grained filtering.

Definition at line 148 of file ErrorSource.h.

00149     {
00150         name = n;
00151     }

void ErrorSource::disableTarget ( ) [inherited]

Definition at line 84 of file ErrorSource.cpp.

References ErrorSource::pErrorTarget.

00085 {
00086     pErrorTarget.reset();
00087 }

uint LbmIntersectExecStream::nMatches [private]

Number of inputs with overlapping rid values.

Must be equal to nInputs for an intersection to take place.

Definition at line 51 of file LbmIntersectExecStream.h.

Referenced by execute(), and open().

uint LbmIntersectExecStream::minLen [private]

Minimum length of overlapping bitmap segments found thus far.

Definition at line 56 of file LbmIntersectExecStream.h.

Referenced by execute(), and open().

DynamicParamId LbmBitOpExecStream::rowLimitParamId [protected, inherited]

Parameter id representing the dynamic parameter used to limit the number of rows producers for this stream should produce on a single execute.

Definition at line 71 of file LbmBitOpExecStream.h.

Referenced by LbmBitOpExecStream::open(), and LbmBitOpExecStream::prepare().

DynamicParamId LbmBitOpExecStream::startRidParamId [protected, inherited]

Parameter id representing the dynamic parameter used to set the starting rid value for bitmap entries to be produced by this stream's producers.

Definition at line 78 of file LbmBitOpExecStream.h.

Referenced by LbmBitOpExecStream::open(), LbmBitOpExecStream::prepare(), and LbmBitOpExecStream::writeStartRidParamValue().

TupleDatum LbmBitOpExecStream::rowLimitDatum [protected, inherited]

Tuple datum used to store dynamic paramter for rowLimit.

Definition at line 83 of file LbmBitOpExecStream.h.

Referenced by LbmBitOpExecStream::open(), and LbmBitOpExecStream::prepare().

TupleDatum LbmBitOpExecStream::startRidDatum [protected, inherited]

Tuple datum used to store dynamic parameter for startRid.

Definition at line 88 of file LbmBitOpExecStream.h.

Referenced by LbmBitOpExecStream::prepare(), and LbmBitOpExecStream::writeStartRidParamValue().

RecordNum LbmBitOpExecStream::rowLimit [protected, inherited]

Number of rows that can be produced.

Definition at line 93 of file LbmBitOpExecStream.h.

Referenced by LbmMinusExecStream::open(), LbmBitOpExecStream::open(), and LbmBitOpExecStream::prepare().

LcsRid LbmBitOpExecStream::startRid [protected, inherited]

Desired starting rid value for bitmap entries.

Definition at line 98 of file LbmBitOpExecStream.h.

Referenced by LbmMinusExecStream::checkNeedForRestart(), LbmMinusExecStream::execute(), execute(), intersectSegments(), LbmBitOpExecStream::open(), LbmBitOpExecStream::prepare(), LbmBitOpExecStream::readInput(), and LbmMinusExecStream::readMinuendInputAndFlush().

boost::scoped_array<LbmSegmentReader> LbmBitOpExecStream::segmentReaders [protected, inherited]

One segment reader for each input stream.

Definition at line 103 of file LbmBitOpExecStream.h.

Referenced by LbmMinusExecStream::advanceSingleSubtrahend(), LbmMinusExecStream::advanceSubtrahends(), LbmMinusExecStream::canSkipMinus(), LbmMinusExecStream::findMinInput(), intersectSegments(), LbmMinusExecStream::minusSegments(), LbmBitOpExecStream::open(), LbmBitOpExecStream::prepare(), LbmBitOpExecStream::readInput(), and LbmMinusExecStream::restartSubtrahends().

uint LbmBitOpExecStream::nInputs [protected, inherited]

Number of input streams.

Definition at line 108 of file LbmBitOpExecStream.h.

Referenced by LbmMinusExecStream::advanceSubtrahends(), LbmMinusExecStream::canSkipMinus(), execute(), LbmMinusExecStream::findMinInput(), intersectSegments(), LbmBitOpExecStream::open(), LbmBitOpExecStream::prepare(), and LbmMinusExecStream::restartSubtrahends().

uint LbmBitOpExecStream::iInput [protected, inherited]

Current input stream being processed.

Definition at line 113 of file LbmBitOpExecStream.h.

Referenced by LbmMinusExecStream::advanceSubtrahends(), execute(), open(), LbmMinusExecStream::readMinuendInputAndFlush(), and LbmMinusExecStream::restartSubtrahends().

boost::scoped_array<TupleData> LbmBitOpExecStream::bitmapSegTuples [protected, inherited]

Tuple data for each input stream.

Definition at line 118 of file LbmBitOpExecStream.h.

Referenced by LbmMinusExecStream::comparePrefixes(), LbmMinusExecStream::copyPrefix(), LbmMinusExecStream::open(), LbmBitOpExecStream::open(), LbmBitOpExecStream::prepare(), and LbmMinusExecStream::restartSubtrahends().

LbmSegmentWriter LbmBitOpExecStream::segmentWriter [protected, inherited]

Segment writer.

Definition at line 123 of file LbmBitOpExecStream.h.

Referenced by LbmBitOpExecStream::addSegments(), LbmBitOpExecStream::flush(), LbmBitOpExecStream::open(), and LbmBitOpExecStream::producePendingOutput().

boost::scoped_array<FixedBuffer> LbmBitOpExecStream::outputBuf [protected, inherited]

Buffer for writing output bitmap segment.

Definition at line 128 of file LbmBitOpExecStream.h.

Referenced by LbmBitOpExecStream::closeImpl(), and LbmBitOpExecStream::open().

boost::scoped_array<FixedBuffer> LbmBitOpExecStream::byteSegBuf [protected, inherited]

Temporary buffer for bit operation.

Definition at line 133 of file LbmBitOpExecStream.h.

Referenced by LbmBitOpExecStream::closeImpl(), and LbmBitOpExecStream::open().

PBuffer LbmBitOpExecStream::pByteSegBuf [protected, inherited]

Pointer to byteSegBuf.

Definition at line 138 of file LbmBitOpExecStream.h.

Referenced by LbmMinusExecStream::execute(), intersectSegments(), LbmMinusExecStream::minusSegments(), LbmBitOpExecStream::open(), and LbmMinusExecStream::readMinuendInputAndFlush().

uint LbmBitOpExecStream::bitmapBufSize [protected, inherited]

Amount of space available in buffer for bitmaps.

Definition at line 143 of file LbmBitOpExecStream.h.

Referenced by LbmBitOpExecStream::open(), LbmBitOpExecStream::readInput(), and LbmMinusExecStream::readMinuendInput().

TupleData LbmBitOpExecStream::outputTuple [protected, inherited]

Output tuple data containing AND'd bitmap segments.

Definition at line 148 of file LbmBitOpExecStream.h.

Referenced by LbmBitOpExecStream::addSegments(), LbmBitOpExecStream::flush(), and LbmBitOpExecStream::producePendingOutput().

bool LbmBitOpExecStream::producePending [protected, inherited]

True if a tuple needs to be written to the output stream.

Definition at line 153 of file LbmBitOpExecStream.h.

Referenced by LbmBitOpExecStream::addSegments(), LbmMinusExecStream::execute(), execute(), LbmBitOpExecStream::flush(), LbmBitOpExecStream::open(), and LbmBitOpExecStream::producePendingOutput().

LcsRid LbmBitOpExecStream::addRid [protected, inherited]

Current rid value to be added to the bitmap segment.

Definition at line 158 of file LbmBitOpExecStream.h.

Referenced by LbmBitOpExecStream::addSegments(), LbmMinusExecStream::execute(), and intersectSegments().

PBuffer LbmBitOpExecStream::addByteSeg [protected, inherited]

Current byte segment to be added.

Definition at line 163 of file LbmBitOpExecStream.h.

Referenced by LbmBitOpExecStream::addSegments(), LbmMinusExecStream::execute(), and intersectSegments().

uint LbmBitOpExecStream::addLen [protected, inherited]

Current length of byte segment to be added.

Definition at line 168 of file LbmBitOpExecStream.h.

Referenced by LbmBitOpExecStream::addSegments(), LbmMinusExecStream::execute(), and intersectSegments().

int LbmBitOpExecStream::nFields [protected, inherited]

Number of non-bitmap fields preceding the bitmap fields.

Definition at line 173 of file LbmBitOpExecStream.h.

Referenced by LbmMinusExecStream::checkNeedForRestart(), LbmMinusExecStream::comparePrefixes(), LbmMinusExecStream::copyPrefix(), LbmMinusExecStream::execute(), LbmMinusExecStream::open(), LbmMinusExecStream::prepare(), LbmBitOpExecStream::prepare(), LbmMinusExecStream::produceTuple(), LbmMinusExecStream::readMinuendInputAndFlush(), and LbmMinusExecStream::restartSubtrahends().

std::vector<SharedExecStreamBufAccessor> ConfluenceExecStream::inAccessors [protected, inherited]

Definition at line 50 of file ConfluenceExecStream.h.

SharedExecStreamBufAccessor SingleOutputExecStream::pOutAccessor [protected, inherited]

Definition at line 56 of file SingleOutputExecStream.h.

bool ExecStream::isOpen [protected, inherited]

Whether this stream is currently open.

Note that this is not quite the opposite of the inherited ClosableObject.needsClose, since a stream needs to be closed before destruction if it has been prepared but never opened.

Definition at line 61 of file ExecStream.h.

Referenced by ExecStream::closeImpl(), ExecStream::ExecStream(), and ExecStream::open().

ExecStreamGraph* ExecStream::pGraph [protected, inherited]

Dataflow graph containing this stream.

Note that we don't use a weak_ptr for this because it needs to be accessed frequently during execution, and the extra locking overhead would be frivolous.

Definition at line 68 of file ExecStream.h.

Referenced by ExecStream::checkAbort(), ExecStream::ExecStream(), CorrelationJoinExecStream::execute(), ExecStream::getGraph(), JavaTransformExecStream::open(), SingleInputExecStream::open(), MergeExecStream::open(), ExecStream::open(), CorrelationJoinExecStream::open(), ConfluenceExecStream::open(), ExecStream::prepare(), CartesianJoinExecStream::prepare(), and LbmMinusExecStream::restartSubtrahends().

ExecStreamId ExecStream::id [protected, inherited]

Identifier for this stream; local to its containing graph.

Definition at line 73 of file ExecStream.h.

Referenced by ExecStream::getStreamId().

std::string ExecStream::name [protected, inherited]

Name of stream, as known by optimizer.

Reimplemented from TraceSource.

Definition at line 78 of file ExecStream.h.

Referenced by ExecStream::ExecStream(), ExecStream::getName(), FlatFileExecStreamImpl::open(), and ExecStream::setName().

SharedDynamicParamManager ExecStream::pDynamicParamManager [protected, inherited]

The dynamic parameter manager available to this stream.

(Obtained at prepare() time. Keep a shared pointer in case the stream is reassigned to another graph for execution; cf ExecStreamGraph::mergeFrom())

Definition at line 85 of file ExecStream.h.

SharedLogicalTxn ExecStream::pTxn [protected, inherited]

The transaction embracing the stream.

Obtained at open() time; but not released at close() time, to allow TableWriters to replay a txn. Keep a shared pointer in case the stream is reassigned to another graph for execution; cf ExecStreamGraph::mergeFrom())

Definition at line 94 of file ExecStream.h.

Referenced by FtrsTableWriterExecStream::commitSavepoint(), FtrsTableWriterExecStream::createSavepoint(), FtrsTableWriterExecStream::open(), ExecStream::open(), and FtrsTableWriterExecStream::rollbackSavepoint().

ExecStreamResourceQuantity ExecStream::resourceAllocation [protected, inherited]

Resource quantities currently allocated to this stream.

Definition at line 100 of file ExecStream.h.

Referenced by ExecStream::setResourceAllocation().

SharedCacheAccessor ExecStream::pQuotaAccessor [protected, inherited]

CacheAccessor used for quota tracking.

Definition at line 105 of file ExecStream.h.

Referenced by ExecStream::open(), ExecStream::prepare(), and ExecStream::setResourceAllocation().

SharedCacheAccessor ExecStream::pScratchQuotaAccessor [protected, inherited]

CacheAccessor used for scratch page quota tracking.

Definition at line 110 of file ExecStream.h.

Referenced by ExecStream::open(), ExecStream::prepare(), and ExecStream::setResourceAllocation().

bool ClosableObject::needsClose [protected, inherited]

Definition at line 44 of file ClosableObject.h.

Referenced by SegStreamAllocation::beginWrite(), ExecStreamGraphImpl::clear(), ClosableObject::ClosableObject(), ClosableObject::close(), FlatFileBuffer::open(), ExecStreamGraphImpl::open(), ExecStream::open(), and ClosableObject::~ClosableObject().

LbmIntersectExecStream Class Reference

Public Member Functions

Protected Member Functions

Protected Attributes

Private Member Functions

Private Attributes

Detailed Description

Member Function Documentation

Member Data Documentation


Public Member Functions
virtual void	prepare (LbmIntersectExecStreamParams const &params)
virtual void	open (bool restart)
	Opens this stream, acquiring any resources needed in order to be able to fetch data.
virtual ExecStreamResult	execute (ExecStreamQuantum const &quantum)
	Executes this stream.
virtual void	closeImpl ()
	Implements ClosableObject.
virtual void	prepare (LbmBitOpExecStreamParams const &params)
virtual void	prepare (ConfluenceExecStreamParams const &params)
virtual void	prepare (SingleOutputExecStreamParams const &params)
virtual void	prepare (ExecStreamParams const &params)
	Prepares this stream for execution.
virtual void	setInputBufAccessors (std::vector< SharedExecStreamBufAccessor > const &inAccessors)
	Initializes the buffer accessors for inputs to this stream.
virtual ExecStreamBufProvision	getInputBufProvision () const
	Queries the BufferProvision which this stream requires of its inputs when consuming their tuples.
virtual void	setOutputBufAccessors (std::vector< SharedExecStreamBufAccessor > const &outAccessors)
	Initializes the buffer accessors for outputs from this stream.
virtual ExecStreamBufProvision	getOutputBufProvision () const
	Queries the BufferProvision which this stream is capable of when producing tuples.
virtual bool	canEarlyClose ()
	Returns: true if the stream can be closed early
ExecStreamGraph &	getGraph () const
	Returns: reference to containing graph
ExecStreamId	getStreamId () const
	Returns: the identifier for this stream within containing graph
virtual void	getResourceRequirements (ExecStreamResourceQuantity &minQuantity, ExecStreamResourceQuantity &optQuantity, ExecStreamResourceSettingType &optType)
	Determines resource requirements for this stream.
virtual void	getResourceRequirements (ExecStreamResourceQuantity &minQuantity, ExecStreamResourceQuantity &optQuantity)
virtual void	setResourceAllocation (ExecStreamResourceQuantity &quantity)
	Sets current resource allocation for this stream.
virtual void	setName (std::string const &)
	Sets unique name of this stream.
virtual std::string const &	getName () const
	Returns: the name of this stream, as known by the optimizer
virtual bool	mayBlock () const
	Queries whether this stream's implementation may block when execute() is called.
virtual void	checkAbort () const
	Checks whether there is an abort request for this stream's scheduler.
virtual ExecStreamBufProvision	getOutputBufConversion () const
	Queries the BufferProvision to which this stream needs its output to be converted, if any.
bool	isClosed () const
	Returns: whether the object has been closed
void	close ()
	Closes this object, releasing any unallocated resources.
virtual void	initTraceSource (SharedTraceTarget pTraceTarget, std::string name)
	For use when initialization has to be deferred until after construction.
void	trace (TraceLevel level, std::string message) const
	Records a trace message.
bool	isTracing () const
	Returns: true iff tracing is enabled for this source
bool	isTracingLevel (TraceLevel level) const
	Determines whether a particular level is being traced.
TraceTarget &	getTraceTarget () const
	Returns: the TraceTarget for this source
SharedTraceTarget	getSharedTraceTarget () const
	Returns: the SharedTraceTarget for this source
std::string	getTraceSourceName () const
	Gets the name of this source.
void	setTraceSourceName (std::string const &n)
	Sets the name of this source.
TraceLevel	getMinimumTraceLevel () const
void	disableTracing ()
virtual void	initErrorSource (SharedErrorTarget pErrorTarget, const std::string &name)
	For use when initialization has to be deferred until after construction.
void	postError (ErrorLevel level, const std::string &message, void *address, long capacity, int index)
	Posts an exception, such as a row exception.
void	postError (ErrorLevel level, const std::string &message, const TupleDescriptor &errorDesc, const TupleData &errorTuple, int index)
	Posts an exception, such as a row exception.
bool	hasTarget () const
	Returns: true iff an error target has been set
ErrorTarget &	getErrorTarget () const
	Returns: the ErrorTarget for this source
SharedErrorTarget	getSharedErrorTarget () const
	Returns: the SharedErrorTarget for this source
std::string	getErrorSourceName () const
	Gets the name of this source.
void	setErrorSourceName (std::string const &n)
	Sets the name of this source.
void	disableTarget ()
Protected Member Functions
ExecStreamResult	producePendingOutput (uint iInput)
	Produces output tuple that previously failed due to buffer overflow.
ExecStreamResult	readInput (uint iInput, LcsRid &currRid, PBuffer &currByteSeg, uint &currLen)
	Reads a byte segment from a specific input stream.
bool	flush ()
	Flushes the segment writer if it has any data.
bool	addSegments ()
	Adds the processed segments to the segment under construction.
void	writeStartRidParamValue ()
	Writes the startRid value to the startRid dynamic parameter, if one exists.
virtual bool	produceTuple (TupleData bitmapTuple)
	Produces a tuple to the output stream, based on a bitmap.
Protected Attributes
DynamicParamId	rowLimitParamId
	Parameter id representing the dynamic parameter used to limit the number of rows producers for this stream should produce on a single execute.
DynamicParamId	startRidParamId
	Parameter id representing the dynamic parameter used to set the starting rid value for bitmap entries to be produced by this stream's producers.
TupleDatum	rowLimitDatum
	Tuple datum used to store dynamic paramter for rowLimit.
TupleDatum	startRidDatum
	Tuple datum used to store dynamic parameter for startRid.
RecordNum	rowLimit
	Number of rows that can be produced.
LcsRid	startRid
	Desired starting rid value for bitmap entries.
boost::scoped_array< LbmSegmentReader >	segmentReaders
	One segment reader for each input stream.
uint	nInputs
	Number of input streams.
uint	iInput
	Current input stream being processed.
boost::scoped_array< TupleData >	bitmapSegTuples
	Tuple data for each input stream.
LbmSegmentWriter	segmentWriter
	Segment writer.
boost::scoped_array< FixedBuffer >	outputBuf
	Buffer for writing output bitmap segment.
boost::scoped_array< FixedBuffer >	byteSegBuf
	Temporary buffer for bit operation.
PBuffer	pByteSegBuf
	Pointer to byteSegBuf.
uint	bitmapBufSize
	Amount of space available in buffer for bitmaps.
TupleData	outputTuple
	Output tuple data containing AND'd bitmap segments.
bool	producePending
	True if a tuple needs to be written to the output stream.
LcsRid	addRid
	Current rid value to be added to the bitmap segment.
PBuffer	addByteSeg
	Current byte segment to be added.
uint	addLen
	Current length of byte segment to be added.
int	nFields
	Number of non-bitmap fields preceding the bitmap fields.
std::vector< SharedExecStreamBufAccessor >	inAccessors
SharedExecStreamBufAccessor	pOutAccessor
bool	isOpen
	Whether this stream is currently open.
ExecStreamGraph *	pGraph
	Dataflow graph containing this stream.
ExecStreamId	id
	Identifier for this stream; local to its containing graph.
std::string	name
	Name of stream, as known by optimizer.
SharedDynamicParamManager	pDynamicParamManager
	The dynamic parameter manager available to this stream.
SharedLogicalTxn	pTxn
	The transaction embracing the stream.
ExecStreamResourceQuantity	resourceAllocation
	Resource quantities currently allocated to this stream.
SharedCacheAccessor	pQuotaAccessor
	CacheAccessor used for quota tracking.
SharedCacheAccessor	pScratchQuotaAccessor
	CacheAccessor used for scratch page quota tracking.
bool	needsClose
Private Member Functions
bool	intersectSegments (uint len)
	Performs intersect operation on all segments.
Private Attributes
uint	nMatches
	Number of inputs with overlapping rid values.
uint	minLen
	Minimum length of overlapping bitmap segments found thus far.