LbmUnionExecStream Class Reference

LbmUnionExecStream is the execution stream used to perform a union on a stream of overlapping bitmap tuple. More...

#include <LbmUnionExecStream.h>

Inheritance diagram for LbmUnionExecStream:

ConfluenceExecStream SingleOutputExecStream ExecStream ClosableObject TraceSource ErrorSource List of all members.

Public Member Functions

virtual void prepare (LbmUnionExecStreamParams const &params)
virtual void open (bool restart)
 Opens this stream, acquiring any resources needed in order to be able to fetch data.
virtual ExecStreamResult execute (ExecStreamQuantum const &quantum)
 Executes this stream.
virtual void getResourceRequirements (ExecStreamResourceQuantity &minQuantity, ExecStreamResourceQuantity &optQuantity, ExecStreamResourceSettingType &optType)
 Determines resource requirements for this stream.
virtual void setResourceAllocation (ExecStreamResourceQuantity &quantity)
 Sets current resource allocation for this stream.
virtual void closeImpl ()
 Implements ClosableObject.
virtual void prepare (ConfluenceExecStreamParams const &params)
virtual void prepare (SingleOutputExecStreamParams const &params)
virtual void prepare (ExecStreamParams const &params)
 Prepares this stream for execution.
virtual void setInputBufAccessors (std::vector< SharedExecStreamBufAccessor > const &inAccessors)
 Initializes the buffer accessors for inputs to this stream.
virtual ExecStreamBufProvision getInputBufProvision () const
 Queries the BufferProvision which this stream requires of its inputs when consuming their tuples.
virtual void setOutputBufAccessors (std::vector< SharedExecStreamBufAccessor > const &outAccessors)
 Initializes the buffer accessors for outputs from this stream.
virtual ExecStreamBufProvision getOutputBufProvision () const
 Queries the BufferProvision which this stream is capable of when producing tuples.
virtual bool canEarlyClose ()
 
Returns:
true if the stream can be closed early

ExecStreamGraph & getGraph () const
 
Returns:
reference to containing graph

ExecStreamId getStreamId () const
 
Returns:
the identifier for this stream within containing graph

virtual void getResourceRequirements (ExecStreamResourceQuantity &minQuantity, ExecStreamResourceQuantity &optQuantity)
virtual void setName (std::string const &)
 Sets unique name of this stream.
virtual std::string const & getName () const
 
Returns:
the name of this stream, as known by the optimizer

virtual bool mayBlock () const
 Queries whether this stream's implementation may block when execute() is called.
virtual void checkAbort () const
 Checks whether there is an abort request for this stream's scheduler.
virtual ExecStreamBufProvision getOutputBufConversion () const
 Queries the BufferProvision to which this stream needs its output to be converted, if any.
bool isClosed () const
 
Returns:
whether the object has been closed

void close ()
 Closes this object, releasing any unallocated resources.
virtual void initTraceSource (SharedTraceTarget pTraceTarget, std::string name)
 For use when initialization has to be deferred until after construction.
void trace (TraceLevel level, std::string message) const
 Records a trace message.
bool isTracing () const
 
Returns:
true iff tracing is enabled for this source

bool isTracingLevel (TraceLevel level) const
 Determines whether a particular level is being traced.
TraceTarget & getTraceTarget () const
 
Returns:
the TraceTarget for this source

SharedTraceTarget getSharedTraceTarget () const
 
Returns:
the SharedTraceTarget for this source

std::string getTraceSourceName () const
 Gets the name of this source.
void setTraceSourceName (std::string const &n)
 Sets the name of this source.
TraceLevel getMinimumTraceLevel () const
void disableTracing ()
virtual void initErrorSource (SharedErrorTarget pErrorTarget, const std::string &name)
 For use when initialization has to be deferred until after construction.
void postError (ErrorLevel level, const std::string &message, void *address, long capacity, int index)
 Posts an exception, such as a row exception.
void postError (ErrorLevel level, const std::string &message, const TupleDescriptor &errorDesc, const TupleData &errorTuple, int index)
 Posts an exception, such as a row exception.
bool hasTarget () const
 
Returns:
true iff an error target has been set

ErrorTarget & getErrorTarget () const
 
Returns:
the ErrorTarget for this source

SharedErrorTarget getSharedErrorTarget () const
 
Returns:
the SharedErrorTarget for this source

std::string getErrorSourceName () const
 Gets the name of this source.
void setErrorSourceName (std::string const &n)
 Sets the name of this source.
void disableTarget ()

Protected Attributes

std::vector< SharedExecStreamBufAccessor > inAccessors
SharedExecStreamBufAccessor pOutAccessor
bool isOpen
 Whether this stream is currently open.
ExecStreamGraph * pGraph
 Dataflow graph containing this stream.
ExecStreamId id
 Identifier for this stream; local to its containing graph.
std::string name
 Name of stream, as known by optimizer.
SharedDynamicParamManager pDynamicParamManager
 The dynamic parameter manager available to this stream.
SharedLogicalTxn pTxn
 The transaction embracing the stream.
ExecStreamResourceQuantity resourceAllocation
 Resource quantities currently allocated to this stream.
SharedCacheAccessor pQuotaAccessor
 CacheAccessor used for quota tracking.
SharedCacheAccessor pScratchQuotaAccessor
 CacheAccessor used for scratch page quota tracking.
bool needsClose

Private Member Functions

uint computeOptWorkspacePages (LcsRid maxRid)
 Compute the optimum number of pages for the union, based on the maximum number of rids in the table.
uint computeRidLimit (uint nWorkspacePages)
 Returns the maximum tuple size the workspace can handle and still produce segments of reasonable size.
bool isConsumerSridSet ()
 Whether stream has a parameter for consumer start rid.
bool isSegmentLimitSet ()
 Whether stream has a segment limit.
ExecStreamResult readSegment ()
 Reads a byte segment.
bool writeSegment ()
 Attempts to write a segment to the workspace.
void transferLast ()
 Transfers any remaining data from workspace to writer.
bool transfer ()
 Transfers data from workspace to writer.
bool produceTuple ()
 Produces an output tuple.

Private Attributes

LcsRid maxRid
DynamicParamId ridLimitParamId
DynamicParamId startRidParamId
DynamicParamId segmentLimitParamId
TupleDatum ridLimitDatum
 Tuple datum used to store dynamic paramter for ridLimit.
RecordNum ridLimit
 Number of rids that should appear in input tuples.
uint pageSize
 Usable page size.
uint nWorkspacePages
 Number of pages reserved for the workspace.
LbmSeqSegmentReader segmentReader
 Reads input tuples.
LbmUnionWorkspace workspace
 Workspace for merging segments.
LbmSegmentWriter segmentWriter
 Segment writer.
SegmentAccessor scratchAccessor
 Scratch accessor for allocating memory for output buffer.
SegPageLock workspacePageLock
 Lock on workspace pages.
SegPageLock writerPageLock
 Lock on writer scratch page.
TupleData inputTuple
 Input tuple data.
LbmByteSegment inputSegment
 Segment currently being read.
PBuffer reverseArea
 Scratch area to use for reversing output segments.
uint reverseAreaSize
TupleData outputTuple
 Output tuple data containing OR'd bitmap segments.
bool writePending
 True if a segment needs to be written to the workspace.
bool producePending
 True if a tuple needs to be written to the output stream.
bool isDone
 True if all input has been processed.
LcsRid requestedSrid
 Start rid requested by a consumer.
uint segmentsRemaining
 Number of segments remaining before hitting the production limit set by a consumer.

Detailed Description

LbmUnionExecStream is the execution stream used to perform a union on a stream of overlapping bitmap tuple.

Author:
John Pham
Version:
Id
//open/dev/fennel/lucidera/bitmap/LbmUnionExecStream.h#11

Definition at line 81 of file LbmUnionExecStream.h.

Member Function Documentation

uint LbmUnionExecStream::computeOptWorkspacePages ( LcsRid  maxRid  )  [private]

Compute the optimum number of pages for the union, based on the maximum number of rids in the table.

Definition at line 193 of file LbmUnionExecStream.cpp.

Referenced by getResourceRequirements().

00194 {
00195     // TODO: come up with a better estimate once we have statistics
00196     return 2;
00197 }

uint LbmUnionExecStream::computeRidLimit ( uint  nWorkspacePages  )  [private]

Returns the maximum tuple size the workspace can handle and still produce segments of reasonable size.

Definition at line 199 of file LbmUnionExecStream.cpp.

References LbmSegment::LbmOneByteSize, and pageSize.

Referenced by setResourceAllocation().

00200 {
00201     // save a quarter page for building segments
00202     // based upon the idea that the largest segment could be
00203     // 1/8 of a page along with 1/8 of a page for "growing" a
00204     // segment before writing it out (not true as of 2006-03-08)
00205     uint bytes = (uint) ((nWorkspacePages - 0.25) * pageSize);
00206     return bytes * LbmSegment::LbmOneByteSize;
00207 }

bool LbmUnionExecStream::isConsumerSridSet (  )  [private]

Whether stream has a parameter for consumer start rid.

Definition at line 209 of file LbmUnionExecStream.cpp.

References opaqueToInt(), and startRidParamId.

Referenced by execute().

00210 {
00211     return (opaqueToInt(startRidParamId) > 0);
00212 }

bool LbmUnionExecStream::isSegmentLimitSet (  )  [private]

Whether stream has a segment limit.

Definition at line 214 of file LbmUnionExecStream.cpp.

References opaqueToInt(), and segmentLimitParamId.

Referenced by execute(), and transfer().

00215 {
00216     return (opaqueToInt(segmentLimitParamId) > 0);
00217 }

ExecStreamResult LbmUnionExecStream::readSegment (  )  [private]

Reads a byte segment.

If the previous byte segment was not written, then the previous segment is returned.

Definition at line 219 of file LbmUnionExecStream.cpp.

References LbmByteSegment::byteNum, LbmByteSegment::byteSeg, EXECRC_YIELD, inputSegment, LbmByteSegment::len, LbmSeqSegmentReader::readSegmentAndAdvance(), segmentReader, and writePending.

Referenced by execute().

00220 {
00221     if (writePending) {
00222         return EXECRC_YIELD;
00223     }
00224     ExecStreamResult status = segmentReader.readSegmentAndAdvance(
00225         inputSegment.byteNum, inputSegment.byteSeg, inputSegment.len);
00226     if (status == EXECRC_YIELD) {
00227         writePending = true;
00228     }
00229     return status;
00230 }

bool LbmUnionExecStream::writeSegment (  )  [private]

Attempts to write a segment to the workspace.

First eagerly flushes contents of workspace. If contents cannot be flushed, returns false. Once contents have been flushed, write the segment to the workspace. This should always succeed due to the limit on tuple size.

Definition at line 232 of file LbmUnionExecStream.cpp.

References LbmUnionWorkspace::addSegment(), LbmUnionWorkspace::advanceToSrid(), LbmSeqSegmentReader::getSrid(), inputSegment, LbmUnionWorkspace::isEmpty(), segmentReader, LbmUnionWorkspace::setProductionLimit(), transfer(), workspace, and writePending.

Referenced by execute().

00233 {
00234     assert(writePending = true);
00235 
00236     // eagerly flush segments
00237     LcsRid currentSrid = segmentReader.getSrid();
00238     workspace.setProductionLimit(currentSrid);
00239     if (!transfer()) {
00240         return false;
00241     }
00242     if (workspace.isEmpty()) {
00243         workspace.advanceToSrid(currentSrid);
00244     }
00245 
00246     // flushing the workspace should make enough room for the next tuple
00247     bool success = workspace.addSegment(inputSegment);
00248     assert(success);
00249     writePending = false;
00250     return true;
00251 }

void LbmUnionExecStream::transferLast (  )  [private]

Transfers any remaining data from workspace to writer.

Transfers as much as possible, then yields.

Definition at line 253 of file LbmUnionExecStream.cpp.

References LbmUnionWorkspace::removeLimit(), transfer(), and workspace.

Referenced by execute().

00254 {
00255     workspace.removeLimit();
00256     transfer();
00257 }

bool LbmUnionExecStream::transfer (  )  [private]

Transfers data from workspace to writer.

Returns false if unable to transfer all completed workspace contents. For example, returns false if the writer is unable to accept more data or if yielding due to the limitation on number of segments.

Definition at line 259 of file LbmUnionExecStream.cpp.

References LbmSegmentWriter::addSegment(), LbmUnionWorkspace::advancePastSegment(), LbmByteSegment::byteSeg, LbmUnionWorkspace::canProduce(), LbmUnionWorkspace::getSegment(), LbmByteSegment::getSrid(), isSegmentLimitSet(), LbmByteSegment::len, reverseArea, reverseAreaSize, segmentsRemaining, segmentWriter, and workspace.

Referenced by transferLast(), and writeSegment().

00260 {
00261     while (workspace.canProduce()) {
00262         if (isSegmentLimitSet() && segmentsRemaining == 0) {
00263             return false;
00264         }
00265 
00266         LbmByteSegment seg = workspace.getSegment();
00267         assert(seg.len < reverseAreaSize);
00268         PBuffer reverseStart = reverseArea + seg.len - 1;
00269         for (int i = 0; i < seg.len; i++) {
00270             reverseStart[-i] = seg.byteSeg[i];
00271         }
00272         LcsRid startRid = seg.getSrid();
00273         if (! segmentWriter.addSegment(startRid, reverseArea, seg.len)) {
00274             return false;
00275         }
00276         workspace.advancePastSegment();
00277 
00278         if (isSegmentLimitSet()) {
00279             segmentsRemaining--;
00280         }
00281     }
00282     return true;
00283 }

bool LbmUnionExecStream::produceTuple (  )  [private]

Produces an output tuple.

Definition at line 285 of file LbmUnionExecStream.cpp.

References LbmSegmentWriter::isEmpty(), outputTuple, SingleOutputExecStream::pOutAccessor, producePending, LbmSegmentWriter::produceSegmentTuple(), LbmSegmentWriter::reset(), and segmentWriter.

Referenced by execute().

00286 {
00287     assert(producePending);
00288     assert(! segmentWriter.isEmpty());
00289 
00290     outputTuple = segmentWriter.produceSegmentTuple();
00291     if (pOutAccessor->produceTuple(outputTuple)) {
00292         segmentWriter.reset();
00293         producePending = false;
00294         return true;
00295     }
00296     return false;
00297 }

void LbmUnionExecStream::prepare ( LbmUnionExecStreamParams const &  params  )  [virtual]

Definition at line 30 of file LbmUnionExecStream.cpp.

References SegPageLock::accessSegment(), TupleDatum::cbData, TupleData::compute(), ConfluenceExecStream::inAccessors, inputTuple, LbmUnionExecStreamParams::maxRid, maxRid, opaqueToInt(), pageSize, TupleDatum::pData, SingleOutputExecStream::pOutAccessor, ConfluenceExecStream::prepare(), SegmentAccessor::pSegment, ridLimit, ridLimitDatum, LbmUnionExecStreamParams::ridLimitParamId, ridLimitParamId, scratchAccessor, LbmUnionExecStreamParams::segmentLimitParamId, segmentLimitParamId, LbmUnionExecStreamParams::startRidParamId, startRidParamId, workspacePageLock, and writerPageLock.

00031 {
00032     ConfluenceExecStream::prepare(params);
00033     maxRid = params.maxRid;
00034 
00035     // set dynanmic parameter ids
00036     ridLimitParamId = params.ridLimitParamId;
00037     assert(opaqueToInt(ridLimitParamId) > 0);
00038 
00039     // optional parameters
00040     startRidParamId = params.startRidParamId;
00041     segmentLimitParamId = params.segmentLimitParamId;
00042 
00043     // setup tupledatums for writing dynamic parameter values
00044     ridLimitDatum.pData = (PConstBuffer) &ridLimit;
00045     ridLimitDatum.cbData = sizeof(ridLimit);
00046 
00047     assert(inAccessors[0]->getTupleDesc() == pOutAccessor->getTupleDesc());
00048 
00049     // initialize reader
00050     inputTuple.compute(inAccessors[0]->getTupleDesc());
00051 
00052     // output buffer will come from scratch segment
00053     scratchAccessor = params.scratchAccessor;
00054     workspacePageLock.accessSegment(scratchAccessor);
00055     writerPageLock.accessSegment(scratchAccessor);
00056     pageSize = scratchAccessor.pSegment->getUsablePageSize();
00057 }

void LbmUnionExecStream::open ( bool  restart  )  [virtual]

Opens this stream, acquiring any resources needed in order to be able to fetch data.

A precondition is that input streams must already be opened. A stream can be closed and reopened.

Parameters:
restart if true, the stream must be already open, and should reset itself to start from the beginning of its result set

Reimplemented from ConfluenceExecStream.

Definition at line 84 of file LbmUnionExecStream.cpp.

References SegPageLock::allocatePage(), LbmEntry::getMaxBitmapSize(), SegPageLock::getPage(), LbmEntry::getScratchBufferSize(), CachePage::getWritableData(), ConfluenceExecStream::inAccessors, LbmSeqSegmentReader::init(), LbmUnionWorkspace::init(), ByteBuffer::init(), LbmSegmentWriter::init(), inputTuple, isDone, nWorkspacePages, ConfluenceExecStream::open(), pageSize, ExecStream::pDynamicParamManager, SingleOutputExecStream::pOutAccessor, producePending, SegmentAccessor::pSegment, LbmSegmentWriter::reset(), LbmUnionWorkspace::reset(), reverseArea, reverseAreaSize, ridLimitDatum, ridLimitParamId, scratchAccessor, segmentReader, segmentWriter, SegPageLock::unlock(), workspace, workspacePageLock, writePending, and writerPageLock.

00085 {
00086     ConfluenceExecStream::open(restart);
00087 
00088     if (!restart) {
00089         uint bitmapColSize = pOutAccessor->getTupleDesc()[1].cbStorage;
00090         uint writerBufSize = LbmEntry::getScratchBufferSize(bitmapColSize);
00091         writerPageLock.allocatePage();
00092         PBuffer writerBuf = writerPageLock.getPage().getWritableData();
00093         segmentWriter.init(
00094             writerBuf, writerBufSize, pOutAccessor->getTupleDesc(), false);
00095         // still have plenty of space for merging
00096         reverseArea = writerBuf + writerBufSize;
00097         reverseAreaSize =
00098             scratchAccessor.pSegment->getUsablePageSize() - writerBufSize;
00099 
00100         // allocate byte buffer for merging segments
00101         boost::shared_array<PBuffer> ppBuffers(new PBuffer[nWorkspacePages]);
00102         assert(ppBuffers != NULL);
00103         for (uint i = 0; i < nWorkspacePages; i++) {
00104             workspacePageLock.allocatePage();
00105             ppBuffers[i] = workspacePageLock.getPage().getWritableData();
00106             workspacePageLock.unlock();
00107         }
00108         ByteBuffer *pBuffer = new ByteBuffer();
00109         pBuffer->init(ppBuffers, nWorkspacePages, pageSize);
00110         SharedByteBuffer pWorkspaceBuffer(pBuffer);
00111         uint maxSegmentSize = LbmEntry::getMaxBitmapSize(bitmapColSize);
00112         workspace.init(pWorkspaceBuffer, maxSegmentSize);
00113 
00114         // create dynamic parameters
00115         pDynamicParamManager->createParam(
00116             ridLimitParamId, pOutAccessor->getTupleDesc()[0]);
00117         pDynamicParamManager->writeParam(ridLimitParamId, ridLimitDatum);
00118     } else {
00119         workspace.reset();
00120         segmentWriter.reset();
00121     }
00122 
00123     writePending = false;
00124     producePending = false;
00125     isDone = false;
00126     segmentReader.init(inAccessors[0], inputTuple);
00127 }

ExecStreamResult LbmUnionExecStream::execute ( ExecStreamQuantum const &  quantum  )  [virtual]

Executes this stream.

Parameters:
quantum governs the maximum amount of execution to perform
Returns:
code indicating reason execution ceased

Implements ExecStream.

Definition at line 129 of file LbmUnionExecStream.cpp.

References LbmUnionWorkspace::advanceToSrid(), EXECRC_BUF_OVERFLOW, EXECRC_EOS, EXECRC_QUANTUM_EXPIRED, EXECRC_YIELD, isConsumerSridSet(), isDone, LbmSegmentWriter::isEmpty(), LbmUnionWorkspace::isEmpty(), isSegmentLimitSet(), ExecStreamQuantum::nTuplesMax, ExecStream::pDynamicParamManager, SingleOutputExecStream::pOutAccessor, producePending, produceTuple(), readSegment(), requestedSrid, segmentLimitParamId, segmentsRemaining, segmentWriter, startRidParamId, transferLast(), workspace, and writeSegment().

00131 {
00132     if (isDone) {
00133         pOutAccessor->markEOS();
00134         return EXECRC_EOS;
00135     }
00136 
00137     if (isConsumerSridSet()) {
00138         // avoid RIDs not required by the downstream consumer
00139         requestedSrid = (LcsRid) *reinterpret_cast<RecordNum const *>(
00140             pDynamicParamManager->getParam(startRidParamId).getDatum().pData);
00141         workspace.advanceToSrid(requestedSrid);
00142     }
00143     if (isSegmentLimitSet()) {
00144         segmentsRemaining = *reinterpret_cast<uint const *>(
00145             pDynamicParamManager->getParam(segmentLimitParamId)
00146             .getDatum().pData);
00147     }
00148 
00149     for (uint i = 0; i < quantum.nTuplesMax; i++) {
00150         while (! producePending) {
00151             // yield control if segment limit is reached
00152             if (isSegmentLimitSet() && segmentsRemaining == 0) {
00153                 return EXECRC_QUANTUM_EXPIRED;
00154             }
00155 
00156             ExecStreamResult status = readSegment();
00157             if (status == EXECRC_EOS) {
00158                 // flush any remaining data as last tuple(s)
00159                 isDone = workspace.isEmpty() && segmentWriter.isEmpty();
00160                 if (! isDone) {
00161                     transferLast();
00162                     producePending = true;
00163                     break;
00164                 }
00165                 return EXECRC_EOS;
00166             }
00167             if (status != EXECRC_YIELD) {
00168                 return status;
00169             }
00170             if (! writeSegment()) {
00171                 producePending = (! segmentWriter.isEmpty());
00172             }
00173         }
00174 
00175         if (! produceTuple()) {
00176             return EXECRC_BUF_OVERFLOW;
00177         }
00178         producePending = false;
00179     }
00180     return EXECRC_QUANTUM_EXPIRED;
00181 }

void LbmUnionExecStream::getResourceRequirements ( ExecStreamResourceQuantity &  minQuantity,
ExecStreamResourceQuantity &  optQuantity,
ExecStreamResourceSettingType &  optType 
) [virtual]

Determines resource requirements for this stream.

Default implementation declares zero resource requirements.

Parameters:
minQuantity receives the minimum resource quantity needed by this stream in order to execute
optQuantity receives the resource quantity needed by this stream in order to execute optimally
optType Receives the value indicating the accuracy of the optQuantity parameter. This parameter is optional and defaults to EXEC_RESOURCE_ACCURATE if omitted. If the optimum setting is an estimate or no value can be specified (e.g., due to lack of statistics), then this parameter needs to be used to indicate a non-accurate optimum resource setting.

Reimplemented from ExecStream.

Definition at line 59 of file LbmUnionExecStream.cpp.

References computeOptWorkspacePages(), EXEC_RESOURCE_ESTIMATE, ExecStream::getResourceRequirements(), maxRid, and ExecStreamResourceQuantity::nCachePages.

00063 {
00064     ConfluenceExecStream::getResourceRequirements(minQuantity, optQuantity);
00065 
00066     // at least 2 scratch pages for constructing output bitmap segments
00067     //   - 1 for workspace
00068     //   - 1 for writer
00069     minQuantity.nCachePages += 2;
00070     optQuantity.nCachePages += 2 + computeOptWorkspacePages(maxRid) + 1;
00071     optType = EXEC_RESOURCE_ESTIMATE;
00072 }

void LbmUnionExecStream::setResourceAllocation ( ExecStreamResourceQuantity &  quantity  )  [virtual]

Sets current resource allocation for this stream.

If called while the stream is open, this indicates a request for the stream to dynamically adjust its memory usage. If the stream is incapable of honoring the request, it should update quantity with the actual amounts still in use.

Parameters:
quantity allocated resource quantity

Reimplemented from ExecStream.

Definition at line 74 of file LbmUnionExecStream.cpp.

References computeRidLimit(), ExecStreamResourceQuantity::nCachePages, nWorkspacePages, ridLimit, and ExecStream::setResourceAllocation().

00076 {
00077     ConfluenceExecStream::setResourceAllocation(quantity);
00078 
00079     // TODO: can we just grab all the remaining pages like this?
00080     nWorkspacePages = quantity.nCachePages - 1;
00081     ridLimit = computeRidLimit(nWorkspacePages);
00082 }

void LbmUnionExecStream::closeImpl (  )  [virtual]

Implements ClosableObject.

ExecStream implementations may override this to release any resources acquired while open.

Reimplemented from ExecStream.

Definition at line 183 of file LbmUnionExecStream.cpp.

References ExecStream::closeImpl(), NULL_PAGE_ID, SegmentAccessor::pSegment, and scratchAccessor.

00184 {
00185     ConfluenceExecStream::closeImpl();
00186 
00187     if (scratchAccessor.pSegment) {
00188         scratchAccessor.pSegment->deallocatePageRange(
00189             NULL_PAGE_ID, NULL_PAGE_ID);
00190     }
00191 }

void ConfluenceExecStream::prepare ( ConfluenceExecStreamParams const &  params  )  [virtual, inherited]

Definition at line 37 of file ConfluenceExecStream.cpp.

References ConfluenceExecStream::getInputBufProvision(), ConfluenceExecStream::inAccessors, and SingleOutputExecStream::prepare().

Referenced by LcsRowScanBaseExecStream::prepare(), prepare(), LbmChopperExecStream::prepare(), LbmBitOpExecStream::prepare(), LhxJoinExecStream::prepare(), MergeExecStream::prepare(), CorrelationJoinExecStream::prepare(), CartesianJoinExecStream::prepare(), and BarrierExecStream::prepare().

00038 {
00039     SingleOutputExecStream::prepare(params);
00040 
00041     for (uint i = 0; i < inAccessors.size(); ++i) {
00042         assert(inAccessors[i]->getProvision() == getInputBufProvision());
00043     }
00044 }

void SingleOutputExecStream::prepare ( SingleOutputExecStreamParams const &  params  )  [virtual, inherited]

Definition at line 48 of file SingleOutputExecStream.cpp.

References SingleOutputExecStream::getOutputBufProvision(), SingleOutputExecStreamParams::outputTupleDesc, SingleOutputExecStreamParams::outputTupleFormat, SingleOutputExecStream::pOutAccessor, and ExecStream::prepare().

Referenced by BTreeExecStream::prepare(), FlatFileExecStreamImpl::prepare(), ValuesExecStream::prepare(), MockResourceExecStream::prepare(), MockProducerExecStream::prepare(), ConfluenceExecStream::prepare(), and ConduitExecStream::prepare().

00049 {
00050     ExecStream::prepare(params);
00051     assert(pOutAccessor);
00052     assert(pOutAccessor->getProvision() == getOutputBufProvision());
00053     if (pOutAccessor->getTupleDesc().empty()) {
00054         assert(!params.outputTupleDesc.empty());
00055         pOutAccessor->setTupleShape(
00056             params.outputTupleDesc,
00057             params.outputTupleFormat);
00058     }
00059 }

void ExecStream::prepare ( ExecStreamParams const &  params  )  [virtual, inherited]

Prepares this stream for execution.

A precondition is that input streams must already be defined and prepared. As an effect of this call, the tuple shape should be defined for all output buffers and remain unchanged for the lifetime of the stream. This method is only ever called once, before the first open. Although this method is virtual, derived classes may choose to define an overloaded version instead with a specialized covariant parameter class.

Parameters:
params instance of stream parameterization class which should be used to prepare this stream

Definition at line 84 of file ExecStream.cpp.

References ExecStreamGraph::getDynamicParamManager(), SegmentAccessor::pCacheAccessor, ExecStreamParams::pCacheAccessor, ExecStream::pDynamicParamManager, ExecStream::pGraph, ExecStream::pQuotaAccessor, ExecStream::pScratchQuotaAccessor, and ExecStreamParams::scratchAccessor.

Referenced by JavaTransformExecStream::prepare(), SingleOutputExecStream::prepare(), and SingleInputExecStream::prepare().

00085 {
00086     if (pGraph) {
00087         pDynamicParamManager = pGraph->getDynamicParamManager();
00088     }
00089     pQuotaAccessor = params.pCacheAccessor;
00090     pScratchQuotaAccessor = params.scratchAccessor.pCacheAccessor;
00091 }

void ConfluenceExecStream::setInputBufAccessors ( std::vector< SharedExecStreamBufAccessor > const &  inAccessors  )  [virtual, inherited]

Initializes the buffer accessors for inputs to this stream.

This method is only ever called once, before prepare.

Parameters:
inAccessors buffer accessors ordered by input stream

Reimplemented from SingleOutputExecStream.

Definition at line 31 of file ConfluenceExecStream.cpp.

References ConfluenceExecStream::inAccessors.

00033 {
00034     inAccessors = inAccessorsInit;
00035 }

ExecStreamBufProvision ConfluenceExecStream::getInputBufProvision (  )  const [virtual, inherited]

Queries the BufferProvision which this stream requires of its inputs when consuming their tuples.

Returns:
required model; default is BUFPROV_NONE

Reimplemented from ExecStream.

Definition at line 58 of file ConfluenceExecStream.cpp.

References BUFPROV_PRODUCER.

Referenced by ConfluenceExecStream::prepare().

00059 {
00060     return BUFPROV_PRODUCER;
00061 }

void SingleOutputExecStream::setOutputBufAccessors ( std::vector< SharedExecStreamBufAccessor > const &  outAccessors  )  [virtual, inherited]

Initializes the buffer accessors for outputs from this stream.

This method is only ever called once, before prepare.

Parameters:
outAccessors buffer accessors ordered by output stream

Implements ExecStream.

Reimplemented in ConduitExecStream.

Definition at line 41 of file SingleOutputExecStream.cpp.

References SingleOutputExecStream::pOutAccessor.

Referenced by ConduitExecStream::setOutputBufAccessors().

00043 {
00044     assert(outAccessors.size() == 1);
00045     pOutAccessor = outAccessors[0];
00046 }

ExecStreamBufProvision SingleOutputExecStream::getOutputBufProvision (  )  const [virtual, inherited]

Queries the BufferProvision which this stream is capable of when producing tuples.

Returns:
supported model; default is BUFPROV_NONE

Reimplemented from ExecStream.

Reimplemented in BarrierExecStream, DoubleBufferExecStream, MergeExecStream, MockResourceExecStream, ScratchBufferExecStream, SegBufferExecStream, SegBufferReaderExecStream, ValuesExecStream, FtrsTableWriterExecStream, and LcsClusterAppendExecStream.

Definition at line 69 of file SingleOutputExecStream.cpp.

References BUFPROV_CONSUMER.

Referenced by SingleOutputExecStream::prepare().

00070 {
00071     return BUFPROV_CONSUMER;
00072 }

bool ExecStream::canEarlyClose (  )  [virtual, inherited]

Returns:
true if the stream can be closed early

Reimplemented in SegBufferWriterExecStream.

Definition at line 49 of file ExecStream.cpp.

00050 {
00051     return true;
00052 }

ExecStreamGraph & ExecStream::getGraph (  )  const [inline, inherited]

Returns:
reference to containing graph

Definition at line 293 of file ExecStream.h.

References ExecStream::pGraph.

Referenced by ExternalSortExecStreamImpl::execute(), JavaSinkExecStream::execute(), SegBufferWriterExecStream::execute(), SegBufferExecStream::execute(), BarrierExecStream::execute(), Java_net_sf_farrago_fennel_FennelStorage_tupleStreamFetch(), Java_net_sf_farrago_fennel_FennelStorage_tupleStreamTransformFetch(), JavaSinkExecStream::stuffByteBuffer(), and ExecStreamScheduler::traceStreamBuffers().

00294 {
00295     assert(pGraph);
00296     return *pGraph;
00297 }

ExecStreamId ExecStream::getStreamId (  )  const [inline, inherited]

Returns:
the identifier for this stream within containing graph

Definition at line 288 of file ExecStream.h.

References ExecStream::id.

Referenced by ExternalSortExecStreamImpl::execute(), SegBufferWriterExecStream::execute(), SegBufferExecStream::execute(), CorrelationJoinExecStream::execute(), BarrierExecStream::execute(), Java_net_sf_farrago_fennel_FennelStorage_tupleStreamTransformFetch(), JavaTransformExecStream::open(), SingleInputExecStream::open(), ConfluenceExecStream::open(), CartesianJoinExecStream::prepare(), ParallelExecStreamScheduler::readStream(), DfsTreeExecStreamScheduler::readStream(), LbmMinusExecStream::restartSubtrahends(), ExecStreamScheduler::tracePostExecution(), ExecStreamScheduler::tracePreExecution(), ExecStreamScheduler::traceStreamBuffers(), and ParallelExecStreamScheduler::tryExecuteTask().

00289 {
00290     return id;
00291 }

void ExecStream::getResourceRequirements ( ExecStreamResourceQuantity &  minQuantity,
ExecStreamResourceQuantity &  optQuantity 
) [virtual, inherited]

Reimplemented in DoubleBufferExecStream, ScratchBufferExecStream, SegBufferExecStream, SegBufferReaderExecStream, SegBufferWriterExecStream, FlatFileExecStreamImpl, BTreeInsertExecStream, BTreeReadExecStream, FtrsTableWriterExecStream, LbmChopperExecStream, LbmSplicerExecStream, LcsClusterAppendExecStream, LcsClusterReplaceExecStream, LcsRowScanBaseExecStream, and LcsRowScanExecStream.

Definition at line 102 of file ExecStream.cpp.

References ExecStreamResourceQuantity::nCachePages, and ExecStreamResourceQuantity::nThreads.

00105 {
00106     minQuantity.nThreads = 0;
00107     minQuantity.nCachePages = 0;
00108     optQuantity = minQuantity;
00109 }

void ExecStream::setName ( std::string const &   )  [virtual, inherited]

Sets unique name of this stream.

Definition at line 157 of file ExecStream.cpp.

References ExecStream::name.

00158 {
00159     name = nameInit;
00160 }

std::string const & ExecStream::getName (  )  const [virtual, inherited]

Returns:
the name of this stream, as known by the optimizer

Definition at line 162 of file ExecStream.cpp.

References ExecStream::name.

Referenced by DfsTreeExecStreamScheduler::findNextConsumer(), ParallelExecStreamScheduler::readStream(), DfsTreeExecStreamScheduler::readStream(), ExecStreamScheduler::tracePostExecution(), and ExecStreamScheduler::tracePreExecution().

00163 {
00164     return name;
00165 }

bool ExecStream::mayBlock (  )  const [virtual, inherited]

Queries whether this stream's implementation may block when execute() is called.

For accurate scheduling, non-blocking implementations are preferred; the scheduler must be aware of the potential for blocking so that it can allocate extra threads accordingly.

Returns:
whether stream may block; default is false

Definition at line 167 of file ExecStream.cpp.

00168 {
00169     return false;
00170 }

void ExecStream::checkAbort (  )  const [virtual, inherited]

Checks whether there is an abort request for this stream's scheduler.

Normally, streams don't need to check this, since the scheduler services abort requests in between quanta. However, streams which enter long-running loops need to check for themselves. If an abort is scheduled, this method will throw an AbortExcn automatically.

Definition at line 72 of file ExecStream.cpp.

References ExecStreamScheduler::checkAbort(), ExecStreamGraph::getScheduler(), and ExecStream::pGraph.

Referenced by LhxJoinExecStream::execute(), LhxAggExecStream::execute(), ExternalSortRunAccessor::fetch(), and ExternalSortMerger::fetch().

00073 {
00074     if (!pGraph) {
00075         return;
00076     }
00077     ExecStreamScheduler *pScheduler = pGraph->getScheduler();
00078     if (!pScheduler) {
00079         return;
00080     }
00081     pScheduler->checkAbort();
00082 }

ExecStreamBufProvision ExecStream::getOutputBufConversion (  )  const [virtual, inherited]

Queries the BufferProvision to which this stream needs its output to be converted, if any.

Returns:
required conversion; default is BUFPROV_NONE

Reimplemented in JavaTransformExecStream.

Definition at line 177 of file ExecStream.cpp.

References BUFPROV_NONE.

00178 {
00179     return BUFPROV_NONE;
00180 }

bool ClosableObject::isClosed (  )  const [inline, inherited]

Returns:
whether the object has been closed

Definition at line 58 of file ClosableObject.h.

00059     {
00060         return !needsClose;
00061     }

void ClosableObject::close (  )  [inherited]

Closes this object, releasing any unallocated resources.

Reimplemented in CollectExecStream, CorrelationJoinExecStream, LcsClusterAppendExecStream, and LcsClusterReplaceExecStream.

Definition at line 39 of file ClosableObject.cpp.

References ClosableObject::closeImpl(), and ClosableObject::needsClose.

Referenced by CacheImpl< PageT, VictimPolicyT >::allocatePages(), LcsRowScanBaseExecStream::closeImpl(), ExecStreamGraphImpl::closeImpl(), FlatFileBuffer::open(), ClosableObjectDestructor::operator()(), and Segment::~Segment().

00040 {
00041     if (!needsClose) {
00042         return;
00043     }
00044     needsClose = false;
00045     closeImpl();
00046 }

void TraceSource::initTraceSource ( SharedTraceTarget  pTraceTarget,
std::string  name 
) [virtual, inherited]

For use when initialization has to be deferred until after construction.

Parameters:
pTraceTarget the TraceTarget to which messages will be sent
name the name of this source

Definition at line 46 of file TraceSource.cpp.

References TraceSource::isTracing(), TraceSource::minimumLevel, TraceSource::name, TraceSource::pTraceTarget, and TRACE_OFF.

Referenced by TestBase::beforeTestCase(), TestBase::TestBase(), and TraceSource::TraceSource().

00049 {
00050     assert(!pTraceTarget.get());
00051 
00052     pTraceTarget = pTraceTargetInit;
00053     name = nameInit;
00054     if (isTracing()) {
00055         minimumLevel = pTraceTarget->getSourceTraceLevel(name);
00056     } else {
00057         minimumLevel = TRACE_OFF;
00058     }
00059 }

void TraceSource::trace ( TraceLevel  level,
std::string  message 
) const [inherited]

Records a trace message.

Normally only called via FENNEL_TRACE.

Parameters:
level severity level of event being trace
message the text of the message

Definition at line 61 of file TraceSource.cpp.

References TraceSource::getTraceTarget(), TraceSource::isTracing(), TraceSource::name, and TraceTarget::notifyTrace().

Referenced by Calculator::exec(), and ExecStreamScheduler::traceStreamBufferContents().

00062 {
00063     if (isTracing()) {
00064         getTraceTarget().notifyTrace(name,level,message);
00065     }
00066 }

bool TraceSource::isTracing (  )  const [inline, inherited]

Returns:
true iff tracing is enabled for this source

Definition at line 88 of file TraceSource.h.

Referenced by TraceSource::initTraceSource(), CalcExecStream::prepare(), and TraceSource::trace().

00089     {
00090         return pTraceTarget.get() ? true : false;
00091     }

bool TraceSource::isTracingLevel ( TraceLevel  level  )  const [inline, inherited]

Determines whether a particular level is being traced.

Parameters:
level trace level to test
Returns:
true iff tracing is enabled for the given level

Definition at line 100 of file TraceSource.h.

Referenced by ExecStreamScheduler::addGraph(), SimpleExecStreamGovernor::assignCachePages(), SimpleExecStreamGovernor::distributeCachePages(), Calculator::exec(), ExecStreamScheduler::ExecStreamScheduler(), LcsClusterNodeWriter::getLastClusterPageForWrite(), LcsClusterNodeWriter::moveFromTempToIndex(), JavaSinkExecStream::stuffByteBuffer(), and ExecStreamScheduler::traceStreamBuffers().

00101     {
00102         return level >= minimumLevel;
00103     }

TraceTarget& TraceSource::getTraceTarget (  )  const [inline, inherited]

Returns:
the TraceTarget for this source

Definition at line 108 of file TraceSource.h.

Referenced by TraceSource::trace().

00109     {
00110         assert(isTracing());
00111         return *(pTraceTarget.get());
00112     }

SharedTraceTarget TraceSource::getSharedTraceTarget (  )  const [inline, inherited]

Returns:
the SharedTraceTarget for this source

Definition at line 117 of file TraceSource.h.

Referenced by Database::init(), LcsClusterAppendExecStream::initLoad(), and CalcExecStream::prepare().

00118     {
00119         return pTraceTarget;
00120     }

std::string TraceSource::getTraceSourceName (  )  const [inline, inherited]

Gets the name of this source.

Useful to construct nested names for subcomponents that are also TraceSources.

Returns:
the name

Definition at line 127 of file TraceSource.h.

Referenced by LcsClusterAppendExecStream::initLoad().

00128     {
00129         return name;
00130     }

void TraceSource::setTraceSourceName ( std::string const &  n  )  [inline, inherited]

Sets the name of this source.

Useful to construct dynamic names for fine-grained filtering.

Definition at line 136 of file TraceSource.h.

00137     {
00138         name = n;
00139     }

TraceLevel TraceSource::getMinimumTraceLevel (  )  const [inline, inherited]

Definition at line 141 of file TraceSource.h.

00142     {
00143         return minimumLevel;
00144     }

void TraceSource::disableTracing (  )  [inherited]

Definition at line 68 of file TraceSource.cpp.

References TraceSource::minimumLevel, TraceSource::pTraceTarget, and TRACE_OFF.

Referenced by TestBase::afterTestCase().

00069 {
00070     pTraceTarget.reset();
00071     minimumLevel = TRACE_OFF;
00072 }

void ErrorSource::initErrorSource ( SharedErrorTarget  pErrorTarget,
const std::string &  name 
) [virtual, inherited]

For use when initialization has to be deferred until after construction.

Parameters:
pErrorTarget the ErrorTarget to which errors will be posted
name the name of this source

Definition at line 47 of file ErrorSource.cpp.

References ErrorSource::name, and ErrorSource::pErrorTarget.

Referenced by ErrorSource::ErrorSource().

00050 {
00051     pErrorTarget = pErrorTargetInit;
00052     name = nameInit;
00053 }

void ErrorSource::postError ( ErrorLevel  level,
const std::string &  message,
void *  address,
long  capacity,
int  index 
) [inherited]

Posts an exception, such as a row exception.

See also:
ErrorTarget for a description of the parameters

Definition at line 55 of file ErrorSource.cpp.

References ErrorSource::getErrorTarget(), ErrorSource::hasTarget(), ErrorSource::name, and ErrorTarget::notifyError().

Referenced by FlatFileExecStreamImpl::logError(), ErrorSource::postError(), and LbmSplicerExecStream::postViolation().

00058 {
00059     if (hasTarget()) {
00060         getErrorTarget().notifyError(
00061             name, level, message, address, capacity, index);
00062     }
00063 }

void ErrorSource::postError ( ErrorLevel  level,
const std::string &  message,
const TupleDescriptor &  errorDesc,
const TupleData &  errorTuple,
int  index 
) [inherited]

Posts an exception, such as a row exception.

See also:
ErrorTarget for a description of the parameters

Definition at line 65 of file ErrorSource.cpp.

References TupleAccessor::compute(), ErrorSource::errorAccessor, FixedBuffer, TupleAccessor::getByteCount(), TupleAccessor::getMaxByteCount(), ErrorSource::hasTarget(), TupleAccessor::marshal(), ErrorSource::pErrorBuf, and ErrorSource::postError().

00068 {
00069     if (!hasTarget()) {
00070         return;
00071     }
00072 
00073     if (!pErrorBuf) {
00074         errorAccessor.compute(errorDesc);
00075         uint cbMax = errorAccessor.getMaxByteCount();
00076         pErrorBuf.reset(new FixedBuffer[cbMax]);
00077     }
00078 
00079     uint cbTuple = errorAccessor.getByteCount(errorTuple);
00080     errorAccessor.marshal(errorTuple, pErrorBuf.get());
00081     postError(level, message, pErrorBuf.get(), cbTuple, index);
00082 }

bool ErrorSource::hasTarget (  )  const [inline, inherited]

Returns:
true iff an error target has been set

Definition at line 112 of file ErrorSource.h.

Referenced by ErrorSource::postError().

00113     {
00114         return pErrorTarget.get() ? true : false;
00115     }

ErrorTarget& ErrorSource::getErrorTarget (  )  const [inline, inherited]

Returns:
the ErrorTarget for this source

Definition at line 120 of file ErrorSource.h.

Referenced by ErrorSource::postError().

00121     {
00122         assert(hasTarget());
00123         return *(pErrorTarget.get());
00124     }

SharedErrorTarget ErrorSource::getSharedErrorTarget (  )  const [inline, inherited]

Returns:
the SharedErrorTarget for this source

Definition at line 129 of file ErrorSource.h.

00130     {
00131         return pErrorTarget;
00132     }

std::string ErrorSource::getErrorSourceName (  )  const [inline, inherited]

Gets the name of this source.

Useful to construct nested names for subcomponents that are also ErrorSources.

Returns:
the name

Definition at line 139 of file ErrorSource.h.

00140     {
00141         return name;
00142     }

void ErrorSource::setErrorSourceName ( std::string const &  n  )  [inline, inherited]

Sets the name of this source.

Useful to construct dynamic names for fine-grained filtering.

Definition at line 148 of file ErrorSource.h.

00149     {
00150         name = n;
00151     }

void ErrorSource::disableTarget (  )  [inherited]

Definition at line 84 of file ErrorSource.cpp.

References ErrorSource::pErrorTarget.

00085 {
00086     pErrorTarget.reset();
00087 }

Member Data Documentation

LcsRid LbmUnionExecStream::maxRid [private]

Definition at line 85 of file LbmUnionExecStream.h.

Referenced by getResourceRequirements(), and prepare().

DynamicParamId LbmUnionExecStream::ridLimitParamId [private]

Definition at line 86 of file LbmUnionExecStream.h.

Referenced by open(), and prepare().

DynamicParamId LbmUnionExecStream::startRidParamId [private]

Definition at line 87 of file LbmUnionExecStream.h.

Referenced by execute(), isConsumerSridSet(), and prepare().

DynamicParamId LbmUnionExecStream::segmentLimitParamId [private]

Definition at line 88 of file LbmUnionExecStream.h.

Referenced by execute(), isSegmentLimitSet(), and prepare().

TupleDatum LbmUnionExecStream::ridLimitDatum [private]

Tuple datum used to store dynamic paramter for ridLimit.

Definition at line 93 of file LbmUnionExecStream.h.

Referenced by open(), and prepare().

RecordNum LbmUnionExecStream::ridLimit [private]

Number of rids that should appear in input tuples.

Definition at line 98 of file LbmUnionExecStream.h.

Referenced by prepare(), and setResourceAllocation().

uint LbmUnionExecStream::pageSize [private]

Usable page size.

Definition at line 103 of file LbmUnionExecStream.h.

Referenced by computeRidLimit(), open(), and prepare().

uint LbmUnionExecStream::nWorkspacePages [private]

Number of pages reserved for the workspace.

Definition at line 108 of file LbmUnionExecStream.h.

Referenced by open(), and setResourceAllocation().

LbmSeqSegmentReader LbmUnionExecStream::segmentReader [private]

Reads input tuples.

Definition at line 113 of file LbmUnionExecStream.h.

Referenced by open(), readSegment(), and writeSegment().

LbmUnionWorkspace LbmUnionExecStream::workspace [private]

Workspace for merging segments.

Definition at line 118 of file LbmUnionExecStream.h.

Referenced by execute(), open(), transfer(), transferLast(), and writeSegment().

LbmSegmentWriter LbmUnionExecStream::segmentWriter [private]

Segment writer.

Definition at line 123 of file LbmUnionExecStream.h.

Referenced by execute(), open(), produceTuple(), and transfer().

SegmentAccessor LbmUnionExecStream::scratchAccessor [private]

Scratch accessor for allocating memory for output buffer.

Definition at line 128 of file LbmUnionExecStream.h.

Referenced by closeImpl(), open(), and prepare().

SegPageLock LbmUnionExecStream::workspacePageLock [private]

Lock on workspace pages.

Definition at line 133 of file LbmUnionExecStream.h.

Referenced by open(), and prepare().

SegPageLock LbmUnionExecStream::writerPageLock [private]

Lock on writer scratch page.

Definition at line 138 of file LbmUnionExecStream.h.

Referenced by open(), and prepare().

TupleData LbmUnionExecStream::inputTuple [private]

Input tuple data.

Definition at line 143 of file LbmUnionExecStream.h.

Referenced by open(), and prepare().

LbmByteSegment LbmUnionExecStream::inputSegment [private]

Segment currently being read.

Definition at line 148 of file LbmUnionExecStream.h.

Referenced by readSegment(), and writeSegment().

PBuffer LbmUnionExecStream::reverseArea [private]

Scratch area to use for reversing output segments.

Definition at line 153 of file LbmUnionExecStream.h.

Referenced by open(), and transfer().

uint LbmUnionExecStream::reverseAreaSize [private]

Definition at line 155 of file LbmUnionExecStream.h.

Referenced by open(), and transfer().

TupleData LbmUnionExecStream::outputTuple [private]

Output tuple data containing OR'd bitmap segments.

Definition at line 160 of file LbmUnionExecStream.h.

Referenced by produceTuple().

bool LbmUnionExecStream::writePending [private]

True if a segment needs to be written to the workspace.

Definition at line 165 of file LbmUnionExecStream.h.

Referenced by open(), readSegment(), and writeSegment().

bool LbmUnionExecStream::producePending [private]

True if a tuple needs to be written to the output stream.

Definition at line 170 of file LbmUnionExecStream.h.

Referenced by execute(), open(), and produceTuple().

bool LbmUnionExecStream::isDone [private]

True if all input has been processed.

Definition at line 175 of file LbmUnionExecStream.h.

Referenced by execute(), and open().

LcsRid LbmUnionExecStream::requestedSrid [private]

Start rid requested by a consumer.

Definition at line 180 of file LbmUnionExecStream.h.

Referenced by execute().

uint LbmUnionExecStream::segmentsRemaining [private]

Number of segments remaining before hitting the production limit set by a consumer.

Definition at line 186 of file LbmUnionExecStream.h.

Referenced by execute(), and transfer().

std::vector<SharedExecStreamBufAccessor> ConfluenceExecStream::inAccessors [protected, inherited]

Definition at line 50 of file ConfluenceExecStream.h.

Referenced by NestedLoopJoinExecStream::checkNumInputs(), CartesianJoinExecStream::checkNumInputs(), LbmMinusExecStream::comparePrefixes(), LbmGeneratorExecStream::execute(), MergeExecStream::execute(), BarrierExecStream::execute(), LbmMinusExecStream::findMinInput(), LcsRowScanExecStream::initializeFiltersIfNeeded(), LcsRowScanExecStream::open(), open(), LbmMinusExecStream::open(), LbmGeneratorExecStream::open(), LbmChopperExecStream::open(), LbmBitOpExecStream::open(), ConfluenceExecStream::open(), LcsRowScanExecStream::prepare(), prepare(), LbmMinusExecStream::prepare(), LbmGeneratorExecStream::prepare(), LbmChopperExecStream::prepare(), LbmBitOpExecStream::prepare(), LhxJoinExecStream::prepare(), MergeExecStream::prepare(), CorrelationJoinExecStream::prepare(), ConfluenceExecStream::prepare(), CartesianJoinExecStream::prepare(), BarrierExecStream::prepare(), NestedLoopJoinExecStream::preProcessRightInput(), BarrierExecStream::processInputTuple(), LbmBitOpExecStream::producePendingOutput(), LbmMinusExecStream::restartSubtrahends(), LhxJoinExecStream::setHashInfo(), and ConfluenceExecStream::setInputBufAccessors().

SharedExecStreamBufAccessor SingleOutputExecStream::pOutAccessor [protected, inherited]

Definition at line 56 of file SingleOutputExecStream.h.

Referenced by LcsClusterAppendExecStream::compress(), ExternalSortExecStreamImpl::execute(), LcsRowScanExecStream::execute(), execute(), LbmNormalizerExecStream::execute(), LbmGeneratorExecStream::execute(), LbmChopperExecStream::execute(), LhxJoinExecStream::execute(), LhxAggExecStream::execute(), FtrsTableWriterExecStream::execute(), BTreeSortExecStream::execute(), BTreeSearchUniqueExecStream::execute(), BTreeScanExecStream::execute(), BTreePrefetchSearchExecStream::execute(), BTreeInsertExecStream::execute(), FlatFileExecStreamImpl::execute(), ValuesExecStream::execute(), UncollectExecStream::execute(), SortedAggExecStream::execute(), SegBufferReaderExecStream::execute(), ScratchBufferExecStream::execute(), ReshapeExecStream::execute(), MockResourceExecStream::execute(), MockProducerExecStream::execute(), MergeExecStream::execute(), DoubleBufferExecStream::execute(), CorrelationJoinExecStream::execute(), CopyExecStream::execute(), CollectExecStream::execute(), CartesianJoinExecStream::execute(), BernoulliSamplingExecStream::execute(), BarrierExecStream::execute(), CalcExecStream::execute(), LbmGeneratorExecStream::flushEntry(), MockProducerExecStream::getProducedRowCount(), ExternalSortExecStreamImpl::getResourceRequirements(), BTreeSearchExecStream::innerFetchLoop(), open(), LbmChopperExecStream::open(), LbmBitOpExecStream::open(), SingleOutputExecStream::open(), SegBufferReaderExecStream::open(), SegBufferExecStream::open(), ScratchBufferExecStream::open(), DoubleBufferExecStream::open(), CollectExecStream::open(), SegBufferExecStream::openBufferForRead(), ConduitExecStream::precheckConduitBuffers(), LcsRowScanExecStream::prepare(), LcsRowScanBaseExecStream::prepare(), LcsClusterAppendExecStream::prepare(), prepare(), LbmGeneratorExecStream::prepare(), LhxJoinExecStream::prepare(), LhxAggExecStream::prepare(), FtrsTableWriterExecStream::prepare(), FlatFileExecStreamImpl::prepare(), UncollectExecStream::prepare(), SortedAggExecStream::prepare(), SingleOutputExecStream::prepare(), ReshapeExecStream::prepare(), MockResourceExecStream::prepare(), MockProducerExecStream::prepare(), MergeExecStream::prepare(), CorrelationJoinExecStream::prepare(), ConduitExecStream::prepare(), CollectExecStream::prepare(), CartesianJoinExecStream::prepare(), BernoulliSamplingExecStream::prepare(), BarrierExecStream::prepare(), CalcExecStream::prepare(), SortedAggExecStream::produce(), LbmBitOpExecStream::producePendingOutput(), produceTuple(), LbmNormalizerExecStream::produceTuple(), LbmMinusExecStream::produceTuple(), LbmChopperExecStream::produceTuple(), LbmBitOpExecStream::produceTuple(), LbmBitOpExecStream::readInput(), LbmMinusExecStream::readMinuendInput(), and SingleOutputExecStream::setOutputBufAccessors().

bool ExecStream::isOpen [protected, inherited]

Whether this stream is currently open.

Note that this is not quite the opposite of the inherited ClosableObject.needsClose, since a stream needs to be closed before destruction if it has been prepared but never opened.

Definition at line 61 of file ExecStream.h.

Referenced by ExecStream::closeImpl(), ExecStream::ExecStream(), and ExecStream::open().

ExecStreamGraph* ExecStream::pGraph [protected, inherited]

Dataflow graph containing this stream.

Note that we don't use a weak_ptr for this because it needs to be accessed frequently during execution, and the extra locking overhead would be frivolous.

Definition at line 68 of file ExecStream.h.

Referenced by ExecStream::checkAbort(), ExecStream::ExecStream(), CorrelationJoinExecStream::execute(), ExecStream::getGraph(), JavaTransformExecStream::open(), SingleInputExecStream::open(), MergeExecStream::open(), ExecStream::open(), CorrelationJoinExecStream::open(), ConfluenceExecStream::open(), ExecStream::prepare(), CartesianJoinExecStream::prepare(), and LbmMinusExecStream::restartSubtrahends().

ExecStreamId ExecStream::id [protected, inherited]

Identifier for this stream; local to its containing graph.

Definition at line 73 of file ExecStream.h.

Referenced by ExecStream::getStreamId().

std::string ExecStream::name [protected, inherited]

Name of stream, as known by optimizer.

Reimplemented from TraceSource.

Definition at line 78 of file ExecStream.h.

Referenced by ExecStream::ExecStream(), ExecStream::getName(), FlatFileExecStreamImpl::open(), and ExecStream::setName().

SharedDynamicParamManager ExecStream::pDynamicParamManager [protected, inherited]

The dynamic parameter manager available to this stream.

(Obtained at prepare() time. Keep a shared pointer in case the stream is reassigned to another graph for execution; cf ExecStreamGraph::mergeFrom())

Definition at line 85 of file ExecStream.h.

Referenced by CorrelationJoinExecStream::close(), SegBufferReaderExecStream::closeImpl(), execute(), LbmSplicerExecStream::execute(), LbmGeneratorExecStream::execute(), LbmChopperExecStream::execute(), CorrelationJoinExecStream::execute(), BarrierExecStream::execute(), LcsClusterReplaceExecStream::open(), open(), LbmSplicerExecStream::open(), LbmGeneratorExecStream::open(), LbmBitOpExecStream::open(), BTreeSearchExecStream::open(), BTreeInsertExecStream::open(), SegBufferWriterExecStream::open(), SegBufferReaderExecStream::open(), NestedLoopJoinExecStream::open(), CorrelationJoinExecStream::open(), ExecStream::prepare(), CalcExecStream::prepare(), NestedLoopJoinExecStream::processLeftInput(), LbmSearchExecStream::reachedTupleLimit(), ReshapeExecStream::readDynamicParams(), SegBufferWriterExecStream::readReaderRefCount(), BTreeSearchExecStream::readSearchKey(), BTreeSearchExecStream::readUpperBoundKey(), LbmSearchExecStream::setAdditionalKeys(), and LbmBitOpExecStream::writeStartRidParamValue().

SharedLogicalTxn ExecStream::pTxn [protected, inherited]

The transaction embracing the stream.

Obtained at open() time; but not released at close() time, to allow TableWriters to replay a txn. Keep a shared pointer in case the stream is reassigned to another graph for execution; cf ExecStreamGraph::mergeFrom())

Definition at line 94 of file ExecStream.h.

Referenced by FtrsTableWriterExecStream::commitSavepoint(), FtrsTableWriterExecStream::createSavepoint(), FtrsTableWriterExecStream::open(), ExecStream::open(), and FtrsTableWriterExecStream::rollbackSavepoint().

ExecStreamResourceQuantity ExecStream::resourceAllocation [protected, inherited]

Resource quantities currently allocated to this stream.

Definition at line 100 of file ExecStream.h.

Referenced by ExecStream::setResourceAllocation().

SharedCacheAccessor ExecStream::pQuotaAccessor [protected, inherited]

CacheAccessor used for quota tracking.

Definition at line 105 of file ExecStream.h.

Referenced by ExecStream::open(), ExecStream::prepare(), and ExecStream::setResourceAllocation().

SharedCacheAccessor ExecStream::pScratchQuotaAccessor [protected, inherited]

CacheAccessor used for scratch page quota tracking.

Definition at line 110 of file ExecStream.h.

Referenced by ExecStream::open(), ExecStream::prepare(), and ExecStream::setResourceAllocation().

bool ClosableObject::needsClose [protected, inherited]

Definition at line 44 of file ClosableObject.h.

Referenced by SegStreamAllocation::beginWrite(), ExecStreamGraphImpl::clear(), ClosableObject::ClosableObject(), ClosableObject::close(), FlatFileBuffer::open(), ExecStreamGraphImpl::open(), ExecStream::open(), and ClosableObject::~ClosableObject().

The documentation for this class was generated from the following files:
Generated on Mon Jun 22 04:00:36 2009 for Fennel by  doxygen 1.5.1