LbmBitOpExecStream Class Reference

LbmBitOpExecStream is a base class for implementing bit operation execution streams that read from N input streams. More...

#include <LbmBitOpExecStream.h>

Inheritance diagram for LbmBitOpExecStream:

List of all members.

Public Member Functions
virtual void	prepare (LbmBitOpExecStreamParams const &params)
virtual void	open (bool restart)
	Opens this stream, acquiring any resources needed in order to be able to fetch data.
virtual void	closeImpl ()
	Implements ClosableObject.
virtual void	prepare (ConfluenceExecStreamParams const &params)
virtual void	prepare (SingleOutputExecStreamParams const &params)
virtual void	prepare (ExecStreamParams const &params)
	Prepares this stream for execution.
virtual void	setInputBufAccessors (std::vector< SharedExecStreamBufAccessor > const &inAccessors)
	Initializes the buffer accessors for inputs to this stream.
virtual ExecStreamBufProvision	getInputBufProvision () const
	Queries the BufferProvision which this stream requires of its inputs when consuming their tuples.
virtual void	setOutputBufAccessors (std::vector< SharedExecStreamBufAccessor > const &outAccessors)
	Initializes the buffer accessors for outputs from this stream.
virtual ExecStreamBufProvision	getOutputBufProvision () const
	Queries the BufferProvision which this stream is capable of when producing tuples.
virtual bool	canEarlyClose ()
	Returns: true if the stream can be closed early
ExecStreamGraph &	getGraph () const
	Returns: reference to containing graph
ExecStreamId	getStreamId () const
	Returns: the identifier for this stream within containing graph
virtual void	getResourceRequirements (ExecStreamResourceQuantity &minQuantity, ExecStreamResourceQuantity &optQuantity, ExecStreamResourceSettingType &optType)
	Determines resource requirements for this stream.
virtual void	getResourceRequirements (ExecStreamResourceQuantity &minQuantity, ExecStreamResourceQuantity &optQuantity)
virtual void	setResourceAllocation (ExecStreamResourceQuantity &quantity)
	Sets current resource allocation for this stream.
virtual void	setName (std::string const &)
	Sets unique name of this stream.
virtual std::string const &	getName () const
	Returns: the name of this stream, as known by the optimizer
virtual ExecStreamResult	execute (ExecStreamQuantum const &quantum)=0
	Executes this stream.
virtual bool	mayBlock () const
	Queries whether this stream's implementation may block when execute() is called.
virtual void	checkAbort () const
	Checks whether there is an abort request for this stream's scheduler.
virtual ExecStreamBufProvision	getOutputBufConversion () const
	Queries the BufferProvision to which this stream needs its output to be converted, if any.
bool	isClosed () const
	Returns: whether the object has been closed
void	close ()
	Closes this object, releasing any unallocated resources.
virtual void	initTraceSource (SharedTraceTarget pTraceTarget, std::string name)
	For use when initialization has to be deferred until after construction.
void	trace (TraceLevel level, std::string message) const
	Records a trace message.
bool	isTracing () const
	Returns: true iff tracing is enabled for this source
bool	isTracingLevel (TraceLevel level) const
	Determines whether a particular level is being traced.
TraceTarget &	getTraceTarget () const
	Returns: the TraceTarget for this source
SharedTraceTarget	getSharedTraceTarget () const
	Returns: the SharedTraceTarget for this source
std::string	getTraceSourceName () const
	Gets the name of this source.
void	setTraceSourceName (std::string const &n)
	Sets the name of this source.
TraceLevel	getMinimumTraceLevel () const
void	disableTracing ()
virtual void	initErrorSource (SharedErrorTarget pErrorTarget, const std::string &name)
	For use when initialization has to be deferred until after construction.
void	postError (ErrorLevel level, const std::string &message, void *address, long capacity, int index)
	Posts an exception, such as a row exception.
void	postError (ErrorLevel level, const std::string &message, const TupleDescriptor &errorDesc, const TupleData &errorTuple, int index)
	Posts an exception, such as a row exception.
bool	hasTarget () const
	Returns: true iff an error target has been set
ErrorTarget &	getErrorTarget () const
	Returns: the ErrorTarget for this source
SharedErrorTarget	getSharedErrorTarget () const
	Returns: the SharedErrorTarget for this source
std::string	getErrorSourceName () const
	Gets the name of this source.
void	setErrorSourceName (std::string const &n)
	Sets the name of this source.
void	disableTarget ()
Protected Member Functions
ExecStreamResult	producePendingOutput (uint iInput)
	Produces output tuple that previously failed due to buffer overflow.
ExecStreamResult	readInput (uint iInput, LcsRid &currRid, PBuffer &currByteSeg, uint &currLen)
	Reads a byte segment from a specific input stream.
bool	flush ()
	Flushes the segment writer if it has any data.
bool	addSegments ()
	Adds the processed segments to the segment under construction.
void	writeStartRidParamValue ()
	Writes the startRid value to the startRid dynamic parameter, if one exists.
virtual bool	produceTuple (TupleData bitmapTuple)
	Produces a tuple to the output stream, based on a bitmap.
Protected Attributes
DynamicParamId	rowLimitParamId
	Parameter id representing the dynamic parameter used to limit the number of rows producers for this stream should produce on a single execute.
DynamicParamId	startRidParamId
	Parameter id representing the dynamic parameter used to set the starting rid value for bitmap entries to be produced by this stream's producers.
TupleDatum	rowLimitDatum
	Tuple datum used to store dynamic paramter for rowLimit.
TupleDatum	startRidDatum
	Tuple datum used to store dynamic parameter for startRid.
RecordNum	rowLimit
	Number of rows that can be produced.
LcsRid	startRid
	Desired starting rid value for bitmap entries.
boost::scoped_array< LbmSegmentReader >	segmentReaders
	One segment reader for each input stream.
uint	nInputs
	Number of input streams.
uint	iInput
	Current input stream being processed.
boost::scoped_array< TupleData >	bitmapSegTuples
	Tuple data for each input stream.
LbmSegmentWriter	segmentWriter
	Segment writer.
boost::scoped_array< FixedBuffer >	outputBuf
	Buffer for writing output bitmap segment.
boost::scoped_array< FixedBuffer >	byteSegBuf
	Temporary buffer for bit operation.
PBuffer	pByteSegBuf
	Pointer to byteSegBuf.
uint	bitmapBufSize
	Amount of space available in buffer for bitmaps.
TupleData	outputTuple
	Output tuple data containing AND'd bitmap segments.
bool	producePending
	True if a tuple needs to be written to the output stream.
LcsRid	addRid
	Current rid value to be added to the bitmap segment.
PBuffer	addByteSeg
	Current byte segment to be added.
uint	addLen
	Current length of byte segment to be added.
int	nFields
	Number of non-bitmap fields preceding the bitmap fields.
std::vector< SharedExecStreamBufAccessor >	inAccessors
SharedExecStreamBufAccessor	pOutAccessor
bool	isOpen
	Whether this stream is currently open.
ExecStreamGraph *	pGraph
	Dataflow graph containing this stream.
ExecStreamId	id
	Identifier for this stream; local to its containing graph.
std::string	name
	Name of stream, as known by optimizer.
SharedDynamicParamManager	pDynamicParamManager
	The dynamic parameter manager available to this stream.
SharedLogicalTxn	pTxn
	The transaction embracing the stream.
ExecStreamResourceQuantity	resourceAllocation
	Resource quantities currently allocated to this stream.
SharedCacheAccessor	pQuotaAccessor
	CacheAccessor used for quota tracking.
SharedCacheAccessor	pScratchQuotaAccessor
	CacheAccessor used for scratch page quota tracking.
bool	needsClose

---

Detailed Description

LbmBitOpExecStream is a base class for implementing bit operation execution streams that read from N input streams.

The first input stream and/or the output stream may be prefixed by key fields.

Author:

Zelaine Fong

Version:

Id

//open/dev/fennel/lucidera/bitmap/LbmBitOpExecStream.h#8

Definition at line 62 of file LbmBitOpExecStream.h.

---

Member Function Documentation

ExecStreamResult LbmBitOpExecStream::producePendingOutput

(

uint

iInput

)

[protected]

Produces output tuple that previously failed due to buffer overflow.

Writes out the remaining segments that could not fit in the previous buffer. Determines if input has reached EOS.

Parameters:

iInput

input to read to determine if EOS reached

Returns:

EXECRC_YIELD if successful

Definition at line 110 of file LbmBitOpExecStream.cpp.

References addSegments(), EXECBUF_EOS, EXECRC_BUF_OVERFLOW, EXECRC_EOS, EXECRC_YIELD, ConfluenceExecStream::inAccessors, LbmSegmentWriter::isEmpty(), outputTuple, SingleOutputExecStream::pOutAccessor, producePending, produceTuple(), LbmSegmentWriter::reset(), and segmentWriter.

Referenced by LbmMinusExecStream::execute(), and LbmIntersectExecStream::execute().

{
    if (!produceTuple(outputTuple)) {
        return EXECRC_BUF_OVERFLOW;
    }
    // in the middle of adding segments when buffer overflow occurred;
    // go back and add the remaining segments
    if (!segmentWriter.isEmpty()) {
        segmentWriter.reset();
        if (!addSegments()) {
            return EXECRC_BUF_OVERFLOW;
        }
    }
    producePending = false;
    if (inAccessors[iInput]->getState() == EXECBUF_EOS) {
        pOutAccessor->markEOS();
        return EXECRC_EOS;
    }

    return EXECRC_YIELD;
}

ExecStreamResult LbmBitOpExecStream::readInput	(	uint	iInput,
		LcsRid &	currRid,
		PBuffer &	currByteSeg,
		uint &	currLen
	)			[protected]

Reads a byte segment from a specific input stream.

Parameters:

	iInput	input to read
	currRid	startRid of the segment read
	currByteSeg	byte segment read; points to the beginning of segment that is stored backwards
	currLen	length of the byte segment read

Returns:

EXECRC_YIELD if read was successful

Definition at line 132 of file LbmBitOpExecStream.cpp.

References bitmapBufSize, EXECRC_BUF_OVERFLOW, EXECRC_EOS, EXECRC_YIELD, flush(), SingleOutputExecStream::pOutAccessor, segmentReaders, and startRid.

Referenced by LbmIntersectExecStream::execute(), and LbmMinusExecStream::readMinuendInputAndFlush().

{
    ExecStreamResult rc = segmentReaders[iInput].advanceToRid(startRid);

    if (rc == EXECRC_EOS) {
        // write out the last pending segment
        if (! flush()) {
            return EXECRC_BUF_OVERFLOW;
        }
        pOutAccessor->markEOS();
        return EXECRC_EOS;
    } else if (rc != EXECRC_YIELD) {
        return rc;
    }

    segmentReaders[iInput].readCurrentByteSegment(
        currRid, currByteSeg, currLen);
    // segment read should never be larger than space available
    // for segments
    assert(currLen <= bitmapBufSize);

    return EXECRC_YIELD;
}

bool LbmBitOpExecStream::flush

(

)

[protected]

Flushes the segment writer if it has any data.

The data is transferred to a bitmap tuple and the segment writer is reset. Finally, the tuple is produced to the output stream. If the tuple cannot be written, then it becomes a pending tuple.

Returns:

false if buffer overflow occured while producing the tuple

Definition at line 157 of file LbmBitOpExecStream.cpp.

References LbmSegmentWriter::isEmpty(), outputTuple, producePending, LbmSegmentWriter::produceSegmentTuple(), produceTuple(), LbmSegmentWriter::reset(), and segmentWriter.

Referenced by readInput(), LbmMinusExecStream::readMinuendInput(), and LbmMinusExecStream::readMinuendInputAndFlush().

{
    assert (!producePending);

    if (!segmentWriter.isEmpty()) {
        outputTuple = segmentWriter.produceSegmentTuple();
        segmentWriter.reset();
        if (!produceTuple(outputTuple)) {
            producePending = true;
        }
    }
    return !producePending;
 }

bool LbmBitOpExecStream::addSegments

(

)

[protected]

Adds the processed segments to the segment under construction.

If the segment fills up, writes it to the output buffer and continues constructing the rest of the segment. Leading, trailing, and intermediate zeros in the segment are removed.

Returns:

false if buffer overflow occurred writing out a segment

Definition at line 171 of file LbmBitOpExecStream.cpp.

References addByteSeg, addLen, addRid, LbmSegmentWriter::addSegment(), outputTuple, producePending, LbmSegmentWriter::produceSegmentTuple(), produceTuple(), LbmSegmentWriter::reset(), and segmentWriter.

Referenced by LbmMinusExecStream::execute(), LbmIntersectExecStream::intersectSegments(), and producePendingOutput().

{
    while (addLen > 0) {
        if (segmentWriter.addSegment(addRid, addByteSeg, addLen)) {
            break;
        }

        outputTuple = segmentWriter.produceSegmentTuple();
        if (!produceTuple(outputTuple)) {
            producePending = true;
            return false;
        }

        // loop back and start creating a new segment for the remainder of
        // the segments that wouldn't fit
        segmentWriter.reset();
    }

    return true;
}

void LbmBitOpExecStream::writeStartRidParamValue

(

)

[protected]

Writes the startRid value to the startRid dynamic parameter, if one exists.

Definition at line 205 of file LbmBitOpExecStream.cpp.

References opaqueToInt(), ExecStream::pDynamicParamManager, startRidDatum, and startRidParamId.

Referenced by LbmIntersectExecStream::execute(), LbmIntersectExecStream::intersectSegments(), open(), and LbmMinusExecStream::readMinuendInputAndFlush().

{
    if (opaqueToInt(startRidParamId) > 0) {
        pDynamicParamManager->writeParam(startRidParamId, startRidDatum);
    }
}

bool LbmBitOpExecStream::produceTuple

(

TupleData

bitmapTuple

)

[protected, virtual]

Produces a tuple to the output stream, based on a bitmap.

Returns:

false if buffer overflow occured while producing the tuple

Reimplemented in LbmMinusExecStream.

Definition at line 192 of file LbmBitOpExecStream.cpp.

References SingleOutputExecStream::pOutAccessor.

Referenced by addSegments(), flush(), and producePendingOutput().

{
    assert(pOutAccessor->getTupleDesc().size() == bitmapTuple.size());
    return pOutAccessor->produceTuple(bitmapTuple);
}

void LbmBitOpExecStream::prepare

(

LbmBitOpExecStreamParams const &

params

)

[virtual]

Definition at line 28 of file LbmBitOpExecStream.cpp.

References bitmapSegTuples, TupleDatum::cbData, ConfluenceExecStream::inAccessors, nFields, nInputs, TupleDatum::pData, ConfluenceExecStream::prepare(), rowLimit, rowLimitDatum, LbmBitOpExecStreamParams::rowLimitParamId, rowLimitParamId, segmentReaders, startRid, startRidDatum, LbmBitOpExecStreamParams::startRidParamId, and startRidParamId.

Referenced by LbmMinusExecStream::prepare(), and LbmIntersectExecStream::prepare().

{
    ConfluenceExecStream::prepare(params);

    // set dynanmic parameter ids
    rowLimitParamId = params.rowLimitParamId;
    startRidParamId = params.startRidParamId;

    // setup tupledatums for writing dynamic parameter values
    rowLimitDatum.pData = (PConstBuffer) &rowLimit;
    rowLimitDatum.cbData = sizeof(rowLimit);
    startRidDatum.pData = (PConstBuffer) &startRid;
    startRidDatum.cbData = sizeof(startRid);

    // initialize segment readers for reading bitmaps from input stream
    nInputs = inAccessors.size();
    segmentReaders.reset(new LbmSegmentReader[nInputs]);
    bitmapSegTuples.reset(new TupleData[nInputs]);
    for (uint i = 0; i < nInputs; i++) {
        bitmapSegTuples[i].compute(inAccessors[i]->getTupleDesc());
    }

    nFields = inAccessors[0]->getTupleDesc().size() - 3;
}

void LbmBitOpExecStream::open

(

bool

restart

)

[virtual]

Opens this stream, acquiring any resources needed in order to be able to fetch data.

A precondition is that input streams must already be opened. A stream can be closed and reopened.

Parameters:

restart

if true, the stream must be already open, and should reset itself to start from the beginning of its result set

Reimplemented from ConfluenceExecStream.

Reimplemented in LbmIntersectExecStream, and LbmMinusExecStream.

Definition at line 53 of file LbmBitOpExecStream.cpp.

References bitmapBufSize, bitmapSegTuples, byteSegBuf, FixedBuffer, LbmEntry::getMaxBitmapSize(), LbmEntry::getScratchBufferSize(), ConfluenceExecStream::inAccessors, LbmSegmentWriter::init(), nInputs, opaqueToInt(), ConfluenceExecStream::open(), outputBuf, pByteSegBuf, ExecStream::pDynamicParamManager, SingleOutputExecStream::pOutAccessor, producePending, LbmSegmentWriter::reset(), rowLimit, rowLimitDatum, rowLimitParamId, segmentReaders, segmentWriter, startRid, startRidParamId, and writeStartRidParamValue().

Referenced by LbmMinusExecStream::open(), and LbmIntersectExecStream::open().

{
    ConfluenceExecStream::open(restart);

    uint nKeys = pOutAccessor->getTupleDesc().size() - 3;

    if (!restart) {
        // create dynamic parameters with the same type as the first bitmap
        // field, a RID, if this is the first time open is called
        if (opaqueToInt(rowLimitParamId) > 0) {
            pDynamicParamManager->createParam(
                rowLimitParamId, pOutAccessor->getTupleDesc()[nKeys]);
        }
        if (opaqueToInt(startRidParamId) > 0) {
            pDynamicParamManager->createParam(
                startRidParamId, pOutAccessor->getTupleDesc()[nKeys]);
        }
    }

    // need to allocate buffers if this is the very first open, or if a
    // previous close freed up the buffers; note that we don't check "restart"
    // in case the stream was closed early, and then reopened in restart mode
    if (!outputBuf) {
        // allocate output buffer; the output buffer size is based on the size
        // required for building a LbmEntry
        uint bitmapColSize = pOutAccessor->getTupleDesc()[nKeys + 1].cbStorage;
        uint outputBufSize = LbmEntry::getScratchBufferSize(bitmapColSize);
        outputBuf.reset(new FixedBuffer[outputBufSize]);

        // initialize the writer to produce bitmap tuples; the second input
        // should be a bitmap input
        segmentWriter.init(
            outputBuf.get(), outputBufSize,
            inAccessors[1]->getTupleDesc(), true);

        // allocate a temporary buffer for the bit operation; the temporary
        // buffer should not be larger than what a LbmEntry supports
        bitmapBufSize = LbmEntry::getMaxBitmapSize(bitmapColSize);
        byteSegBuf.reset(new FixedBuffer[bitmapBufSize]);
        pByteSegBuf = byteSegBuf.get();

    } else {
        segmentWriter.reset();
    }

    startRid = LcsRid(0);
    rowLimit = 1;
    producePending = false;
    writeStartRidParamValue();
    if (opaqueToInt(rowLimitParamId) > 0) {
        pDynamicParamManager->writeParam(rowLimitParamId, rowLimitDatum);
    }
    for (uint i = 0; i < nInputs; i++) {
        segmentReaders[i].init(inAccessors[i], bitmapSegTuples[i]);
    }
}

void LbmBitOpExecStream::closeImpl

(

)

[virtual]

Implements ClosableObject.

ExecStream implementations may override this to release any resources acquired while open.

Reimplemented from ExecStream.

Reimplemented in LbmIntersectExecStream, and LbmMinusExecStream.

Definition at line 198 of file LbmBitOpExecStream.cpp.

References byteSegBuf, ExecStream::closeImpl(), and outputBuf.

Referenced by LbmMinusExecStream::closeImpl(), and LbmIntersectExecStream::closeImpl().

{
    ConfluenceExecStream::closeImpl();
    outputBuf.reset();
    byteSegBuf.reset();
}

void ConfluenceExecStream::prepare

(

ConfluenceExecStreamParams const &

params

)

[virtual, inherited]

Definition at line 37 of file ConfluenceExecStream.cpp.

References ConfluenceExecStream::getInputBufProvision(), ConfluenceExecStream::inAccessors, and SingleOutputExecStream::prepare().

Referenced by LcsRowScanBaseExecStream::prepare(), LbmUnionExecStream::prepare(), LbmChopperExecStream::prepare(), prepare(), LhxJoinExecStream::prepare(), MergeExecStream::prepare(), CorrelationJoinExecStream::prepare(), CartesianJoinExecStream::prepare(), and BarrierExecStream::prepare().

{
    SingleOutputExecStream::prepare(params);

    for (uint i = 0; i < inAccessors.size(); ++i) {
        assert(inAccessors[i]->getProvision() == getInputBufProvision());
    }
}

void SingleOutputExecStream::prepare

(

SingleOutputExecStreamParams const &

params

)

[virtual, inherited]

Definition at line 48 of file SingleOutputExecStream.cpp.

References SingleOutputExecStream::getOutputBufProvision(), SingleOutputExecStreamParams::outputTupleDesc, SingleOutputExecStreamParams::outputTupleFormat, SingleOutputExecStream::pOutAccessor, and ExecStream::prepare().

Referenced by BTreeExecStream::prepare(), FlatFileExecStreamImpl::prepare(), ValuesExecStream::prepare(), MockResourceExecStream::prepare(), MockProducerExecStream::prepare(), ConfluenceExecStream::prepare(), and ConduitExecStream::prepare().

{
    ExecStream::prepare(params);
    assert(pOutAccessor);
    assert(pOutAccessor->getProvision() == getOutputBufProvision());
    if (pOutAccessor->getTupleDesc().empty()) {
        assert(!params.outputTupleDesc.empty());
        pOutAccessor->setTupleShape(
            params.outputTupleDesc,
            params.outputTupleFormat);
    }
}

void ExecStream::prepare

(

ExecStreamParams const &

params

)

[virtual, inherited]

Prepares this stream for execution.

A precondition is that input streams must already be defined and prepared. As an effect of this call, the tuple shape should be defined for all output buffers and remain unchanged for the lifetime of the stream. This method is only ever called once, before the first open. Although this method is virtual, derived classes may choose to define an overloaded version instead with a specialized covariant parameter class.

Parameters:

params

instance of stream parameterization class which should be used to prepare this stream

Definition at line 84 of file ExecStream.cpp.

References ExecStreamGraph::getDynamicParamManager(), SegmentAccessor::pCacheAccessor, ExecStreamParams::pCacheAccessor, ExecStream::pDynamicParamManager, ExecStream::pGraph, ExecStream::pQuotaAccessor, ExecStream::pScratchQuotaAccessor, and ExecStreamParams::scratchAccessor.

Referenced by JavaTransformExecStream::prepare(), SingleOutputExecStream::prepare(), and SingleInputExecStream::prepare().

{
    if (pGraph) {
        pDynamicParamManager = pGraph->getDynamicParamManager();
    }
    pQuotaAccessor = params.pCacheAccessor;
    pScratchQuotaAccessor = params.scratchAccessor.pCacheAccessor;
}

void ConfluenceExecStream::setInputBufAccessors

(

std::vector< SharedExecStreamBufAccessor > const &

inAccessors

)

[virtual, inherited]

Initializes the buffer accessors for inputs to this stream.

This method is only ever called once, before prepare.

Parameters:

inAccessors

buffer accessors ordered by input stream

Reimplemented from SingleOutputExecStream.

Definition at line 31 of file ConfluenceExecStream.cpp.

References ConfluenceExecStream::inAccessors.

{
    inAccessors = inAccessorsInit;
}

ExecStreamBufProvision ConfluenceExecStream::getInputBufProvision

(

)

const [virtual, inherited]

Queries the BufferProvision which this stream requires of its inputs when consuming their tuples.

Returns:

required model; default is BUFPROV_NONE

Reimplemented from ExecStream.

Definition at line 58 of file ConfluenceExecStream.cpp.

References BUFPROV_PRODUCER.

Referenced by ConfluenceExecStream::prepare().

{
    return BUFPROV_PRODUCER;
}

void SingleOutputExecStream::setOutputBufAccessors

(

std::vector< SharedExecStreamBufAccessor > const &

outAccessors

)

[virtual, inherited]

Initializes the buffer accessors for outputs from this stream.

This method is only ever called once, before prepare.

Parameters:

outAccessors

buffer accessors ordered by output stream

Implements ExecStream.

Reimplemented in ConduitExecStream.

Definition at line 41 of file SingleOutputExecStream.cpp.

References SingleOutputExecStream::pOutAccessor.

Referenced by ConduitExecStream::setOutputBufAccessors().

{
    assert(outAccessors.size() == 1);
    pOutAccessor = outAccessors[0];
}

ExecStreamBufProvision SingleOutputExecStream::getOutputBufProvision

(

)

const [virtual, inherited]

Queries the BufferProvision which this stream is capable of when producing tuples.

Returns:

supported model; default is BUFPROV_NONE

Reimplemented from ExecStream.

Reimplemented in BarrierExecStream, DoubleBufferExecStream, MergeExecStream, MockResourceExecStream, ScratchBufferExecStream, SegBufferExecStream, SegBufferReaderExecStream, ValuesExecStream, FtrsTableWriterExecStream, and LcsClusterAppendExecStream.

Definition at line 69 of file SingleOutputExecStream.cpp.

References BUFPROV_CONSUMER.

Referenced by SingleOutputExecStream::prepare().

{
    return BUFPROV_CONSUMER;
}

bool ExecStream::canEarlyClose

(

)

[virtual, inherited]

Returns:

true if the stream can be closed early

Reimplemented in SegBufferWriterExecStream.

Definition at line 49 of file ExecStream.cpp.

{
    return true;
}

ExecStreamGraph & ExecStream::getGraph

(

)

const [inline, inherited]

Returns:

reference to containing graph

Definition at line 293 of file ExecStream.h.

References ExecStream::pGraph.

Referenced by ExternalSortExecStreamImpl::execute(), JavaSinkExecStream::execute(), SegBufferWriterExecStream::execute(), SegBufferExecStream::execute(), BarrierExecStream::execute(), Java_net_sf_farrago_fennel_FennelStorage_tupleStreamFetch(), Java_net_sf_farrago_fennel_FennelStorage_tupleStreamTransformFetch(), JavaSinkExecStream::stuffByteBuffer(), and ExecStreamScheduler::traceStreamBuffers().

{
    assert(pGraph);
    return *pGraph;
}

ExecStreamId ExecStream::getStreamId

(

)

const [inline, inherited]

Returns:

the identifier for this stream within containing graph

Definition at line 288 of file ExecStream.h.

References ExecStream::id.

Referenced by ExternalSortExecStreamImpl::execute(), SegBufferWriterExecStream::execute(), SegBufferExecStream::execute(), CorrelationJoinExecStream::execute(), BarrierExecStream::execute(), Java_net_sf_farrago_fennel_FennelStorage_tupleStreamTransformFetch(), JavaTransformExecStream::open(), SingleInputExecStream::open(), ConfluenceExecStream::open(), CartesianJoinExecStream::prepare(), ParallelExecStreamScheduler::readStream(), DfsTreeExecStreamScheduler::readStream(), LbmMinusExecStream::restartSubtrahends(), ExecStreamScheduler::tracePostExecution(), ExecStreamScheduler::tracePreExecution(), ExecStreamScheduler::traceStreamBuffers(), and ParallelExecStreamScheduler::tryExecuteTask().

{
    return id;
}

void ExecStream::getResourceRequirements	(	ExecStreamResourceQuantity &	minQuantity,
		ExecStreamResourceQuantity &	optQuantity,
		ExecStreamResourceSettingType &	optType
	)			[virtual, inherited]

Determines resource requirements for this stream.

Default implementation declares zero resource requirements.

Parameters:

	minQuantity	receives the minimum resource quantity needed by this stream in order to execute
	optQuantity	receives the resource quantity needed by this stream in order to execute optimally
	optType	Receives the value indicating the accuracy of the optQuantity parameter. This parameter is optional and defaults to EXEC_RESOURCE_ACCURATE if omitted. If the optimum setting is an estimate or no value can be specified (e.g., due to lack of statistics), then this parameter needs to be used to indicate a non-accurate optimum resource setting.

Reimplemented in MockResourceExecStream, BTreePrefetchSearchExecStream, LhxAggExecStream, LhxJoinExecStream, LbmGeneratorExecStream, LbmUnionExecStream, and ExternalSortExecStreamImpl.

Definition at line 93 of file ExecStream.cpp.

References EXEC_RESOURCE_ACCURATE.

Referenced by ExternalSortExecStreamImpl::getResourceRequirements(), LcsRowScanBaseExecStream::getResourceRequirements(), LcsClusterAppendExecStream::getResourceRequirements(), LbmUnionExecStream::getResourceRequirements(), LbmSplicerExecStream::getResourceRequirements(), LbmGeneratorExecStream::getResourceRequirements(), LbmChopperExecStream::getResourceRequirements(), LhxJoinExecStream::getResourceRequirements(), LhxAggExecStream::getResourceRequirements(), FtrsTableWriterExecStream::getResourceRequirements(), BTreeReadExecStream::getResourceRequirements(), BTreeInsertExecStream::getResourceRequirements(), FlatFileExecStreamImpl::getResourceRequirements(), SegBufferWriterExecStream::getResourceRequirements(), SegBufferReaderExecStream::getResourceRequirements(), SegBufferExecStream::getResourceRequirements(), ScratchBufferExecStream::getResourceRequirements(), and DoubleBufferExecStream::getResourceRequirements().

{
    getResourceRequirements(minQuantity, optQuantity);
    optType = EXEC_RESOURCE_ACCURATE;
}

void ExecStream::getResourceRequirements	(	ExecStreamResourceQuantity &	minQuantity,
		ExecStreamResourceQuantity &	optQuantity
	)			[virtual, inherited]

Reimplemented in DoubleBufferExecStream, ScratchBufferExecStream, SegBufferExecStream, SegBufferReaderExecStream, SegBufferWriterExecStream, FlatFileExecStreamImpl, BTreeInsertExecStream, BTreeReadExecStream, FtrsTableWriterExecStream, LbmChopperExecStream, LbmSplicerExecStream, LcsClusterAppendExecStream, LcsClusterReplaceExecStream, LcsRowScanBaseExecStream, and LcsRowScanExecStream.

Definition at line 102 of file ExecStream.cpp.

References ExecStreamResourceQuantity::nCachePages, and ExecStreamResourceQuantity::nThreads.

{
    minQuantity.nThreads = 0;
    minQuantity.nCachePages = 0;
    optQuantity = minQuantity;
}

void ExecStream::setResourceAllocation

(

ExecStreamResourceQuantity &

quantity

)

[virtual, inherited]

Sets current resource allocation for this stream.

If called while the stream is open, this indicates a request for the stream to dynamically adjust its memory usage. If the stream is incapable of honoring the request, it should update quantity with the actual amounts still in use.

Parameters:

quantity

allocated resource quantity

Reimplemented in MockResourceExecStream, BTreePrefetchSearchExecStream, LhxAggExecStream, LhxJoinExecStream, LbmGeneratorExecStream, LbmUnionExecStream, and ExternalSortExecStreamImpl.

Definition at line 111 of file ExecStream.cpp.

References ExecStreamResourceQuantity::nCachePages, ExecStream::pQuotaAccessor, ExecStream::pScratchQuotaAccessor, and ExecStream::resourceAllocation.

Referenced by ExternalSortExecStreamImpl::setResourceAllocation(), LbmUnionExecStream::setResourceAllocation(), LbmGeneratorExecStream::setResourceAllocation(), LhxJoinExecStream::setResourceAllocation(), LhxAggExecStream::setResourceAllocation(), and BTreePrefetchSearchExecStream::setResourceAllocation().

{
    resourceAllocation = quantity;
    if (pQuotaAccessor) {
        pQuotaAccessor->setMaxLockedPages(quantity.nCachePages);
    }
    if (pScratchQuotaAccessor) {
        pScratchQuotaAccessor->setMaxLockedPages(quantity.nCachePages);
    }
}

void ExecStream::setName

(

std::string const &

)

[virtual, inherited]

Sets unique name of this stream.

Definition at line 157 of file ExecStream.cpp.

References ExecStream::name.

{
    name = nameInit;
}

std::string const & ExecStream::getName

(

)

const [virtual, inherited]

Returns:

the name of this stream, as known by the optimizer

Definition at line 162 of file ExecStream.cpp.

References ExecStream::name.

Referenced by DfsTreeExecStreamScheduler::findNextConsumer(), ParallelExecStreamScheduler::readStream(), DfsTreeExecStreamScheduler::readStream(), ExecStreamScheduler::tracePostExecution(), and ExecStreamScheduler::tracePreExecution().

{
    return name;
}

virtual ExecStreamResult ExecStream::execute

(

ExecStreamQuantum const &

quantum

)

[pure virtual, inherited]

Executes this stream.

Parameters:

quantum

governs the maximum amount of execution to perform

Returns:

code indicating reason execution ceased

Implemented in CalcExecStream, BarrierExecStream, BernoulliSamplingExecStream, CartesianJoinExecStream, CollectExecStream, CopyExecStream, CorrelationJoinExecStream, DoubleBufferExecStream, MergeExecStream, MockConsumerExecStream, MockProducerExecStream, MockResourceExecStream, ReshapeExecStream, ScratchBufferExecStream, SegBufferExecStream, SegBufferReaderExecStream, SegBufferWriterExecStream, SortedAggExecStream, SplitterExecStream, UncollectExecStream, ValuesExecStream, JavaSinkExecStream, JavaTransformExecStream, FlatFileExecStreamImpl, BTreeInsertExecStream, BTreePrefetchSearchExecStream, BTreeScanExecStream, BTreeSearchExecStream, BTreeSearchUniqueExecStream, BTreeSortExecStream, FtrsTableWriterExecStream, LhxAggExecStream, LhxJoinExecStream, LbmChopperExecStream, LbmGeneratorExecStream, LbmIntersectExecStream, LbmMinusExecStream, LbmNormalizerExecStream, LbmSplicerExecStream, LbmUnionExecStream, LcsClusterAppendExecStream, LcsRowScanExecStream, and ExternalSortExecStreamImpl.

Referenced by ExecStreamScheduler::executeStream().

bool ExecStream::mayBlock

(

)

const [virtual, inherited]

Queries whether this stream's implementation may block when execute() is called.

For accurate scheduling, non-blocking implementations are preferred; the scheduler must be aware of the potential for blocking so that it can allocate extra threads accordingly.

Returns:

whether stream may block; default is false

Definition at line 167 of file ExecStream.cpp.

{
    return false;
}

void ExecStream::checkAbort

(

)

const [virtual, inherited]

Checks whether there is an abort request for this stream's scheduler.

Normally, streams don't need to check this, since the scheduler services abort requests in between quanta. However, streams which enter long-running loops need to check for themselves. If an abort is scheduled, this method will throw an AbortExcn automatically.

Definition at line 72 of file ExecStream.cpp.

References ExecStreamScheduler::checkAbort(), ExecStreamGraph::getScheduler(), and ExecStream::pGraph.

Referenced by LhxJoinExecStream::execute(), LhxAggExecStream::execute(), ExternalSortRunAccessor::fetch(), and ExternalSortMerger::fetch().

{
    if (!pGraph) {
        return;
    }
    ExecStreamScheduler *pScheduler = pGraph->getScheduler();
    if (!pScheduler) {
        return;
    }
    pScheduler->checkAbort();
}

ExecStreamBufProvision ExecStream::getOutputBufConversion

(

)

const [virtual, inherited]

Queries the BufferProvision to which this stream needs its output to be converted, if any.

Returns:

required conversion; default is BUFPROV_NONE

Reimplemented in JavaTransformExecStream.

Definition at line 177 of file ExecStream.cpp.

References BUFPROV_NONE.

{
    return BUFPROV_NONE;
}

bool ClosableObject::isClosed

(

)

const [inline, inherited]

Returns:

whether the object has been closed

Definition at line 58 of file ClosableObject.h.

    {
        return !needsClose;
    }

void ClosableObject::close

(

)

[inherited]

Closes this object, releasing any unallocated resources.

Reimplemented in CollectExecStream, CorrelationJoinExecStream, LcsClusterAppendExecStream, and LcsClusterReplaceExecStream.

Definition at line 39 of file ClosableObject.cpp.

References ClosableObject::closeImpl(), and ClosableObject::needsClose.

Referenced by CacheImpl< PageT, VictimPolicyT >::allocatePages(), LcsRowScanBaseExecStream::closeImpl(), ExecStreamGraphImpl::closeImpl(), FlatFileBuffer::open(), ClosableObjectDestructor::operator()(), and Segment::~Segment().

{
    if (!needsClose) {
        return;
    }
    needsClose = false;
    closeImpl();
}

void TraceSource::initTraceSource	(	SharedTraceTarget	pTraceTarget,
		std::string	name
	)			[virtual, inherited]

For use when initialization has to be deferred until after construction.

Parameters:

	pTraceTarget	the TraceTarget to which messages will be sent
	name	the name of this source

Definition at line 46 of file TraceSource.cpp.

References TraceSource::isTracing(), TraceSource::minimumLevel, TraceSource::name, TraceSource::pTraceTarget, and TRACE_OFF.

Referenced by TestBase::beforeTestCase(), TestBase::TestBase(), and TraceSource::TraceSource().

{
    assert(!pTraceTarget.get());

    pTraceTarget = pTraceTargetInit;
    name = nameInit;
    if (isTracing()) {
        minimumLevel = pTraceTarget->getSourceTraceLevel(name);
    } else {
        minimumLevel = TRACE_OFF;
    }
}

void TraceSource::trace	(	TraceLevel	level,
		std::string	message
	)			const [inherited]

Records a trace message.

Normally only called via FENNEL_TRACE.

Parameters:

	level	severity level of event being trace
	message	the text of the message

Definition at line 61 of file TraceSource.cpp.

References TraceSource::getTraceTarget(), TraceSource::isTracing(), TraceSource::name, and TraceTarget::notifyTrace().

Referenced by Calculator::exec(), and ExecStreamScheduler::traceStreamBufferContents().

{
    if (isTracing()) {
        getTraceTarget().notifyTrace(name,level,message);
    }
}

bool TraceSource::isTracing

(

)

const [inline, inherited]

Returns:

true iff tracing is enabled for this source

Definition at line 88 of file TraceSource.h.

Referenced by TraceSource::initTraceSource(), CalcExecStream::prepare(), and TraceSource::trace().

    {
        return pTraceTarget.get() ? true : false;
    }

bool TraceSource::isTracingLevel

(

TraceLevel

level

)

const [inline, inherited]

Determines whether a particular level is being traced.

Parameters:

level

trace level to test

Returns:

true iff tracing is enabled for the given level

Definition at line 100 of file TraceSource.h.

Referenced by ExecStreamScheduler::addGraph(), SimpleExecStreamGovernor::assignCachePages(), SimpleExecStreamGovernor::distributeCachePages(), Calculator::exec(), ExecStreamScheduler::ExecStreamScheduler(), LcsClusterNodeWriter::getLastClusterPageForWrite(), LcsClusterNodeWriter::moveFromTempToIndex(), JavaSinkExecStream::stuffByteBuffer(), and ExecStreamScheduler::traceStreamBuffers().

    {
        return level >= minimumLevel;
    }

TraceTarget& TraceSource::getTraceTarget

(

)

const [inline, inherited]

Returns:

the TraceTarget for this source

Definition at line 108 of file TraceSource.h.

Referenced by TraceSource::trace().

    {
        assert(isTracing());
        return *(pTraceTarget.get());
    }

SharedTraceTarget TraceSource::getSharedTraceTarget

(

)

const [inline, inherited]

Returns:

the SharedTraceTarget for this source

Definition at line 117 of file TraceSource.h.

Referenced by Database::init(), LcsClusterAppendExecStream::initLoad(), and CalcExecStream::prepare().

    {
        return pTraceTarget;
    }

std::string TraceSource::getTraceSourceName

(

)

const [inline, inherited]

Gets the name of this source.

Useful to construct nested names for subcomponents that are also TraceSources.

Returns:

the name

Definition at line 127 of file TraceSource.h.

Referenced by LcsClusterAppendExecStream::initLoad().

    {
        return name;
    }

void TraceSource::setTraceSourceName

(

std::string const &

n

)

[inline, inherited]

Sets the name of this source.

Useful to construct dynamic names for fine-grained filtering.

Definition at line 136 of file TraceSource.h.

    {
        name = n;
    }

TraceLevel TraceSource::getMinimumTraceLevel

(

)

const [inline, inherited]

Definition at line 141 of file TraceSource.h.

    {
        return minimumLevel;
    }

void TraceSource::disableTracing

(

)

[inherited]

Definition at line 68 of file TraceSource.cpp.

References TraceSource::minimumLevel, TraceSource::pTraceTarget, and TRACE_OFF.

Referenced by TestBase::afterTestCase().

{
    pTraceTarget.reset();
    minimumLevel = TRACE_OFF;
}

void ErrorSource::initErrorSource	(	SharedErrorTarget	pErrorTarget,
		const std::string &	name
	)			[virtual, inherited]

For use when initialization has to be deferred until after construction.

Parameters:

	pErrorTarget	the ErrorTarget to which errors will be posted
	name	the name of this source

Definition at line 47 of file ErrorSource.cpp.

References ErrorSource::name, and ErrorSource::pErrorTarget.

Referenced by ErrorSource::ErrorSource().

{
    pErrorTarget = pErrorTargetInit;
    name = nameInit;
}

void ErrorSource::postError	(	ErrorLevel	level,
		const std::string &	message,
		void *	address,
		long	capacity,
		int	index
	)			[inherited]

Posts an exception, such as a row exception.

See also:

ErrorTarget for a description of the parameters

Definition at line 55 of file ErrorSource.cpp.

References ErrorSource::getErrorTarget(), ErrorSource::hasTarget(), ErrorSource::name, and ErrorTarget::notifyError().

Referenced by FlatFileExecStreamImpl::logError(), ErrorSource::postError(), and LbmSplicerExecStream::postViolation().

{
    if (hasTarget()) {
        getErrorTarget().notifyError(
            name, level, message, address, capacity, index);
    }
}

void ErrorSource::postError	(	ErrorLevel	level,
		const std::string &	message,
		const TupleDescriptor &	errorDesc,
		const TupleData &	errorTuple,
		int	index
	)			[inherited]

Posts an exception, such as a row exception.

See also:

ErrorTarget for a description of the parameters

Definition at line 65 of file ErrorSource.cpp.

References TupleAccessor::compute(), ErrorSource::errorAccessor, FixedBuffer, TupleAccessor::getByteCount(), TupleAccessor::getMaxByteCount(), ErrorSource::hasTarget(), TupleAccessor::marshal(), ErrorSource::pErrorBuf, and ErrorSource::postError().

{
    if (!hasTarget()) {
        return;
    }

    if (!pErrorBuf) {
        errorAccessor.compute(errorDesc);
        uint cbMax = errorAccessor.getMaxByteCount();
        pErrorBuf.reset(new FixedBuffer[cbMax]);
    }

    uint cbTuple = errorAccessor.getByteCount(errorTuple);
    errorAccessor.marshal(errorTuple, pErrorBuf.get());
    postError(level, message, pErrorBuf.get(), cbTuple, index);
}

bool ErrorSource::hasTarget

(

)

const [inline, inherited]

Returns:

true iff an error target has been set

Definition at line 112 of file ErrorSource.h.

Referenced by ErrorSource::postError().

    {
        return pErrorTarget.get() ? true : false;
    }

ErrorTarget& ErrorSource::getErrorTarget

(

)

const [inline, inherited]

Returns:

the ErrorTarget for this source

Definition at line 120 of file ErrorSource.h.

Referenced by ErrorSource::postError().

    {
        assert(hasTarget());
        return *(pErrorTarget.get());
    }

SharedErrorTarget ErrorSource::getSharedErrorTarget

(

)

const [inline, inherited]

Returns:

the SharedErrorTarget for this source

Definition at line 129 of file ErrorSource.h.

    {
        return pErrorTarget;
    }

std::string ErrorSource::getErrorSourceName

(

)

const [inline, inherited]

Gets the name of this source.

Useful to construct nested names for subcomponents that are also ErrorSources.

Returns:

the name

Definition at line 139 of file ErrorSource.h.

    {
        return name;
    }

void ErrorSource::setErrorSourceName

(

std::string const &

n

)

[inline, inherited]

Sets the name of this source.

Useful to construct dynamic names for fine-grained filtering.

Definition at line 148 of file ErrorSource.h.

    {
        name = n;
    }

void ErrorSource::disableTarget

(

)

[inherited]

Definition at line 84 of file ErrorSource.cpp.

References ErrorSource::pErrorTarget.

{
    pErrorTarget.reset();
}

---

Member Data Documentation

DynamicParamId LbmBitOpExecStream::rowLimitParamId [protected]

Parameter id representing the dynamic parameter used to limit the number of rows producers for this stream should produce on a single execute.

Definition at line 71 of file LbmBitOpExecStream.h.

Referenced by open(), and prepare().

DynamicParamId LbmBitOpExecStream::startRidParamId [protected]

Parameter id representing the dynamic parameter used to set the starting rid value for bitmap entries to be produced by this stream's producers.

Definition at line 78 of file LbmBitOpExecStream.h.

Referenced by open(), prepare(), and writeStartRidParamValue().

TupleDatum LbmBitOpExecStream::rowLimitDatum [protected]

Tuple datum used to store dynamic paramter for rowLimit.

Definition at line 83 of file LbmBitOpExecStream.h.

Referenced by open(), and prepare().

TupleDatum LbmBitOpExecStream::startRidDatum [protected]

Tuple datum used to store dynamic parameter for startRid.

Definition at line 88 of file LbmBitOpExecStream.h.

Referenced by prepare(), and writeStartRidParamValue().

RecordNum LbmBitOpExecStream::rowLimit [protected]

Number of rows that can be produced.

Definition at line 93 of file LbmBitOpExecStream.h.

Referenced by LbmMinusExecStream::open(), open(), and prepare().

LcsRid LbmBitOpExecStream::startRid [protected]

Desired starting rid value for bitmap entries.

Definition at line 98 of file LbmBitOpExecStream.h.

Referenced by LbmMinusExecStream::checkNeedForRestart(), LbmMinusExecStream::execute(), LbmIntersectExecStream::execute(), LbmIntersectExecStream::intersectSegments(), open(), prepare(), readInput(), and LbmMinusExecStream::readMinuendInputAndFlush().

boost::scoped_array<LbmSegmentReader> LbmBitOpExecStream::segmentReaders [protected]

One segment reader for each input stream.

Definition at line 103 of file LbmBitOpExecStream.h.

Referenced by LbmMinusExecStream::advanceSingleSubtrahend(), LbmMinusExecStream::advanceSubtrahends(), LbmMinusExecStream::canSkipMinus(), LbmMinusExecStream::findMinInput(), LbmIntersectExecStream::intersectSegments(), LbmMinusExecStream::minusSegments(), open(), prepare(), readInput(), and LbmMinusExecStream::restartSubtrahends().

uint LbmBitOpExecStream::nInputs [protected]

Number of input streams.

Definition at line 108 of file LbmBitOpExecStream.h.

Referenced by LbmMinusExecStream::advanceSubtrahends(), LbmMinusExecStream::canSkipMinus(), LbmIntersectExecStream::execute(), LbmMinusExecStream::findMinInput(), LbmIntersectExecStream::intersectSegments(), open(), prepare(), and LbmMinusExecStream::restartSubtrahends().

uint LbmBitOpExecStream::iInput [protected]

Current input stream being processed.

Definition at line 113 of file LbmBitOpExecStream.h.

Referenced by LbmMinusExecStream::advanceSubtrahends(), LbmIntersectExecStream::execute(), LbmIntersectExecStream::open(), LbmMinusExecStream::readMinuendInputAndFlush(), and LbmMinusExecStream::restartSubtrahends().

boost::scoped_array<TupleData> LbmBitOpExecStream::bitmapSegTuples [protected]

Tuple data for each input stream.

Definition at line 118 of file LbmBitOpExecStream.h.

Referenced by LbmMinusExecStream::comparePrefixes(), LbmMinusExecStream::copyPrefix(), LbmMinusExecStream::open(), open(), prepare(), and LbmMinusExecStream::restartSubtrahends().

LbmSegmentWriter LbmBitOpExecStream::segmentWriter [protected]

Segment writer.

Definition at line 123 of file LbmBitOpExecStream.h.

Referenced by addSegments(), flush(), open(), and producePendingOutput().

boost::scoped_array<FixedBuffer> LbmBitOpExecStream::outputBuf [protected]

Buffer for writing output bitmap segment.

Definition at line 128 of file LbmBitOpExecStream.h.

Referenced by closeImpl(), and open().

boost::scoped_array<FixedBuffer> LbmBitOpExecStream::byteSegBuf [protected]

Temporary buffer for bit operation.

Definition at line 133 of file LbmBitOpExecStream.h.

Referenced by closeImpl(), and open().

PBuffer LbmBitOpExecStream::pByteSegBuf [protected]

Pointer to byteSegBuf.

Definition at line 138 of file LbmBitOpExecStream.h.

Referenced by LbmMinusExecStream::execute(), LbmIntersectExecStream::intersectSegments(), LbmMinusExecStream::minusSegments(), open(), and LbmMinusExecStream::readMinuendInputAndFlush().

uint LbmBitOpExecStream::bitmapBufSize [protected]

Amount of space available in buffer for bitmaps.

Definition at line 143 of file LbmBitOpExecStream.h.

Referenced by open(), readInput(), and LbmMinusExecStream::readMinuendInput().

TupleData LbmBitOpExecStream::outputTuple [protected]

Output tuple data containing AND'd bitmap segments.

Definition at line 148 of file LbmBitOpExecStream.h.

Referenced by addSegments(), flush(), and producePendingOutput().

bool LbmBitOpExecStream::producePending [protected]

True if a tuple needs to be written to the output stream.

Definition at line 153 of file LbmBitOpExecStream.h.

Referenced by addSegments(), LbmMinusExecStream::execute(), LbmIntersectExecStream::execute(), flush(), open(), and producePendingOutput().

LcsRid LbmBitOpExecStream::addRid [protected]

Current rid value to be added to the bitmap segment.

Definition at line 158 of file LbmBitOpExecStream.h.

Referenced by addSegments(), LbmMinusExecStream::execute(), and LbmIntersectExecStream::intersectSegments().

PBuffer LbmBitOpExecStream::addByteSeg [protected]

Current byte segment to be added.

Definition at line 163 of file LbmBitOpExecStream.h.

Referenced by addSegments(), LbmMinusExecStream::execute(), and LbmIntersectExecStream::intersectSegments().

uint LbmBitOpExecStream::addLen [protected]

Current length of byte segment to be added.

Definition at line 168 of file LbmBitOpExecStream.h.

Referenced by addSegments(), LbmMinusExecStream::execute(), and LbmIntersectExecStream::intersectSegments().

int LbmBitOpExecStream::nFields [protected]

Number of non-bitmap fields preceding the bitmap fields.

Definition at line 173 of file LbmBitOpExecStream.h.

Referenced by LbmMinusExecStream::checkNeedForRestart(), LbmMinusExecStream::comparePrefixes(), LbmMinusExecStream::copyPrefix(), LbmMinusExecStream::execute(), LbmMinusExecStream::open(), LbmMinusExecStream::prepare(), prepare(), LbmMinusExecStream::produceTuple(), LbmMinusExecStream::readMinuendInputAndFlush(), and LbmMinusExecStream::restartSubtrahends().

std::vector<SharedExecStreamBufAccessor> ConfluenceExecStream::inAccessors [protected, inherited]

Definition at line 50 of file ConfluenceExecStream.h.

Referenced by NestedLoopJoinExecStream::checkNumInputs(), CartesianJoinExecStream::checkNumInputs(), LbmMinusExecStream::comparePrefixes(), LbmGeneratorExecStream::execute(), MergeExecStream::execute(), BarrierExecStream::execute(), LbmMinusExecStream::findMinInput(), LcsRowScanExecStream::initializeFiltersIfNeeded(), LcsRowScanExecStream::open(), LbmUnionExecStream::open(), LbmMinusExecStream::open(), LbmGeneratorExecStream::open(), LbmChopperExecStream::open(), open(), ConfluenceExecStream::open(), LcsRowScanExecStream::prepare(), LbmUnionExecStream::prepare(), LbmMinusExecStream::prepare(), LbmGeneratorExecStream::prepare(), LbmChopperExecStream::prepare(), prepare(), LhxJoinExecStream::prepare(), MergeExecStream::prepare(), CorrelationJoinExecStream::prepare(), ConfluenceExecStream::prepare(), CartesianJoinExecStream::prepare(), BarrierExecStream::prepare(), NestedLoopJoinExecStream::preProcessRightInput(), BarrierExecStream::processInputTuple(), producePendingOutput(), LbmMinusExecStream::restartSubtrahends(), LhxJoinExecStream::setHashInfo(), and ConfluenceExecStream::setInputBufAccessors().

SharedExecStreamBufAccessor SingleOutputExecStream::pOutAccessor [protected, inherited]

Definition at line 56 of file SingleOutputExecStream.h.

Referenced by LcsClusterAppendExecStream::compress(), ExternalSortExecStreamImpl::execute(), LcsRowScanExecStream::execute(), LbmUnionExecStream::execute(), LbmNormalizerExecStream::execute(), LbmGeneratorExecStream::execute(), LbmChopperExecStream::execute(), LhxJoinExecStream::execute(), LhxAggExecStream::execute(), FtrsTableWriterExecStream::execute(), BTreeSortExecStream::execute(), BTreeSearchUniqueExecStream::execute(), BTreeScanExecStream::execute(), BTreePrefetchSearchExecStream::execute(), BTreeInsertExecStream::execute(), FlatFileExecStreamImpl::execute(), ValuesExecStream::execute(), UncollectExecStream::execute(), SortedAggExecStream::execute(), SegBufferReaderExecStream::execute(), ScratchBufferExecStream::execute(), ReshapeExecStream::execute(), MockResourceExecStream::execute(), MockProducerExecStream::execute(), MergeExecStream::execute(), DoubleBufferExecStream::execute(), CorrelationJoinExecStream::execute(), CopyExecStream::execute(), CollectExecStream::execute(), CartesianJoinExecStream::execute(), BernoulliSamplingExecStream::execute(), BarrierExecStream::execute(), CalcExecStream::execute(), LbmGeneratorExecStream::flushEntry(), MockProducerExecStream::getProducedRowCount(), ExternalSortExecStreamImpl::getResourceRequirements(), BTreeSearchExecStream::innerFetchLoop(), LbmUnionExecStream::open(), LbmChopperExecStream::open(), open(), SingleOutputExecStream::open(), SegBufferReaderExecStream::open(), SegBufferExecStream::open(), ScratchBufferExecStream::open(), DoubleBufferExecStream::open(), CollectExecStream::open(), SegBufferExecStream::openBufferForRead(), ConduitExecStream::precheckConduitBuffers(), LcsRowScanExecStream::prepare(), LcsRowScanBaseExecStream::prepare(), LcsClusterAppendExecStream::prepare(), LbmUnionExecStream::prepare(), LbmGeneratorExecStream::prepare(), LhxJoinExecStream::prepare(), LhxAggExecStream::prepare(), FtrsTableWriterExecStream::prepare(), FlatFileExecStreamImpl::prepare(), UncollectExecStream::prepare(), SortedAggExecStream::prepare(), SingleOutputExecStream::prepare(), ReshapeExecStream::prepare(), MockResourceExecStream::prepare(), MockProducerExecStream::prepare(), MergeExecStream::prepare(), CorrelationJoinExecStream::prepare(), ConduitExecStream::prepare(), CollectExecStream::prepare(), CartesianJoinExecStream::prepare(), BernoulliSamplingExecStream::prepare(), BarrierExecStream::prepare(), CalcExecStream::prepare(), SortedAggExecStream::produce(), producePendingOutput(), LbmUnionExecStream::produceTuple(), LbmNormalizerExecStream::produceTuple(), LbmMinusExecStream::produceTuple(), LbmChopperExecStream::produceTuple(), produceTuple(), readInput(), LbmMinusExecStream::readMinuendInput(), and SingleOutputExecStream::setOutputBufAccessors().

bool ExecStream::isOpen [protected, inherited]

Whether this stream is currently open.

Note that this is not quite the opposite of the inherited ClosableObject.needsClose, since a stream needs to be closed before destruction if it has been prepared but never opened.

Definition at line 61 of file ExecStream.h.

Referenced by ExecStream::closeImpl(), ExecStream::ExecStream(), and ExecStream::open().

ExecStreamGraph* ExecStream::pGraph [protected, inherited]

Dataflow graph containing this stream.

Note that we don't use a weak_ptr for this because it needs to be accessed frequently during execution, and the extra locking overhead would be frivolous.

Definition at line 68 of file ExecStream.h.

Referenced by ExecStream::checkAbort(), ExecStream::ExecStream(), CorrelationJoinExecStream::execute(), ExecStream::getGraph(), JavaTransformExecStream::open(), SingleInputExecStream::open(), MergeExecStream::open(), ExecStream::open(), CorrelationJoinExecStream::open(), ConfluenceExecStream::open(), ExecStream::prepare(), CartesianJoinExecStream::prepare(), and LbmMinusExecStream::restartSubtrahends().

ExecStreamId ExecStream::id [protected, inherited]

Identifier for this stream; local to its containing graph.

Definition at line 73 of file ExecStream.h.

Referenced by ExecStream::getStreamId().

std::string ExecStream::name [protected, inherited]

Name of stream, as known by optimizer.

Reimplemented from TraceSource.

Definition at line 78 of file ExecStream.h.

Referenced by ExecStream::ExecStream(), ExecStream::getName(), FlatFileExecStreamImpl::open(), and ExecStream::setName().

SharedDynamicParamManager ExecStream::pDynamicParamManager [protected, inherited]

The dynamic parameter manager available to this stream.

(Obtained at prepare() time. Keep a shared pointer in case the stream is reassigned to another graph for execution; cf ExecStreamGraph::mergeFrom())

Definition at line 85 of file ExecStream.h.

Referenced by CorrelationJoinExecStream::close(), SegBufferReaderExecStream::closeImpl(), LbmUnionExecStream::execute(), LbmSplicerExecStream::execute(), LbmGeneratorExecStream::execute(), LbmChopperExecStream::execute(), CorrelationJoinExecStream::execute(), BarrierExecStream::execute(), LcsClusterReplaceExecStream::open(), LbmUnionExecStream::open(), LbmSplicerExecStream::open(), LbmGeneratorExecStream::open(), open(), BTreeSearchExecStream::open(), BTreeInsertExecStream::open(), SegBufferWriterExecStream::open(), SegBufferReaderExecStream::open(), NestedLoopJoinExecStream::open(), CorrelationJoinExecStream::open(), ExecStream::prepare(), CalcExecStream::prepare(), NestedLoopJoinExecStream::processLeftInput(), LbmSearchExecStream::reachedTupleLimit(), ReshapeExecStream::readDynamicParams(), SegBufferWriterExecStream::readReaderRefCount(), BTreeSearchExecStream::readSearchKey(), BTreeSearchExecStream::readUpperBoundKey(), LbmSearchExecStream::setAdditionalKeys(), and writeStartRidParamValue().

SharedLogicalTxn ExecStream::pTxn [protected, inherited]

The transaction embracing the stream.

Obtained at open() time; but not released at close() time, to allow TableWriters to replay a txn. Keep a shared pointer in case the stream is reassigned to another graph for execution; cf ExecStreamGraph::mergeFrom())

Definition at line 94 of file ExecStream.h.

Referenced by FtrsTableWriterExecStream::commitSavepoint(), FtrsTableWriterExecStream::createSavepoint(), FtrsTableWriterExecStream::open(), ExecStream::open(), and FtrsTableWriterExecStream::rollbackSavepoint().

ExecStreamResourceQuantity ExecStream::resourceAllocation [protected, inherited]

Resource quantities currently allocated to this stream.

Definition at line 100 of file ExecStream.h.

Referenced by ExecStream::setResourceAllocation().

SharedCacheAccessor ExecStream::pQuotaAccessor [protected, inherited]

CacheAccessor used for quota tracking.

Definition at line 105 of file ExecStream.h.

Referenced by ExecStream::open(), ExecStream::prepare(), and ExecStream::setResourceAllocation().

SharedCacheAccessor ExecStream::pScratchQuotaAccessor [protected, inherited]

CacheAccessor used for scratch page quota tracking.

Definition at line 110 of file ExecStream.h.

Referenced by ExecStream::open(), ExecStream::prepare(), and ExecStream::setResourceAllocation().

bool ClosableObject::needsClose [protected, inherited]

Definition at line 44 of file ClosableObject.h.

Referenced by SegStreamAllocation::beginWrite(), ExecStreamGraphImpl::clear(), ClosableObject::ClosableObject(), ClosableObject::close(), FlatFileBuffer::open(), ExecStreamGraphImpl::open(), ExecStream::open(), and ClosableObject::~ClosableObject().

---

The documentation for this class was generated from the following files:

• /home/pub/open/dev/fennel/lucidera/bitmap/LbmBitOpExecStream.h
• /home/pub/open/dev/fennel/lucidera/bitmap/LbmBitOpExecStream.cpp

---