Fennel: ReshapeExecStream Class Reference

void ReshapeExecStream::initCompareData	(	ReshapeExecStreamParams const &	params,
		TupleDescriptor const &	inputDesc,
		TupleAccessor const &	inputAccessor
	)			`[private]`

Initializes tuple descriptors and data that will be used in comparisons.

Parameters:

	params	stream parameters
	inputDesc	descriptor of the input tuple stream
	inputAccessor	accessor for the input tuple stream

Definition at line 91 of file ReshapeExecStream.cpp.

References TupleProjectionAccessor::bind(), compTupleDesc, TupleData::compute(), copyCompareTuple(), dynamicParameters, inputCompareData, ReshapeExecStreamParams::inputCompareProj, inputCompareProjAccessor, isMAXU(), lastKey, lastKeyDesc, numCompDynParams, paramCompareData, ReshapeExecStreamParams::pCompTupleBuffer, and TupleDescriptor::projectFrom().

Referenced by prepare().

00095 {
00096     // Setup the comparison tuple descriptor
00097     assert(params.inputCompareProj.size() > 0);
00098     TupleProjection inputCompareProj = params.inputCompareProj;
00099     compTupleDesc.projectFrom(inputDesc, inputCompareProj);
00100     // Adjust the descriptor to allow NULLs in case we're filtering out NULLs
00101     for (uint i = 0; i < compTupleDesc.size(); i++) {
00102         compTupleDesc[i].isNullable = true;
00103     }
00104 
00105     // Setup the projection of the columns for comparison
00106     inputCompareProjAccessor.bind(inputAccessor, inputCompareProj);
00107     inputCompareData.compute(compTupleDesc);
00108 
00109     // Setup a descriptor that excludes the dynamic parameters that will
00110     // be used in comparisons, if there are dynamic parameters.  The dynamic
00111     // parameters appear at the end of the descriptor.
00112     TupleDescriptor partialCompTupleDesc;
00113     numCompDynParams = 0;
00114     for (uint i = 0; i < dynamicParameters.size(); i++) {
00115         if (!isMAXU(dynamicParameters[i].compareOffset)) {
00116             numCompDynParams++;
00117         }
00118     }
00119     if (numCompDynParams > 0) {
00120         partialCompTupleDesc.resize(compTupleDesc.size() - numCompDynParams);
00121         std::copy(
00122             compTupleDesc.begin(),
00123             compTupleDesc.end() - numCompDynParams,
00124             partialCompTupleDesc.begin());
00125     }
00126 
00127     paramCompareData.compute(compTupleDesc);
00128     if (numCompDynParams == 0) {
00129         copyCompareTuple(
00130             compTupleDesc,
00131             paramCompareData,
00132             params.pCompTupleBuffer.get());
00133     } else if (partialCompTupleDesc.size() > 0) {
00134         TupleData partialCompareData;
00135         partialCompareData.compute(partialCompTupleDesc);
00136         copyCompareTuple(
00137             partialCompTupleDesc,
00138             partialCompareData,
00139             params.pCompTupleBuffer.get());
00140 
00141         // Copy the partial tuple data to the tuple data that will
00142         // be used in the actual comparisons
00143         std::copy(
00144             partialCompareData.begin(),
00145             partialCompareData.end(),
00146             paramCompareData.begin());
00147     }
00148 
00149     // Setup a tuple projection to project the last key for use in
00150     // non-equality comparisons
00151     lastKey.push_back(paramCompareData.size() - 1);
00152     lastKeyDesc.projectFrom(compTupleDesc, lastKey);
00153 }

void ReshapeExecStream::copyCompareTuple	(	TupleDescriptor const &	tupleDesc,
		TupleData &	tupleData,
		PBuffer	tupleBuffer
	)			`[private]`

Copies the comparison tuple from the input buffer and unmarshals it into a specified TupleData.

Parameters:

	tupleDesc	TupleDescriptor corresponding to the TupleData
	tupleData	TupleData that the tuple will be unmarshalled into
	tupleBuffer	input tuple buffer

Definition at line 155 of file ReshapeExecStream.cpp.

References compTupleBuffer, TupleAccessor::compute(), FixedBuffer, TupleAccessor::getCurrentByteCount(), TupleAccessor::setCurrentTupleBuf(), and TupleAccessor::unmarshal().

Referenced by initCompareData().

00159 {
00160     TupleAccessor tupleAccessor;
00161     tupleAccessor.compute(tupleDesc);
00162     tupleAccessor.setCurrentTupleBuf(tupleBuffer);
00163     uint nBytes = tupleAccessor.getCurrentByteCount();
00164     compTupleBuffer.reset(new FixedBuffer[nBytes]);
00165     memcpy(compTupleBuffer.get(), tupleBuffer, nBytes);
00166     tupleAccessor.setCurrentTupleBuf(compTupleBuffer.get());
00167     tupleAccessor.unmarshal(tupleData);
00168 }

bool ReshapeExecStream::checkCastTypes	(	const TupleProjection &	compareProj,
		const TupleDescriptor &	inputTupleDesc,
		const TupleDescriptor &	outputTupleDesc
	)			`[private]`

Verifies that in the case where simple casting is required, the types of the input and output meet the restrictions on the type of casting supported.

Parameters:

	compareProj	projection representing the input columns that have filters on them
	inputTupleDesc	tuple descriptor for the input columns
	outputTupleDesc	tuple descriptor for the output columns

Returns:: always returns true

Definition at line 170 of file ReshapeExecStream.cpp.

References nullFilter(), STANDARD_TYPE_CHAR, STANDARD_TYPE_UNICODE_CHAR, STANDARD_TYPE_UNICODE_VARCHAR, and STANDARD_TYPE_VARCHAR.

Referenced by prepare().

00174 {
00175     for (uint i = 0; i < inputTupleDesc.size(); i++) {
00176         if (!(inputTupleDesc[i] == outputTupleDesc[i])) {
00177             // only allow not nullable -> nullable, unless nulls are being
00178             // filtered out from that column
00179             if (inputTupleDesc[i].isNullable &&
00180                 !outputTupleDesc[i].isNullable)
00181             {
00182                 assert(nullFilter(compareProj, i));
00183             } else {
00184                 assert(
00185                     (inputTupleDesc[i].isNullable ==
00186                         outputTupleDesc[i].isNullable)
00187                     || (!inputTupleDesc[i].isNullable
00188                         && outputTupleDesc[i].isNullable));
00189             }
00190             StoredTypeDescriptor::Ordinal inputType =
00191                 inputTupleDesc[i].pTypeDescriptor->getOrdinal();
00192             StoredTypeDescriptor::Ordinal outputType =
00193                 outputTupleDesc[i].pTypeDescriptor->getOrdinal();
00194 
00195             // can't convert between unicode and non-unicode;
00196             // normalize types to non-unicode to make other checks
00197             // easier, but verify that either both or neither are unicode
00198             bool inputUnicode = false;
00199             if (inputType == STANDARD_TYPE_UNICODE_CHAR) {
00200                 inputType = STANDARD_TYPE_CHAR;
00201                 inputUnicode = true;
00202             }
00203             if (inputType == STANDARD_TYPE_UNICODE_VARCHAR) {
00204                 inputType = STANDARD_TYPE_VARCHAR;
00205                 inputUnicode = true;
00206             }
00207 
00208             bool outputUnicode = false;
00209             if (outputType == STANDARD_TYPE_UNICODE_CHAR) {
00210                 outputType = STANDARD_TYPE_CHAR;
00211                 outputUnicode = true;
00212             }
00213             if (outputType == STANDARD_TYPE_UNICODE_VARCHAR) {
00214                 outputType = STANDARD_TYPE_VARCHAR;
00215                 outputUnicode = true;
00216             }
00217 
00218             if (inputUnicode || outputUnicode) {
00219                 assert(inputUnicode && outputUnicode);
00220             }
00221 
00222             if (inputType != outputType) {
00223                 // if types are different, must be casting from char to
00224                 // varchar
00225                 assert(
00226                     (inputType == STANDARD_TYPE_CHAR)
00227                     && (outputType == STANDARD_TYPE_VARCHAR));
00228             }
00229             if (inputTupleDesc[i].cbStorage != outputTupleDesc[i].cbStorage) {
00230                 // if lengths are different, must be casting from char or
00231                 // varchar to varchar
00232                 assert(
00233                     ((inputType == STANDARD_TYPE_VARCHAR)
00234                         || (inputType == STANDARD_TYPE_CHAR))
00235                     && (outputType == STANDARD_TYPE_VARCHAR));
00236             }
00237         }
00238     }
00239     return true;
00240 }

bool ReshapeExecStream::nullFilter	(	const TupleProjection &	compareProj,
		uint	colno
	)			`[private]`

Determines whether a column has a null filter on it.

Parameters:

	compareProj	projection representing the input columns that have filters on them
	colno	column number being checked

Returns:: true if the column does have a null filter check on it

Definition at line 242 of file ReshapeExecStream.cpp.

References paramCompareData.

Referenced by checkCastTypes().

00244 {
00245     for (uint i = 0; i < compareProj.size(); i++) {
00246         if (compareProj[i] == colno) {
00247             if (!paramCompareData[i].pData) {
00248                 return true;
00249             } else {
00250                 break;
00251             }
00252         }
00253     }
00254     return false;
00255 }

void ReshapeExecStream::readDynamicParams ( ) [private]

Reads the dynamic parameters and setups the comparison and output tuple datas to point to the parameter tuple datas.

Note that the parameter values are accessed by reference, so it's expected that the parameter values will remain fixed during stream execution.

Definition at line 317 of file ReshapeExecStream.cpp.

References dynamicParameters, inputOutputDesc, isMAXU(), numCompDynParams, outputData, paramCompareData, and ExecStream::pDynamicParamManager.

Referenced by execute().

00318 {
00319     uint currCompIdx = paramCompareData.size() - numCompDynParams;
00320     uint currOutputIdx = inputOutputDesc.size();
00321     for (uint i = 0; i < dynamicParameters.size(); i++) {
00322         if (!isMAXU(dynamicParameters[i].compareOffset)) {
00323             TupleDatum const &param =
00324                 pDynamicParamManager->getParam(
00325                     dynamicParameters[i].dynamicParamId).getDatum();
00326             paramCompareData[currCompIdx++] = param;
00327         }
00328         if (dynamicParameters[i].outputParam) {
00329             TupleDatum const &param =
00330                 pDynamicParamManager->getParam(
00331                     dynamicParameters[i].dynamicParamId).getDatum();
00332             outputData[currOutputIdx++] = param;
00333         }
00334     }
00335 }

bool ReshapeExecStream::compareInput ( ) [private]

Compares input data against the base comparison tuple.

Returns:: true if the data returns true for the comparison operation

Definition at line 337 of file ReshapeExecStream.cpp.

References COMP_EQ, COMP_GE, COMP_GT, COMP_LE, COMP_LT, COMP_NE, TupleDescriptor::compareTuples(), TupleDescriptor::compareTuplesKey(), compOp, compTupleDesc, inputCompareData, inputCompareProjAccessor, lastKey, lastKeyDesc, paramCompareData, and TupleProjectionAccessor::unmarshal().

Referenced by execute().

00338 {
00339     inputCompareProjAccessor.unmarshal(inputCompareData);
00340     int rc;
00341 
00342     // if the comparison is non-equality, first compare the first n-1 keys
00343     // for equality; if those keys are equal, then do the non-equality
00344     // comparison on just the last key
00345     if (compOp == COMP_EQ) {
00346         rc = compTupleDesc.compareTuples(inputCompareData, paramCompareData);
00347     } else {
00348         rc =
00349             compTupleDesc.compareTuplesKey(
00350                 inputCompareData, paramCompareData,
00351                 paramCompareData.size() - 1);
00352         if (rc != 0) {
00353             return false;
00354         }
00355         // ignore NULLs if doing a comparison against a non-NULL value
00356         if (!paramCompareData[paramCompareData.size() - 1].pData) {
00357             rc =
00358                 lastKeyDesc.compareTuples(
00359                     inputCompareData, lastKey, paramCompareData, lastKey);
00360         } else {
00361             bool containsNullKey;
00362             rc =
00363                 lastKeyDesc.compareTuples(
00364                     inputCompareData, lastKey, paramCompareData, lastKey,
00365                     &containsNullKey);
00366             if (containsNullKey) {
00367                 return false;
00368             }
00369         }
00370     }
00371 
00372     bool pass;
00373     switch (compOp) {
00374     case COMP_EQ:
00375         pass = (rc == 0);
00376         break;
00377     case COMP_NE:
00378         pass = (rc != 0);
00379         break;
00380     case COMP_LT:
00381         pass = (rc < 0);
00382         break;
00383     case COMP_LE:
00384         pass = (rc <= 0);
00385         break;
00386     case COMP_GT:
00387         pass = (rc > 0);
00388         break;
00389     case COMP_GE:
00390         pass = (rc >= 0);
00391         break;
00392     default:
00393         pass = false;
00394         permAssert(false);
00395     }
00396     return pass;
00397 }

void ReshapeExecStream::castOutput ( ) [private]

Copies the columns from the input tuple to the output tuple, performing truncation as part of casting, if needed.

Definition at line 399 of file ReshapeExecStream.cpp.

References inputOutputData, min(), outputData, outputDesc, outputProjAccessor, and TupleProjectionAccessor::unmarshal().

Referenced by execute().

00400 {
00401     outputProjAccessor.unmarshal(inputOutputData);
00402     for (uint i = 0; i < inputOutputData.size(); i++) {
00403         // truncate value if it exceeds the destination size
00404         uint len = std::min(
00405             inputOutputData[i].cbData, outputDesc[i].cbStorage);
00406         outputData[i].cbData = len;
00407         if (inputOutputData[i].pData) {
00408             outputData[i].pData = inputOutputData[i].pData;
00409         } else {
00410             outputData[i].pData = NULL;
00411         }
00412     }
00413 }

void ReshapeExecStream::prepare ( ReshapeExecStreamParams const & params ) [virtual]

Definition at line 33 of file ReshapeExecStream.cpp.

References TupleProjectionAccessor::bind(), castRequired, checkCastTypes(), COMP_NE, COMP_NOOP, ReshapeExecStreamParams::compOp, compOp, TupleData::compute(), ReshapeExecStreamParams::dynamicParameters, dynamicParameters, initCompareData(), ReshapeExecStreamParams::inputCompareProj, inputOutputData, inputOutputDesc, outputData, outputDesc, ReshapeExecStreamParams::outputProj, outputProjAccessor, SingleInputExecStream::pInAccessor, SingleOutputExecStream::pOutAccessor, ConduitExecStream::prepare(), and TupleDescriptor::projectFrom().

00034 {
00035     ConduitExecStream::prepare(params);
00036 
00037     TupleDescriptor const &inputDesc = pInAccessor->getTupleDesc();
00038     TupleAccessor &inputAccessor = pInAccessor->getConsumptionTupleAccessor();
00039     dynamicParameters.assign(
00040         params.dynamicParameters.begin(),
00041         params.dynamicParameters.end());
00042 
00043     compOp = params.compOp;
00044     if (compOp != COMP_NOOP) {
00045         initCompareData(params, inputDesc, inputAccessor);
00046     }
00047 
00048     // Setup the output projection that projects from the input tuple
00049     outputProjAccessor.bind(inputAccessor, params.outputProj);
00050     inputOutputDesc.projectFrom(inputDesc, params.outputProj);
00051 
00052     // Setup the output descriptor and data
00053     outputDesc = pOutAccessor->getTupleDesc();
00054     outputData.compute(outputDesc);
00055 
00056     // Determine how many of the dynamic parameters need to be written into
00057     // the output tuple
00058     uint numOutputDynParams = 0;
00059     for (uint i = 0; i < dynamicParameters.size(); i++) {
00060         if (dynamicParameters[i].outputParam) {
00061             numOutputDynParams++;
00062         }
00063     }
00064 
00065     // Setup the output descriptor and data that excludes dynamic parameters
00066     // in the output tuple
00067     assert(inputOutputDesc.size() == outputDesc.size() - numOutputDynParams);
00068     TupleDescriptor partialOutputDesc;
00069     if (numOutputDynParams == 0) {
00070         partialOutputDesc = outputDesc;
00071     } else if (inputOutputDesc.size() > 0) {
00072         partialOutputDesc.resize(inputOutputDesc.size());
00073         std::copy(
00074             outputDesc.begin(),
00075             outputDesc.end() - numOutputDynParams,
00076             partialOutputDesc.begin());
00077     }
00078 
00079     // determine if simple casting is required
00080     castRequired = (inputOutputDesc != partialOutputDesc);
00081     if (castRequired) {
00082         TupleProjection proj;
00083         if (compOp == COMP_NE) {
00084             proj = params.inputCompareProj;
00085         }
00086         assert(checkCastTypes(proj, inputOutputDesc, partialOutputDesc));
00087         inputOutputData.compute(inputOutputDesc);
00088     }
00089 }

void ReshapeExecStream::open ( bool restart ) [virtual]

Opens this stream, acquiring any resources needed in order to be able to fetch data.

A precondition is that input streams must already be opened. A stream can be closed and reopened.

Parameters:

restart

if true, the stream must be already open, and should reset itself to start from the beginning of its result set

Reimplemented from ConduitExecStream.

Definition at line 257 of file ReshapeExecStream.cpp.

References ConduitExecStream::open(), paramsRead, and producePending.

00258 {
00259     ConduitExecStream::open(restart);
00260     producePending = false;
00261     paramsRead = false;
00262 }

ExecStreamResult ReshapeExecStream::execute ( ExecStreamQuantum const & quantum ) [virtual]

Executes this stream.

Parameters:

quantum

governs the maximum amount of execution to perform

Returns:: code indicating reason execution ceased

Implements ExecStream.

Definition at line 264 of file ReshapeExecStream.cpp.

References castOutput(), castRequired, COMP_NOOP, compareInput(), compOp, EXECRC_BUF_OVERFLOW, EXECRC_BUF_UNDERFLOW, EXECRC_QUANTUM_EXPIRED, EXECRC_YIELD, ExecStreamQuantum::nTuplesMax, outputData, outputProjAccessor, paramsRead, SingleInputExecStream::pInAccessor, SingleOutputExecStream::pOutAccessor, ConduitExecStream::precheckConduitBuffers(), producePending, readDynamicParams(), and TupleProjectionAccessor::unmarshal().

00266 {
00267     if (!paramsRead) {
00268         readDynamicParams();
00269         paramsRead = true;
00270     }
00271 
00272     ExecStreamResult rc = precheckConduitBuffers();
00273     if (rc != EXECRC_YIELD) {
00274         return rc;
00275     }
00276 
00277     if (producePending) {
00278         if (!pOutAccessor->produceTuple(outputData)) {
00279             return EXECRC_BUF_OVERFLOW;
00280         }
00281         pInAccessor->consumeTuple();
00282         producePending = false;
00283     }
00284 
00285     for (uint i = 0; i < quantum.nTuplesMax; i++) {
00286         if (!pInAccessor->demandData()) {
00287             return EXECRC_BUF_UNDERFLOW;
00288         }
00289 
00290         pInAccessor->accessConsumptionTuple();
00291 
00292         // filter the data, if filtering criteria provided
00293         if (compOp != COMP_NOOP) {
00294             bool pass = compareInput();
00295             if (!pass) {
00296                 pInAccessor->consumeTuple();
00297                 continue;
00298             }
00299         }
00300 
00301         if (castRequired) {
00302             castOutput();
00303         } else {
00304             outputProjAccessor.unmarshal(outputData);
00305         }
00306         producePending = true;
00307         if (!pOutAccessor->produceTuple(outputData)) {
00308             return EXECRC_BUF_OVERFLOW;
00309         }
00310         producePending = false;
00311         pInAccessor->consumeTuple();
00312     }
00313 
00314     return EXECRC_QUANTUM_EXPIRED;
00315 }

void ReshapeExecStream::closeImpl ( ) [virtual]

Implements ClosableObject.

ExecStream implementations may override this to release any resources acquired while open.

Reimplemented from ExecStream.

Definition at line 415 of file ReshapeExecStream.cpp.

References ExecStream::closeImpl().

00416 {
00417     ConduitExecStream::closeImpl();
00418 }

ExecStreamResult ConduitExecStream::precheckConduitBuffers ( ) [protected, inherited]

Checks the state of the input and output buffers.

If input empty, requests production. If input EOS, propagates that to output buffer. If output full, returns EXECRC_OVERFLOW.

Returns:: result of precheck; anything but EXECRC_YIELD indicates that execution should terminate immediately with returned code

Definition at line 61 of file ConduitExecStream.cpp.

References EXECBUF_EMPTY, EXECBUF_EOS, EXECBUF_NONEMPTY, EXECBUF_OVERFLOW, EXECBUF_UNDERFLOW, EXECRC_BUF_OVERFLOW, EXECRC_BUF_UNDERFLOW, EXECRC_EOS, EXECRC_YIELD, SingleInputExecStream::pInAccessor, and SingleOutputExecStream::pOutAccessor.

Referenced by ExternalSortExecStreamImpl::execute(), FtrsTableWriterExecStream::execute(), BTreeSearchUniqueExecStream::execute(), BTreeSearchExecStream::execute(), BTreeInsertExecStream::execute(), UncollectExecStream::execute(), SortedAggExecStream::execute(), execute(), CopyExecStream::execute(), CollectExecStream::execute(), BernoulliSamplingExecStream::execute(), and CalcExecStream::execute().

00062 {
00063     switch (pInAccessor->getState()) {
00064     case EXECBUF_EMPTY:
00065         pInAccessor->requestProduction();
00066         return EXECRC_BUF_UNDERFLOW;
00067     case EXECBUF_UNDERFLOW:
00068         return EXECRC_BUF_UNDERFLOW;
00069     case EXECBUF_EOS:
00070         pOutAccessor->markEOS();
00071         return EXECRC_EOS;
00072     case EXECBUF_NONEMPTY:
00073     case EXECBUF_OVERFLOW:
00074         break;
00075     default:
00076         permAssert(false);
00077     }
00078     if (pOutAccessor->getState() == EXECBUF_OVERFLOW) {
00079         return EXECRC_BUF_OVERFLOW;
00080     }
00081     return EXECRC_YIELD;
00082 }

void ConduitExecStream::setOutputBufAccessors ( std::vector< SharedExecStreamBufAccessor > const & outAccessors ) [virtual, inherited]

Initializes the buffer accessors for outputs from this stream.

This method is only ever called once, before prepare.

Parameters:

outAccessors

buffer accessors ordered by output stream

Reimplemented from SingleInputExecStream.

Definition at line 36 of file ConduitExecStream.cpp.

References SingleOutputExecStream::setOutputBufAccessors().

00038 {
00039     SingleOutputExecStream::setOutputBufAccessors(outAccessors);
00040 }

void ConduitExecStream::setInputBufAccessors ( std::vector< SharedExecStreamBufAccessor > const & inAccessors ) [virtual, inherited]

Initializes the buffer accessors for inputs to this stream.

This method is only ever called once, before prepare.

Parameters:

inAccessors

buffer accessors ordered by input stream

Reimplemented from SingleInputExecStream.

Definition at line 30 of file ConduitExecStream.cpp.

References SingleInputExecStream::setInputBufAccessors().

00032 {
00033     SingleInputExecStream::setInputBufAccessors(inAccessors);
00034 }

void ConduitExecStream::prepare ( ConduitExecStreamParams const & params ) [virtual, inherited]

Definition at line 42 of file ConduitExecStream.cpp.

References SingleInputExecStream::pInAccessor, SingleOutputExecStream::pOutAccessor, SingleOutputExecStream::prepare(), and SingleInputExecStream::prepare().

00043 {
00044     SingleInputExecStream::prepare(params);
00045 
00046     if (params.outputTupleDesc.empty()) {
00047         pOutAccessor->setTupleShape(
00048             pInAccessor->getTupleDesc(),
00049             pInAccessor->getTupleFormat());
00050     }
00051 
00052     SingleOutputExecStream::prepare(params);
00053 }

void SingleInputExecStream::prepare ( SingleInputExecStreamParams const & params ) [virtual, inherited]

Definition at line 44 of file SingleInputExecStream.cpp.

References SingleInputExecStream::getInputBufProvision(), SingleInputExecStream::pInAccessor, and ExecStream::prepare().

Referenced by JavaSinkExecStream::prepare(), MockConsumerExecStream::prepare(), DiffluenceExecStream::prepare(), and ConduitExecStream::prepare().

00045 {
00046     ExecStream::prepare(params);
00047 
00048     assert(pInAccessor);
00049     assert(pInAccessor->getProvision() == getInputBufProvision());
00050 }

void ExecStream::prepare ( ExecStreamParams const & params ) [virtual, inherited]

Prepares this stream for execution.

A precondition is that input streams must already be defined and prepared. As an effect of this call, the tuple shape should be defined for all output buffers and remain unchanged for the lifetime of the stream. This method is only ever called once, before the first open. Although this method is virtual, derived classes may choose to define an overloaded version instead with a specialized covariant parameter class.

Parameters:

params

instance of stream parameterization class which should be used to prepare this stream

Definition at line 84 of file ExecStream.cpp.

References ExecStreamGraph::getDynamicParamManager(), SegmentAccessor::pCacheAccessor, ExecStreamParams::pCacheAccessor, ExecStream::pDynamicParamManager, ExecStream::pGraph, ExecStream::pQuotaAccessor, ExecStream::pScratchQuotaAccessor, and ExecStreamParams::scratchAccessor.

Referenced by JavaTransformExecStream::prepare(), SingleOutputExecStream::prepare(), and SingleInputExecStream::prepare().

00085 {
00086     if (pGraph) {
00087         pDynamicParamManager = pGraph->getDynamicParamManager();
00088     }
00089     pQuotaAccessor = params.pCacheAccessor;
00090     pScratchQuotaAccessor = params.scratchAccessor.pCacheAccessor;
00091 }

void SingleOutputExecStream::prepare ( SingleOutputExecStreamParams const & params ) [virtual, inherited]

Definition at line 48 of file SingleOutputExecStream.cpp.

References SingleOutputExecStream::getOutputBufProvision(), SingleOutputExecStreamParams::outputTupleDesc, SingleOutputExecStreamParams::outputTupleFormat, SingleOutputExecStream::pOutAccessor, and ExecStream::prepare().

Referenced by BTreeExecStream::prepare(), FlatFileExecStreamImpl::prepare(), ValuesExecStream::prepare(), MockResourceExecStream::prepare(), MockProducerExecStream::prepare(), ConfluenceExecStream::prepare(), and ConduitExecStream::prepare().

00049 {
00050     ExecStream::prepare(params);
00051     assert(pOutAccessor);
00052     assert(pOutAccessor->getProvision() == getOutputBufProvision());
00053     if (pOutAccessor->getTupleDesc().empty()) {
00054         assert(!params.outputTupleDesc.empty());
00055         pOutAccessor->setTupleShape(
00056             params.outputTupleDesc,
00057             params.outputTupleFormat);
00058     }
00059 }

ExecStreamBufProvision SingleInputExecStream::getInputBufProvision ( ) const [virtual, inherited]

Queries the BufferProvision which this stream requires of its inputs when consuming their tuples.

Returns:: required model; default is BUFPROV_NONE

Reimplemented from ExecStream.

Reimplemented in DoubleBufferExecStream, ScratchBufferExecStream, SegBufferExecStream, and SegBufferWriterExecStream.

Definition at line 62 of file SingleInputExecStream.cpp.

References BUFPROV_PRODUCER.

Referenced by SingleInputExecStream::prepare().

00063 {
00064     return BUFPROV_PRODUCER;
00065 }

bool ExecStream::canEarlyClose ( ) [virtual, inherited]

Returns:: true if the stream can be closed early

Reimplemented in SegBufferWriterExecStream.

Definition at line 49 of file ExecStream.cpp.

00050 {
00051     return true;
00052 }

ExecStreamGraph & ExecStream::getGraph ( ) const [inline, inherited]

Returns:: reference to containing graph

Definition at line 293 of file ExecStream.h.

References ExecStream::pGraph.

Referenced by ExternalSortExecStreamImpl::execute(), JavaSinkExecStream::execute(), SegBufferWriterExecStream::execute(), SegBufferExecStream::execute(), BarrierExecStream::execute(), Java_net_sf_farrago_fennel_FennelStorage_tupleStreamFetch(), Java_net_sf_farrago_fennel_FennelStorage_tupleStreamTransformFetch(), JavaSinkExecStream::stuffByteBuffer(), and ExecStreamScheduler::traceStreamBuffers().

00294 {
00295     assert(pGraph);
00296     return *pGraph;
00297 }

ExecStreamId ExecStream::getStreamId ( ) const [inline, inherited]

Returns:: the identifier for this stream within containing graph

Definition at line 288 of file ExecStream.h.

References ExecStream::id.

Referenced by ExternalSortExecStreamImpl::execute(), SegBufferWriterExecStream::execute(), SegBufferExecStream::execute(), CorrelationJoinExecStream::execute(), BarrierExecStream::execute(), Java_net_sf_farrago_fennel_FennelStorage_tupleStreamTransformFetch(), JavaTransformExecStream::open(), SingleInputExecStream::open(), ConfluenceExecStream::open(), CartesianJoinExecStream::prepare(), ParallelExecStreamScheduler::readStream(), DfsTreeExecStreamScheduler::readStream(), LbmMinusExecStream::restartSubtrahends(), ExecStreamScheduler::tracePostExecution(), ExecStreamScheduler::tracePreExecution(), ExecStreamScheduler::traceStreamBuffers(), and ParallelExecStreamScheduler::tryExecuteTask().

00289 {
00290     return id;
00291 }

void ExecStream::getResourceRequirements	(	ExecStreamResourceQuantity &	minQuantity,
		ExecStreamResourceQuantity &	optQuantity,
		ExecStreamResourceSettingType &	optType
	)			`[virtual, inherited]`

Determines resource requirements for this stream.

Default implementation declares zero resource requirements.

Parameters:

	minQuantity	receives the minimum resource quantity needed by this stream in order to execute
	optQuantity	receives the resource quantity needed by this stream in order to execute optimally
	optType	Receives the value indicating the accuracy of the optQuantity parameter. This parameter is optional and defaults to EXEC_RESOURCE_ACCURATE if omitted. If the optimum setting is an estimate or no value can be specified (e.g., due to lack of statistics), then this parameter needs to be used to indicate a non-accurate optimum resource setting.

Reimplemented in MockResourceExecStream, BTreePrefetchSearchExecStream, LhxAggExecStream, LhxJoinExecStream, LbmGeneratorExecStream, LbmUnionExecStream, and ExternalSortExecStreamImpl.

Definition at line 93 of file ExecStream.cpp.

References EXEC_RESOURCE_ACCURATE.

00097 {
00098     getResourceRequirements(minQuantity, optQuantity);
00099     optType = EXEC_RESOURCE_ACCURATE;
00100 }

void ExecStream::getResourceRequirements	(	ExecStreamResourceQuantity &	minQuantity,
		ExecStreamResourceQuantity &	optQuantity
	)			`[virtual, inherited]`

Reimplemented in DoubleBufferExecStream, ScratchBufferExecStream, SegBufferExecStream, SegBufferReaderExecStream, SegBufferWriterExecStream, FlatFileExecStreamImpl, BTreeInsertExecStream, BTreeReadExecStream, FtrsTableWriterExecStream, LbmChopperExecStream, LbmSplicerExecStream, LcsClusterAppendExecStream, LcsClusterReplaceExecStream, LcsRowScanBaseExecStream, and LcsRowScanExecStream.

Definition at line 102 of file ExecStream.cpp.

References ExecStreamResourceQuantity::nCachePages, and ExecStreamResourceQuantity::nThreads.

00105 {
00106     minQuantity.nThreads = 0;
00107     minQuantity.nCachePages = 0;
00108     optQuantity = minQuantity;
00109 }

void ExecStream::setResourceAllocation ( ExecStreamResourceQuantity & quantity ) [virtual, inherited]

Sets current resource allocation for this stream.

If called while the stream is open, this indicates a request for the stream to dynamically adjust its memory usage. If the stream is incapable of honoring the request, it should update quantity with the actual amounts still in use.

Parameters:

quantity

allocated resource quantity

Reimplemented in MockResourceExecStream, BTreePrefetchSearchExecStream, LhxAggExecStream, LhxJoinExecStream, LbmGeneratorExecStream, LbmUnionExecStream, and ExternalSortExecStreamImpl.

Definition at line 111 of file ExecStream.cpp.

References ExecStreamResourceQuantity::nCachePages, ExecStream::pQuotaAccessor, ExecStream::pScratchQuotaAccessor, and ExecStream::resourceAllocation.

Referenced by ExternalSortExecStreamImpl::setResourceAllocation(), LbmUnionExecStream::setResourceAllocation(), LbmGeneratorExecStream::setResourceAllocation(), LhxJoinExecStream::setResourceAllocation(), LhxAggExecStream::setResourceAllocation(), and BTreePrefetchSearchExecStream::setResourceAllocation().

00113 {
00114     resourceAllocation = quantity;
00115     if (pQuotaAccessor) {
00116         pQuotaAccessor->setMaxLockedPages(quantity.nCachePages);
00117     }
00118     if (pScratchQuotaAccessor) {
00119         pScratchQuotaAccessor->setMaxLockedPages(quantity.nCachePages);
00120     }
00121 }

void ExecStream::setName ( std::string const & ) [virtual, inherited]

Sets unique name of this stream.

Definition at line 157 of file ExecStream.cpp.

References ExecStream::name.

00158 {
00159     name = nameInit;
00160 }

std::string const & ExecStream::getName ( ) const [virtual, inherited]

Returns:: the name of this stream, as known by the optimizer

Definition at line 162 of file ExecStream.cpp.

References ExecStream::name.

Referenced by DfsTreeExecStreamScheduler::findNextConsumer(), ParallelExecStreamScheduler::readStream(), DfsTreeExecStreamScheduler::readStream(), ExecStreamScheduler::tracePostExecution(), and ExecStreamScheduler::tracePreExecution().

00163 {
00164     return name;
00165 }

bool ExecStream::mayBlock ( ) const [virtual, inherited]

Queries whether this stream's implementation may block when execute() is called.

For accurate scheduling, non-blocking implementations are preferred; the scheduler must be aware of the potential for blocking so that it can allocate extra threads accordingly.

Returns:: whether stream may block; default is false

Definition at line 167 of file ExecStream.cpp.

00168 {
00169     return false;
00170 }

void ExecStream::checkAbort ( ) const [virtual, inherited]

Checks whether there is an abort request for this stream's scheduler.

Normally, streams don't need to check this, since the scheduler services abort requests in between quanta. However, streams which enter long-running loops need to check for themselves. If an abort is scheduled, this method will throw an AbortExcn automatically.

Definition at line 72 of file ExecStream.cpp.

References ExecStreamScheduler::checkAbort(), ExecStreamGraph::getScheduler(), and ExecStream::pGraph.

Referenced by LhxJoinExecStream::execute(), LhxAggExecStream::execute(), ExternalSortRunAccessor::fetch(), and ExternalSortMerger::fetch().

00073 {
00074     if (!pGraph) {
00075         return;
00076     }
00077     ExecStreamScheduler *pScheduler = pGraph->getScheduler();
00078     if (!pScheduler) {
00079         return;
00080     }
00081     pScheduler->checkAbort();
00082 }

ExecStreamBufProvision ExecStream::getOutputBufProvision ( ) const [virtual, inherited]

Queries the BufferProvision which this stream is capable of when producing tuples.

Returns:: supported model; default is BUFPROV_NONE

Reimplemented in BarrierExecStream, DiffluenceExecStream, DoubleBufferExecStream, MergeExecStream, MockResourceExecStream, ScratchBufferExecStream, SegBufferExecStream, SegBufferReaderExecStream, SegBufferWriterExecStream, SingleOutputExecStream, SplitterExecStream, ValuesExecStream, JavaTransformExecStream, FtrsTableWriterExecStream, and LcsClusterAppendExecStream.

Definition at line 172 of file ExecStream.cpp.

References BUFPROV_NONE.

00173 {
00174     return BUFPROV_NONE;
00175 }

ExecStreamBufProvision SingleOutputExecStream::getOutputBufProvision ( ) const [virtual, inherited]

Queries the BufferProvision which this stream is capable of when producing tuples.

Returns:: supported model; default is BUFPROV_NONE

Reimplemented from ExecStream.

Reimplemented in BarrierExecStream, DoubleBufferExecStream, MergeExecStream, MockResourceExecStream, ScratchBufferExecStream, SegBufferExecStream, SegBufferReaderExecStream, ValuesExecStream, FtrsTableWriterExecStream, and LcsClusterAppendExecStream.

Definition at line 69 of file SingleOutputExecStream.cpp.

References BUFPROV_CONSUMER.

Referenced by SingleOutputExecStream::prepare().

00070 {
00071     return BUFPROV_CONSUMER;
00072 }

ExecStreamBufProvision ExecStream::getOutputBufConversion ( ) const [virtual, inherited]

Queries the BufferProvision to which this stream needs its output to be converted, if any.

Returns:: required conversion; default is BUFPROV_NONE

Reimplemented in JavaTransformExecStream.

Definition at line 177 of file ExecStream.cpp.

References BUFPROV_NONE.

00178 {
00179     return BUFPROV_NONE;
00180 }

bool ClosableObject::isClosed ( ) const [inline, inherited]

Returns:: whether the object has been closed

Definition at line 58 of file ClosableObject.h.

00059     {
00060         return !needsClose;
00061     }

void ClosableObject::close ( ) [inherited]

Closes this object, releasing any unallocated resources.

Reimplemented in CollectExecStream, CorrelationJoinExecStream, LcsClusterAppendExecStream, and LcsClusterReplaceExecStream.

Definition at line 39 of file ClosableObject.cpp.

References ClosableObject::closeImpl(), and ClosableObject::needsClose.

Referenced by CacheImpl< PageT, VictimPolicyT >::allocatePages(), LcsRowScanBaseExecStream::closeImpl(), ExecStreamGraphImpl::closeImpl(), FlatFileBuffer::open(), ClosableObjectDestructor::operator()(), and Segment::~Segment().

00040 {
00041     if (!needsClose) {
00042         return;
00043     }
00044     needsClose = false;
00045     closeImpl();
00046 }

void TraceSource::initTraceSource	(	SharedTraceTarget	pTraceTarget,
		std::string	name
	)			`[virtual, inherited]`

For use when initialization has to be deferred until after construction.

Parameters:

	pTraceTarget	the TraceTarget to which messages will be sent
	name	the name of this source

Definition at line 46 of file TraceSource.cpp.

References TraceSource::isTracing(), TraceSource::minimumLevel, TraceSource::name, TraceSource::pTraceTarget, and TRACE_OFF.

Referenced by TestBase::beforeTestCase(), TestBase::TestBase(), and TraceSource::TraceSource().

00049 {
00050     assert(!pTraceTarget.get());
00051 
00052     pTraceTarget = pTraceTargetInit;
00053     name = nameInit;
00054     if (isTracing()) {
00055         minimumLevel = pTraceTarget->getSourceTraceLevel(name);
00056     } else {
00057         minimumLevel = TRACE_OFF;
00058     }
00059 }

void TraceSource::trace	(	TraceLevel	level,
		std::string	message
	)			const `[inherited]`

Records a trace message.

Normally only called via FENNEL_TRACE.

Parameters:

	level	severity level of event being trace
	message	the text of the message

Definition at line 61 of file TraceSource.cpp.

References TraceSource::getTraceTarget(), TraceSource::isTracing(), TraceSource::name, and TraceTarget::notifyTrace().

Referenced by Calculator::exec(), and ExecStreamScheduler::traceStreamBufferContents().

00062 {
00063     if (isTracing()) {
00064         getTraceTarget().notifyTrace(name,level,message);
00065     }
00066 }

bool TraceSource::isTracing ( ) const [inline, inherited]

Returns:: true iff tracing is enabled for this source

Definition at line 88 of file TraceSource.h.

Referenced by TraceSource::initTraceSource(), CalcExecStream::prepare(), and TraceSource::trace().

00089     {
00090         return pTraceTarget.get() ? true : false;
00091     }

bool TraceSource::isTracingLevel ( TraceLevel level ) const [inline, inherited]

Determines whether a particular level is being traced.

Parameters:

level

trace level to test

Returns:: true iff tracing is enabled for the given level

Definition at line 100 of file TraceSource.h.

Referenced by ExecStreamScheduler::addGraph(), SimpleExecStreamGovernor::assignCachePages(), SimpleExecStreamGovernor::distributeCachePages(), Calculator::exec(), ExecStreamScheduler::ExecStreamScheduler(), LcsClusterNodeWriter::getLastClusterPageForWrite(), LcsClusterNodeWriter::moveFromTempToIndex(), JavaSinkExecStream::stuffByteBuffer(), and ExecStreamScheduler::traceStreamBuffers().

00101     {
00102         return level >= minimumLevel;
00103     }

TraceTarget& TraceSource::getTraceTarget ( ) const [inline, inherited]

Returns:: the TraceTarget for this source

Definition at line 108 of file TraceSource.h.

Referenced by TraceSource::trace().

00109     {
00110         assert(isTracing());
00111         return *(pTraceTarget.get());
00112     }

SharedTraceTarget TraceSource::getSharedTraceTarget ( ) const [inline, inherited]

Returns:: the SharedTraceTarget for this source

Definition at line 117 of file TraceSource.h.

Referenced by Database::init(), LcsClusterAppendExecStream::initLoad(), and CalcExecStream::prepare().

00118     {
00119         return pTraceTarget;
00120     }

std::string TraceSource::getTraceSourceName ( ) const [inline, inherited]

Gets the name of this source.

Useful to construct nested names for subcomponents that are also TraceSources.

Returns:: the name

Definition at line 127 of file TraceSource.h.

Referenced by LcsClusterAppendExecStream::initLoad().

00128     {
00129         return name;
00130     }

void TraceSource::setTraceSourceName ( std::string const & n ) [inline, inherited]

Sets the name of this source.

Useful to construct dynamic names for fine-grained filtering.

Definition at line 136 of file TraceSource.h.

00137     {
00138         name = n;
00139     }

TraceLevel TraceSource::getMinimumTraceLevel ( ) const [inline, inherited]

Definition at line 141 of file TraceSource.h.

00142     {
00143         return minimumLevel;
00144     }

void TraceSource::disableTracing ( ) [inherited]

Definition at line 68 of file TraceSource.cpp.

References TraceSource::minimumLevel, TraceSource::pTraceTarget, and TRACE_OFF.

Referenced by TestBase::afterTestCase().

00069 {
00070     pTraceTarget.reset();
00071     minimumLevel = TRACE_OFF;
00072 }

void ErrorSource::initErrorSource	(	SharedErrorTarget	pErrorTarget,
		const std::string &	name
	)			`[virtual, inherited]`

For use when initialization has to be deferred until after construction.

Parameters:

	pErrorTarget	the ErrorTarget to which errors will be posted
	name	the name of this source

Definition at line 47 of file ErrorSource.cpp.

References ErrorSource::name, and ErrorSource::pErrorTarget.

Referenced by ErrorSource::ErrorSource().

00050 {
00051     pErrorTarget = pErrorTargetInit;
00052     name = nameInit;
00053 }

void ErrorSource::postError	(	ErrorLevel	level,
		const std::string &	message,
		void *	address,
		long	capacity,
		int	index
	)			`[inherited]`

Posts an exception, such as a row exception.

See also:: ErrorTarget for a description of the parameters

Definition at line 55 of file ErrorSource.cpp.

References ErrorSource::getErrorTarget(), ErrorSource::hasTarget(), ErrorSource::name, and ErrorTarget::notifyError().

Referenced by FlatFileExecStreamImpl::logError(), ErrorSource::postError(), and LbmSplicerExecStream::postViolation().

00058 {
00059     if (hasTarget()) {
00060         getErrorTarget().notifyError(
00061             name, level, message, address, capacity, index);
00062     }
00063 }

void ErrorSource::postError	(	ErrorLevel	level,
		const std::string &	message,
		const TupleDescriptor &	errorDesc,
		const TupleData &	errorTuple,
		int	index
	)			`[inherited]`

Posts an exception, such as a row exception.

See also:: ErrorTarget for a description of the parameters

Definition at line 65 of file ErrorSource.cpp.

References TupleAccessor::compute(), ErrorSource::errorAccessor, FixedBuffer, TupleAccessor::getByteCount(), TupleAccessor::getMaxByteCount(), ErrorSource::hasTarget(), TupleAccessor::marshal(), ErrorSource::pErrorBuf, and ErrorSource::postError().

00068 {
00069     if (!hasTarget()) {
00070         return;
00071     }
00072 
00073     if (!pErrorBuf) {
00074         errorAccessor.compute(errorDesc);
00075         uint cbMax = errorAccessor.getMaxByteCount();
00076         pErrorBuf.reset(new FixedBuffer[cbMax]);
00077     }
00078 
00079     uint cbTuple = errorAccessor.getByteCount(errorTuple);
00080     errorAccessor.marshal(errorTuple, pErrorBuf.get());
00081     postError(level, message, pErrorBuf.get(), cbTuple, index);
00082 }

bool ErrorSource::hasTarget ( ) const [inline, inherited]

Returns:: true iff an error target has been set

Definition at line 112 of file ErrorSource.h.

Referenced by ErrorSource::postError().

00113     {
00114         return pErrorTarget.get() ? true : false;
00115     }

ErrorTarget& ErrorSource::getErrorTarget ( ) const [inline, inherited]

Returns:: the ErrorTarget for this source

Definition at line 120 of file ErrorSource.h.

Referenced by ErrorSource::postError().

00121     {
00122         assert(hasTarget());
00123         return *(pErrorTarget.get());
00124     }

SharedErrorTarget ErrorSource::getSharedErrorTarget ( ) const [inline, inherited]

Returns:: the SharedErrorTarget for this source

Definition at line 129 of file ErrorSource.h.

00130     {
00131         return pErrorTarget;
00132     }

std::string ErrorSource::getErrorSourceName ( ) const [inline, inherited]

Gets the name of this source.

Useful to construct nested names for subcomponents that are also ErrorSources.

Returns:: the name

Definition at line 139 of file ErrorSource.h.

00140     {
00141         return name;
00142     }

void ErrorSource::setErrorSourceName ( std::string const & n ) [inline, inherited]

Sets the name of this source.

Useful to construct dynamic names for fine-grained filtering.

Definition at line 148 of file ErrorSource.h.

00149     {
00150         name = n;
00151     }

void ErrorSource::disableTarget ( ) [inherited]

Definition at line 84 of file ErrorSource.cpp.

References ErrorSource::pErrorTarget.

00085 {
00086     pErrorTarget.reset();
00087 }

CompOperator ReshapeExecStream::compOp [private]

Comparison operator.

Definition at line 136 of file ReshapeExecStream.h.

Referenced by compareInput(), execute(), and prepare().

std::vector<ReshapeParameter> ReshapeExecStream::dynamicParameters [private]

Dynamic parameters.

Definition at line 141 of file ReshapeExecStream.h.

Referenced by initCompareData(), prepare(), and readDynamicParams().

bool ReshapeExecStream::paramsRead [private]

True if the dynamic parameters have been read.

They are only read once when the stream is first executed

Definition at line 147 of file ReshapeExecStream.h.

Referenced by execute(), and open().

TupleData ReshapeExecStream::paramCompareData [private]

TupleData corresponding to the comparison tuple that will be compared with the input stream.

Definition at line 153 of file ReshapeExecStream.h.

Referenced by compareInput(), initCompareData(), nullFilter(), and readDynamicParams().

TupleProjectionAccessor ReshapeExecStream::inputCompareProjAccessor [private]

Tuple projection accessor for the columns to be compared against the comparison tuple.

Definition at line 159 of file ReshapeExecStream.h.

Referenced by compareInput(), and initCompareData().

TupleData ReshapeExecStream::inputCompareData [private]

Tuple data corresponding to the input tuple data to be used in comparisons.

Definition at line 165 of file ReshapeExecStream.h.

Referenced by compareInput(), and initCompareData().

boost::shared_array<FixedBuffer> ReshapeExecStream::compTupleBuffer [private]

Buffer used to store comparison tuple passed in as a parameter.

Definition at line 170 of file ReshapeExecStream.h.

Referenced by copyCompareTuple().

TupleDescriptor ReshapeExecStream::compTupleDesc [private]

Tuple descriptor for the comparison tuple.

Definition at line 175 of file ReshapeExecStream.h.

Referenced by compareInput(), and initCompareData().

uint ReshapeExecStream::numCompDynParams [private]

Number of dynamic parameters that are to be compared.

Definition at line 180 of file ReshapeExecStream.h.

Referenced by initCompareData(), and readDynamicParams().

TupleProjection ReshapeExecStream::lastKey [private]

Tuple projection used in the case when the comparison is non-equality.

Definition at line 185 of file ReshapeExecStream.h.

Referenced by compareInput(), and initCompareData().

TupleDescriptor ReshapeExecStream::lastKeyDesc [private]

Tuple descriptor corresponding to the last key.

Definition at line 190 of file ReshapeExecStream.h.

Referenced by compareInput(), and initCompareData().

TupleProjectionAccessor ReshapeExecStream::outputProjAccessor [private]

Tuple projection accessor for the output tuple.

Definition at line 195 of file ReshapeExecStream.h.

Referenced by castOutput(), execute(), and prepare().

TupleDescriptor ReshapeExecStream::inputOutputDesc [private]

Tuple descriptor from the input stream corresponding to the output projection columns.

Definition at line 201 of file ReshapeExecStream.h.

Referenced by prepare(), and readDynamicParams().

TupleData ReshapeExecStream::inputOutputData [private]

Tuple data corresponding to the output row in the input stream.

Definition at line 206 of file ReshapeExecStream.h.

Referenced by castOutput(), and prepare().

TupleDescriptor ReshapeExecStream::outputDesc [private]

Output descriptor.

Definition at line 211 of file ReshapeExecStream.h.

Referenced by castOutput(), and prepare().

TupleData ReshapeExecStream::outputData [private]

Tuple data for the output tuple.

Definition at line 216 of file ReshapeExecStream.h.

Referenced by castOutput(), execute(), prepare(), and readDynamicParams().

bool ReshapeExecStream::castRequired [private]

True if the inputOutputTupleDesc does not match the output tuple descriptor, and therefore simple casting is required.

Definition at line 222 of file ReshapeExecStream.h.

Referenced by execute(), and prepare().

bool ReshapeExecStream::producePending [private]

True if production of a tuple to the output stream is pending.

Definition at line 227 of file ReshapeExecStream.h.

Referenced by execute(), and open().

SharedExecStreamBufAccessor SingleInputExecStream::pInAccessor [protected, inherited]

Definition at line 51 of file SingleInputExecStream.h.

bool ExecStream::isOpen [protected, inherited]

Whether this stream is currently open.

Note that this is not quite the opposite of the inherited ClosableObject.needsClose, since a stream needs to be closed before destruction if it has been prepared but never opened.

Definition at line 61 of file ExecStream.h.

Referenced by ExecStream::closeImpl(), ExecStream::ExecStream(), and ExecStream::open().

ExecStreamGraph* ExecStream::pGraph [protected, inherited]

Dataflow graph containing this stream.

Note that we don't use a weak_ptr for this because it needs to be accessed frequently during execution, and the extra locking overhead would be frivolous.

Definition at line 68 of file ExecStream.h.

Referenced by ExecStream::checkAbort(), ExecStream::ExecStream(), CorrelationJoinExecStream::execute(), ExecStream::getGraph(), JavaTransformExecStream::open(), SingleInputExecStream::open(), MergeExecStream::open(), ExecStream::open(), CorrelationJoinExecStream::open(), ConfluenceExecStream::open(), ExecStream::prepare(), CartesianJoinExecStream::prepare(), and LbmMinusExecStream::restartSubtrahends().

ExecStreamId ExecStream::id [protected, inherited]

Identifier for this stream; local to its containing graph.

Definition at line 73 of file ExecStream.h.

Referenced by ExecStream::getStreamId().

std::string ExecStream::name [protected, inherited]

Name of stream, as known by optimizer.

Reimplemented from TraceSource.

Definition at line 78 of file ExecStream.h.

Referenced by ExecStream::ExecStream(), ExecStream::getName(), FlatFileExecStreamImpl::open(), and ExecStream::setName().

SharedDynamicParamManager ExecStream::pDynamicParamManager [protected, inherited]

The dynamic parameter manager available to this stream.

(Obtained at prepare() time. Keep a shared pointer in case the stream is reassigned to another graph for execution; cf ExecStreamGraph::mergeFrom())

Definition at line 85 of file ExecStream.h.

SharedLogicalTxn ExecStream::pTxn [protected, inherited]

The transaction embracing the stream.

Obtained at open() time; but not released at close() time, to allow TableWriters to replay a txn. Keep a shared pointer in case the stream is reassigned to another graph for execution; cf ExecStreamGraph::mergeFrom())

Definition at line 94 of file ExecStream.h.

Referenced by FtrsTableWriterExecStream::commitSavepoint(), FtrsTableWriterExecStream::createSavepoint(), FtrsTableWriterExecStream::open(), ExecStream::open(), and FtrsTableWriterExecStream::rollbackSavepoint().

ExecStreamResourceQuantity ExecStream::resourceAllocation [protected, inherited]

Resource quantities currently allocated to this stream.

Definition at line 100 of file ExecStream.h.

Referenced by ExecStream::setResourceAllocation().

SharedCacheAccessor ExecStream::pQuotaAccessor [protected, inherited]

CacheAccessor used for quota tracking.

Definition at line 105 of file ExecStream.h.

Referenced by ExecStream::open(), ExecStream::prepare(), and ExecStream::setResourceAllocation().

SharedCacheAccessor ExecStream::pScratchQuotaAccessor [protected, inherited]

CacheAccessor used for scratch page quota tracking.

Definition at line 110 of file ExecStream.h.

Referenced by ExecStream::open(), ExecStream::prepare(), and ExecStream::setResourceAllocation().

bool ClosableObject::needsClose [protected, inherited]

Definition at line 44 of file ClosableObject.h.

Referenced by SegStreamAllocation::beginWrite(), ExecStreamGraphImpl::clear(), ClosableObject::ClosableObject(), ClosableObject::close(), FlatFileBuffer::open(), ExecStreamGraphImpl::open(), ExecStream::open(), and ClosableObject::~ClosableObject().

SharedExecStreamBufAccessor SingleOutputExecStream::pOutAccessor [protected, inherited]

Definition at line 56 of file SingleOutputExecStream.h.

ReshapeExecStream Class Reference

Public Member Functions

Protected Member Functions

Protected Attributes

Private Member Functions

Private Attributes

Detailed Description

Member Function Documentation

Member Data Documentation


Public Member Functions
virtual void	prepare (ReshapeExecStreamParams const &params)
virtual void	open (bool restart)
	Opens this stream, acquiring any resources needed in order to be able to fetch data.
virtual ExecStreamResult	execute (ExecStreamQuantum const &quantum)
	Executes this stream.
virtual void	closeImpl ()
	Implements ClosableObject.
virtual void	setOutputBufAccessors (std::vector< SharedExecStreamBufAccessor > const &outAccessors)
	Initializes the buffer accessors for outputs from this stream.
virtual void	setInputBufAccessors (std::vector< SharedExecStreamBufAccessor > const &inAccessors)
	Initializes the buffer accessors for inputs to this stream.
virtual void	prepare (ConduitExecStreamParams const &params)
virtual void	prepare (SingleInputExecStreamParams const &params)
virtual void	prepare (ExecStreamParams const &params)
	Prepares this stream for execution.
virtual void	prepare (SingleOutputExecStreamParams const &params)
virtual ExecStreamBufProvision	getInputBufProvision () const
	Queries the BufferProvision which this stream requires of its inputs when consuming their tuples.
virtual bool	canEarlyClose ()
	Returns: true if the stream can be closed early
ExecStreamGraph &	getGraph () const
	Returns: reference to containing graph
ExecStreamId	getStreamId () const
	Returns: the identifier for this stream within containing graph
virtual void	getResourceRequirements (ExecStreamResourceQuantity &minQuantity, ExecStreamResourceQuantity &optQuantity, ExecStreamResourceSettingType &optType)
	Determines resource requirements for this stream.
virtual void	getResourceRequirements (ExecStreamResourceQuantity &minQuantity, ExecStreamResourceQuantity &optQuantity)
virtual void	setResourceAllocation (ExecStreamResourceQuantity &quantity)
	Sets current resource allocation for this stream.
virtual void	setName (std::string const &)
	Sets unique name of this stream.
virtual std::string const &	getName () const
	Returns: the name of this stream, as known by the optimizer
virtual bool	mayBlock () const
	Queries whether this stream's implementation may block when execute() is called.
virtual void	checkAbort () const
	Checks whether there is an abort request for this stream's scheduler.
virtual ExecStreamBufProvision	getOutputBufProvision () const
	Queries the BufferProvision which this stream is capable of when producing tuples.
virtual ExecStreamBufProvision	getOutputBufProvision () const
	Queries the BufferProvision which this stream is capable of when producing tuples.
virtual ExecStreamBufProvision	getOutputBufConversion () const
	Queries the BufferProvision to which this stream needs its output to be converted, if any.
bool	isClosed () const
	Returns: whether the object has been closed
void	close ()
	Closes this object, releasing any unallocated resources.
virtual void	initTraceSource (SharedTraceTarget pTraceTarget, std::string name)
	For use when initialization has to be deferred until after construction.
void	trace (TraceLevel level, std::string message) const
	Records a trace message.
bool	isTracing () const
	Returns: true iff tracing is enabled for this source
bool	isTracingLevel (TraceLevel level) const
	Determines whether a particular level is being traced.
TraceTarget &	getTraceTarget () const
	Returns: the TraceTarget for this source
SharedTraceTarget	getSharedTraceTarget () const
	Returns: the SharedTraceTarget for this source
std::string	getTraceSourceName () const
	Gets the name of this source.
void	setTraceSourceName (std::string const &n)
	Sets the name of this source.
TraceLevel	getMinimumTraceLevel () const
void	disableTracing ()
virtual void	initErrorSource (SharedErrorTarget pErrorTarget, const std::string &name)
	For use when initialization has to be deferred until after construction.
void	postError (ErrorLevel level, const std::string &message, void *address, long capacity, int index)
	Posts an exception, such as a row exception.
void	postError (ErrorLevel level, const std::string &message, const TupleDescriptor &errorDesc, const TupleData &errorTuple, int index)
	Posts an exception, such as a row exception.
bool	hasTarget () const
	Returns: true iff an error target has been set
ErrorTarget &	getErrorTarget () const
	Returns: the ErrorTarget for this source
SharedErrorTarget	getSharedErrorTarget () const
	Returns: the SharedErrorTarget for this source
std::string	getErrorSourceName () const
	Gets the name of this source.
void	setErrorSourceName (std::string const &n)
	Sets the name of this source.
void	disableTarget ()
Protected Member Functions
ExecStreamResult	precheckConduitBuffers ()
	Checks the state of the input and output buffers.
Protected Attributes
SharedExecStreamBufAccessor	pInAccessor
bool	isOpen
	Whether this stream is currently open.
ExecStreamGraph *	pGraph
	Dataflow graph containing this stream.
ExecStreamId	id
	Identifier for this stream; local to its containing graph.
std::string	name
	Name of stream, as known by optimizer.
SharedDynamicParamManager	pDynamicParamManager
	The dynamic parameter manager available to this stream.
SharedLogicalTxn	pTxn
	The transaction embracing the stream.
ExecStreamResourceQuantity	resourceAllocation
	Resource quantities currently allocated to this stream.
SharedCacheAccessor	pQuotaAccessor
	CacheAccessor used for quota tracking.
SharedCacheAccessor	pScratchQuotaAccessor
	CacheAccessor used for scratch page quota tracking.
bool	needsClose
SharedExecStreamBufAccessor	pOutAccessor
Private Member Functions
void	initCompareData (ReshapeExecStreamParams const &params, TupleDescriptor const &inputDesc, TupleAccessor const &inputAccessor)
	Initializes tuple descriptors and data that will be used in comparisons.
void	copyCompareTuple (TupleDescriptor const &tupleDesc, TupleData &tupleData, PBuffer tupleBuffer)
	Copies the comparison tuple from the input buffer and unmarshals it into a specified TupleData.
bool	checkCastTypes (const TupleProjection &compareProj, const TupleDescriptor &inputTupleDesc, const TupleDescriptor &outputTupleDesc)
	Verifies that in the case where simple casting is required, the types of the input and output meet the restrictions on the type of casting supported.
bool	nullFilter (const TupleProjection &compareProj, uint colno)
	Determines whether a column has a null filter on it.
void	readDynamicParams ()
	Reads the dynamic parameters and setups the comparison and output tuple datas to point to the parameter tuple datas.
bool	compareInput ()
	Compares input data against the base comparison tuple.
void	castOutput ()
	Copies the columns from the input tuple to the output tuple, performing truncation as part of casting, if needed.
Private Attributes
CompOperator	compOp
	Comparison operator.
std::vector< ReshapeParameter >	dynamicParameters
	Dynamic parameters.
bool	paramsRead
	True if the dynamic parameters have been read.
TupleData	paramCompareData
	TupleData corresponding to the comparison tuple that will be compared with the input stream.
TupleProjectionAccessor	inputCompareProjAccessor
	Tuple projection accessor for the columns to be compared against the comparison tuple.
TupleData	inputCompareData
	Tuple data corresponding to the input tuple data to be used in comparisons.
boost::shared_array< FixedBuffer >	compTupleBuffer
	Buffer used to store comparison tuple passed in as a parameter.
TupleDescriptor	compTupleDesc
	Tuple descriptor for the comparison tuple.
uint	numCompDynParams
	Number of dynamic parameters that are to be compared.
TupleProjection	lastKey
	Tuple projection used in the case when the comparison is non-equality.
TupleDescriptor	lastKeyDesc
	Tuple descriptor corresponding to the last key.
TupleProjectionAccessor	outputProjAccessor
	Tuple projection accessor for the output tuple.
TupleDescriptor	inputOutputDesc
	Tuple descriptor from the input stream corresponding to the output projection columns.
TupleData	inputOutputData
	Tuple data corresponding to the output row in the input stream.
TupleDescriptor	outputDesc
	Output descriptor.
TupleData	outputData
	Tuple data for the output tuple.
bool	castRequired
	True if the inputOutputTupleDesc does not match the output tuple descriptor, and therefore simple casting is required.
bool	producePending
	True if production of a tuple to the output stream is pending.