LhxPartitionInfo Struct Reference

#include <LhxPartition.h>

List of all members.

Public Member Functions
	LhxPartitionInfo ()
void	init (LhxHashInfo *hashInfoInit)
	Set up the recursive partitioning context.
void	open (LhxHashTableReader *hashTableReaderInit, LhxPartitionReader *buildReader, TupleData &buildTuple, SharedLhxPartition probePartition, uint buildInputIndex)
	Prepare to partition the inputs: Build inout reads from both hash table and an existing reader.
void	open (LhxHashTableReader *hashTableReaderInit, LhxPartitionReader *buildReader, TupleData &buildTuple, AggComputerList *aggList)
	Prepare to aggregate and partition the (build) input which reads from both hash table and an existing reader.
void	close ()
	Close the reader stream and the writer streams.
Public Attributes
LhxHashTableReader *	hashTableReader
	Hash Table Reader for hash aggregates.
LhxPartitionReader	probeReader
	Src partition reader.
LhxPartitionReader *	reader
TupleData	buildTuple
	holding the tuple which is not yet inserted into hash table
vector< SharedLhxPartitionWriter >	writerList
	Child partitions for both inputs have to be complete before child plans can be created.
vector< SharedLhxPartition >	destPartitionList
vector< shared_ptr< dynamic_bitset<> > >	joinFilterList
shared_array< uint >	filteredRowCountList
vector< shared_array< uint > >	subPartStatList
uint	numInputs
uint	curInputIndex
LhxHashInfo *	hashInfo
bool	partitionMemory

---

Detailed Description

Definition at line 180 of file LhxPartition.h.

---

Constructor & Destructor Documentation

LhxPartitionInfo::LhxPartitionInfo

(

)

[inline]

Definition at line 246 of file LhxPartition.h.

References hashTableReader, and reader.

    {
        reader = NULL;
        hashTableReader = NULL;
    }

---

Member Function Documentation

void LhxPartitionInfo::init

(

LhxHashInfo *

hashInfoInit

)

Set up the recursive partitioning context.

Definition at line 260 of file LhxPartition.cpp.

References filteredRowCountList, hashInfo, LhxHashInfo::inputDesc, LhxPlan::LhxChildPartCount, numInputs, and writerList.

Referenced by LhxJoinExecStream::open(), and LhxAggExecStream::open().

{
    hashInfo = hashInfoInit;
    numInputs = (hashInfo->inputDesc).size();

    writerList.clear();
    /*
     * writerList is shared across iterations of partitioning. At each
     * iteration, writerList is initialized with new destination partitions.
     */
    for (uint i = 0; i < numInputs * LhxPlan::LhxChildPartCount; i ++) {
        writerList.push_back(
            SharedLhxPartitionWriter(new LhxPartitionWriter()));
    }

    filteredRowCountList.reset(
        new uint[numInputs * LhxPlan::LhxChildPartCount]);
}

void LhxPartitionInfo::open	(	LhxHashTableReader *	hashTableReaderInit,
		LhxPartitionReader *	buildReader,
		TupleData &	buildTuple,
		SharedLhxPartition	probePartition,
		uint	buildInputIndex
	)

Prepare to partition the inputs: Build inout reads from both hash table and an existing reader.

There is also a inflight tuple that is part of this partition. Probe input reads from a partition, which could be either disk partition or execution buffer stream.

Definition at line 279 of file LhxPartition.cpp.

References LhxHashTableReader::bindKey(), buildTuple, curInputIndex, destPartitionList, filteredRowCountList, hashInfo, hashTableReader, joinFilterList, LhxPlan::LhxChildPartCount, LhxPlan::LhxSubPartCount, numInputs, LhxPartitionReader::open(), partitionMemory, probeReader, reader, subPartStatList, and writerList.

Referenced by LhxJoinExecStream::execute(), and LhxAggExecStream::execute().

{
    uint i, j;

    probeReader.open(probePartition, *hashInfo);

    /*
     * Start partitioning from the build side.
     */
    curInputIndex = buildInputIndex;

    hashTableReader = hashTableReaderInit;
    /*
     * Reset the reader and bind it to no particular key(will read the
     * whole hash table).
     */
    hashTableReader->bindKey(NULL);

    /*
     * The build reader is from the LhxJoinExecStream and is already open.
     */
    reader = buildReader;

    /*
     * The inflight (between disk partition and hash table) build tuple.
     */
    buildTuple = buildTupleInit;

    destPartitionList.clear();
    subPartStatList.clear();
    joinFilterList.clear();
    shared_array<uint> curSubPartStat;

    for (i = 0; i < numInputs * LhxPlan::LhxChildPartCount; i ++) {
        destPartitionList.push_back(
            SharedLhxPartition(new LhxPartition(probePartition->pExecStream)));
        destPartitionList[i]->inputIndex = (i / LhxPlan::LhxChildPartCount);
        subPartStatList.push_back(
            shared_array<uint>(new uint[LhxPlan::LhxSubPartCount]));

        curSubPartStat = subPartStatList[i];

        for (j = 0; j < LhxPlan::LhxSubPartCount; j ++) {
            curSubPartStat[j] = 0;
        }

        /*
         * One filter for each partition
         * filter bitmap is only allocated when a partition is written to
         */
        joinFilterList.push_back(shared_ptr<dynamic_bitset<> >());

        writerList[i]->open(destPartitionList[i], *hashInfo);
        filteredRowCountList[i] = 0;
    }

    /*
     * Tuples will come from memory (hash table) first.
     */
    partitionMemory = true;
}

void LhxPartitionInfo::open	(	LhxHashTableReader *	hashTableReaderInit,
		LhxPartitionReader *	buildReader,
		TupleData &	buildTuple,
		AggComputerList *	aggList
	)

Prepare to aggregate and partition the (build) input which reads from both hash table and an existing reader.

There is also a inflight tuple that is part of this partition. aggList contains the agg computers which aggregate the input.

Definition at line 346 of file LhxPartition.cpp.

References LhxHashTableReader::bindKey(), buildTuple, curInputIndex, destPartitionList, filteredRowCountList, LhxPartitionReader::getSourcePartition(), hashInfo, hashTableReader, joinFilterList, LhxPlan::LhxChildPartCount, LhxPlan::LhxSubPartCount, LhxHashInfo::numCachePages, numInputs, partitionMemory, reader, subPartStatList, and writerList.

{
    uint i, j;
    assert (numInputs == 1);
    uint buildIndex = numInputs - 1;

    curInputIndex = buildIndex;

    hashTableReader = hashTableReaderInit;
    /*
     * Reset the reader and bind it to no particular key (will read the
     * whole hash table).
     */
    hashTableReader->bindKey(NULL);

    // REVIEW jvs 26-Aug-2006:  no join if doing agg...
    /*
     * The build reader is from the LhxJoinExecStream and is already open.
     */
    reader = buildReader;

    /*
     * The inflight(between disk partition and hash table) build tuple.
     */
    buildTuple = buildTupleInit;

    /*
     * The hash table contained in the writer should only use up to the child's
     * share of scratch buffer.
     */
    uint numWriterCachePages =
        hashInfo->numCachePages / LhxPlan::LhxChildPartCount;

    destPartitionList.clear();
    subPartStatList.clear();
    joinFilterList.clear();
    shared_array<uint> curSubPartStat;

    for (i = 0; i < numInputs * LhxPlan::LhxChildPartCount; i ++) {
        destPartitionList.push_back(
            SharedLhxPartition(
                new LhxPartition(reader->getSourcePartition()->pExecStream)));
        destPartitionList[i]->inputIndex = (i / LhxPlan::LhxChildPartCount);
        subPartStatList.push_back(
            shared_array<uint>(new uint[LhxPlan::LhxSubPartCount]));

        curSubPartStat = subPartStatList[i];

        for (j = 0; j < LhxPlan::LhxSubPartCount; j ++) {
            curSubPartStat[j] = 0;
        }

        // REVIEW jvs 26-Aug-2006:  no join if doing agg...
        /*
         * One filter for each partition
         * filter bitmap is only allocated when a partition is written to
         */
        joinFilterList.push_back(shared_ptr<dynamic_bitset<> >());

        writerList[i]->open(
            destPartitionList[i],
            *hashInfo,
            aggList,
            numWriterCachePages);
        filteredRowCountList[i] = 0;
    }

    /*
     * Tuples will come from memory(hash table) first.
     */
    partitionMemory = true;
}

void LhxPartitionInfo::close

(

)

Close the reader stream and the writer streams.

Definition at line 423 of file LhxPartition.cpp.

References LhxPartitionReader::close(), reader, and writerList.

Referenced by LhxJoinExecStream::execute(), and LhxAggExecStream::execute().

{
    reader->close();

    uint numWriters = writerList.size();

    for (uint i = 0; i < numWriters; i ++) {
        writerList[i]->close();
    }

    /*
     * Partial aggregate hash tables used inside the writers(one HT
     * for each writer) share the same scratch buffer space.
     * Release the buffer pages used by these hash tables at the end,
     * after all writers have been closed(so that there will be no more
     * scratch page alloc calls).
     */
    for (uint i = 0; i < numWriters; i ++) {
        writerList[i]->releaseResources();
    }
}

---

Member Data Documentation

LhxHashTableReader* LhxPartitionInfo::hashTableReader

Hash Table Reader for hash aggregates.

Definition at line 185 of file LhxPartition.h.

Referenced by LhxPlan::generatePartitions(), LhxPartitionInfo(), and open().

LhxPartitionReader LhxPartitionInfo::probeReader

Src partition reader.

It could either be a local reader (probeReader), or a reader passed in from the exec stream in open() method. The latter is used to partition a build input. When the hash table overflows, all the data from the hash table, plus the remaining data from the build partition, as well as the inflight tuple which caused the hash table overflow, need to be repartitioned.

Definition at line 197 of file LhxPartition.h.

Referenced by LhxPlan::generatePartitions(), and open().

LhxPartitionReader* LhxPartitionInfo::reader

Definition at line 198 of file LhxPartition.h.

Referenced by close(), LhxPlan::generatePartitions(), LhxPartitionInfo(), and open().

TupleData LhxPartitionInfo::buildTuple

holding the tuple which is not yet inserted into hash table

Definition at line 203 of file LhxPartition.h.

Referenced by LhxPlan::generatePartitions(), and open().

vector<SharedLhxPartitionWriter> LhxPartitionInfo::writerList

Child partitions for both inputs have to be complete before child plans can be created.

writerList.size() == destPartitionList.size() == numInputs * LhxPlan::LhxChildPartCount

Input filters are used to filter only one input (the build input) of each child partition.

joinFilterList.size() == LhxPlan::LhxChildPartCount

Definition at line 220 of file LhxPartition.h.

Referenced by close(), LhxPlan::generatePartitions(), init(), and open().

vector<SharedLhxPartition> LhxPartitionInfo::destPartitionList

Definition at line 221 of file LhxPartition.h.

Referenced by LhxPlan::createChildren(), and open().

vector<shared_ptr<dynamic_bitset<> > > LhxPartitionInfo::joinFilterList

Definition at line 224 of file LhxPartition.h.

Referenced by LhxPlan::createChildren(), LhxPlan::generatePartitions(), and open().

shared_array<uint> LhxPartitionInfo::filteredRowCountList

Definition at line 225 of file LhxPartition.h.

Referenced by LhxPlan::createChildren(), LhxPlan::generatePartitions(), init(), and open().

vector<shared_array<uint> > LhxPartitionInfo::subPartStatList

Definition at line 233 of file LhxPartition.h.

Referenced by LhxPlan::createChildren(), LhxPlan::generatePartitions(), and open().

uint LhxPartitionInfo::numInputs

Definition at line 235 of file LhxPartition.h.

Referenced by LhxPlan::createChildren(), LhxPlan::generatePartitions(), init(), and open().

uint LhxPartitionInfo::curInputIndex

Definition at line 236 of file LhxPartition.h.

Referenced by LhxPlan::generatePartitions(), and open().

LhxHashInfo* LhxPartitionInfo::hashInfo

Definition at line 238 of file LhxPartition.h.

Referenced by init(), and open().

bool LhxPartitionInfo::partitionMemory

Definition at line 244 of file LhxPartition.h.

Referenced by LhxPlan::generatePartitions(), and open().

---

The documentation for this struct was generated from the following files:

• /home/pub/open/dev/fennel/hashexe/LhxPartition.h
• /home/pub/open/dev/fennel/hashexe/LhxPartition.cpp

---