FlatFileBuffer Class Reference

FlatFileBuffer provides data for FlatFileExecStream. More...

#include <FlatFileBuffer.h>

Inheritance diagram for FlatFileBuffer:

ClosableObject TraceSource List of all members.

Public Member Functions

 FlatFileBuffer (const std::string &path)
 Constructs a buffer.
virtual ~FlatFileBuffer ()
void open ()
 Opens a file and obtains resources needed for reading the file.
void setStorage (char *pBuffer, uint size)
 Sets internal buffers.
uint read ()
 Reads more data into buffer, preserving unread data.
char * getReadPtr ()
 Returns a pointer to the current row.
char * getEndPtr ()
 Returns a pointer to the end of buffer contents.
int getSize ()
 Returns the difference between getEndPtr() and getReadPtr().
bool isFull ()
 Returns whether buffer has been filled to maximum capacity.
bool isComplete ()
 Returns whether entire file has been read.
bool isDone ()
 Returns whether the entire file has been read AND the read pointer is equal to (or past) the end pointer.
void setReadPtr (char *ptr)
 Consumes buffer contents up to pointer.
bool isClosed () const
 
Returns:
whether the object has been closed

void close ()
 Closes this object, releasing any unallocated resources.
virtual void initTraceSource (SharedTraceTarget pTraceTarget, std::string name)
 For use when initialization has to be deferred until after construction.
void trace (TraceLevel level, std::string message) const
 Records a trace message.
bool isTracing () const
 
Returns:
true iff tracing is enabled for this source

bool isTracingLevel (TraceLevel level) const
 Determines whether a particular level is being traced.
TraceTarget & getTraceTarget () const
 
Returns:
the TraceTarget for this source

SharedTraceTarget getSharedTraceTarget () const
 
Returns:
the SharedTraceTarget for this source

std::string getTraceSourceName () const
 Gets the name of this source.
void setTraceSourceName (std::string const &n)
 Sets the name of this source.
TraceLevel getMinimumTraceLevel () const
void disableTracing ()

Protected Attributes

bool needsClose

Private Member Functions

void closeImpl ()
 Must be implemented by derived class to release any resources.

Private Attributes

std::string path
FILE * pFile
FileSize filePosition
char * pBuffer
uint bufferSize
uint contentSize
char * pCurrent

Detailed Description

FlatFileBuffer provides data for FlatFileExecStream.

The data is provided as a stream of characters stored in a buffer.

This implementation sequentially reads an ascii file page by page. It handles the details of file access, and manages a pointer to unread data. The file is opened during a call to open() and is closed either by an explicit call to close() or on deallocation.

Storage must be provided with a call to setStorage() before calling read(). Once read() has been called, getReadPtr() will return a pointer to the current data pointer. A call to getEnd() will return a pointer to the end of data. A call to isFull() describes whether more data can be read without consuming some. A call to isComplete() describes whether or not the file has been completely read.

The method setReadPtr() may be used to consume input data. Upon reaching the end of the buffer, subsequent calls to read() are used to fetch more data. Unread data is preserved, but the current read pointer may be updated.

FIXME: This class should use a special character pointer. It may be updated to handle unicode. It may be refined to prefetch pages for better performance.

Author:
John Pham
Version:
Id
//open/dev/fennel/flatfile/FlatFileBuffer.h#2

Definition at line 73 of file FlatFileBuffer.h.

Constructor & Destructor Documentation

FlatFileBuffer::FlatFileBuffer ( const std::string &  path  ) 

Constructs a buffer.

Parameters:
path location of flat file to be opened

Definition at line 31 of file FlatFileBuffer.cpp.

References bufferSize, contentSize, pBuffer, pCurrent, and pFile.

00032 {
00033     this->path = path;
00034     pBuffer = NULL;
00035     bufferSize = 0;
00036     contentSize = 0;
00037     pCurrent = NULL;
00038     pFile = NULL;
00039 }

FlatFileBuffer::~FlatFileBuffer (  )  [virtual]

Definition at line 41 of file FlatFileBuffer.cpp.

00042 {
00043 }

Member Function Documentation

void FlatFileBuffer::closeImpl (  )  [private, virtual]

Must be implemented by derived class to release any resources.

Implements ClosableObject.

Definition at line 45 of file FlatFileBuffer.cpp.

References contentSize, pCurrent, and pFile.

00046 {
00047     if (pFile) {
00048         fclose(pFile);
00049         pFile = NULL;
00050     }
00051     contentSize = 0;
00052     pCurrent = NULL;
00053 }

void FlatFileBuffer::open (  ) 

Opens a file and obtains resources needed for reading the file.

Definition at line 63 of file FlatFileBuffer.cpp.

References ClosableObject::close(), contentSize, filePosition, ClosableObject::needsClose, path, pCurrent, and pFile.

00064 {
00065     // in case we are reopening
00066     close();
00067 
00068     // NOTE jvs 17-Oct-2008:  we use fopen here instead of ifstream
00069     // in case we want to support popen("gunzip") in the future.
00070 
00071     pFile = fopen(path.c_str(), "r");
00072     if (!pFile) {
00073         throw FennelExcn(
00074             FennelResource::instance().readDataFailed(path));
00075     }
00076 
00077     needsClose = true;
00078 
00079     filePosition = 0;
00080     contentSize = 0;
00081     pCurrent = NULL;
00082 }

void FlatFileBuffer::setStorage ( char *  pBuffer,
uint  size 
)

Sets internal buffers.

Parameters:
pBuffer storage for characters read
size size of buffer, in characters

Definition at line 55 of file FlatFileBuffer.cpp.

References bufferSize, contentSize, and pCurrent.

00056 {
00057     this->pBuffer = pBuffer;
00058     bufferSize = size;
00059     contentSize = 0;
00060     pCurrent = NULL;
00061 }

uint FlatFileBuffer::read (  ) 

Reads more data into buffer, preserving unread data.

Invalidates previous read pointers.

Returns:
number of characters read

Definition at line 84 of file FlatFileBuffer.cpp.

References bufferSize, contentSize, filePosition, free(), getEndPtr(), path, pBuffer, pCurrent, and pFile.

Referenced by FlatFileExecStreamTest::checkRead().

00085 {
00086     int residual = 0;
00087     if (pCurrent != NULL) {
00088         assert(pBuffer <= pCurrent && pCurrent <= getEndPtr());
00089         residual = getEndPtr() - pCurrent;
00090         memmove(pBuffer, pCurrent, residual);
00091         contentSize = residual;
00092     }
00093     pCurrent = pBuffer;
00094 
00095     uint free = bufferSize - residual;
00096     char *target = pBuffer + residual;
00097     uint targetSize = free;
00098 
00099     size_t actualSize = fread(target, 1, targetSize, pFile);
00100     if (ferror(pFile)) {
00101         // FIXME jvs 19-Oct-2008:  the error message here is confusingly
00102         // switched with the error message for opening the file.
00103         throw FennelExcn(
00104             FennelResource::instance().dataTransferFailed(path, targetSize));
00105     }
00106     filePosition += actualSize;
00107     contentSize = residual + actualSize;
00108     return actualSize;
00109 }

char * FlatFileBuffer::getReadPtr (  ) 

Returns a pointer to the current row.

Definition at line 111 of file FlatFileBuffer.cpp.

References pBuffer, and pCurrent.

Referenced by FlatFileExecStreamTest::checkRead(), getSize(), and isDone().

00112 {
00113     assert(pCurrent != NULL && pBuffer <= pCurrent);
00114     return pCurrent;
00115 }

char * FlatFileBuffer::getEndPtr (  ) 

Returns a pointer to the end of buffer contents.

Definition at line 117 of file FlatFileBuffer.cpp.

References contentSize, and pBuffer.

Referenced by FlatFileExecStreamTest::checkRead(), getSize(), isDone(), read(), and setReadPtr().

00118 {
00119     assert(pBuffer != NULL);
00120     return pBuffer + contentSize;
00121 }

int FlatFileBuffer::getSize (  ) 

Returns the difference between getEndPtr() and getReadPtr().

Definition at line 123 of file FlatFileBuffer.cpp.

References getEndPtr(), and getReadPtr().

00124 {
00125     return getEndPtr() - getReadPtr();
00126 }

bool FlatFileBuffer::isFull (  ) 

Returns whether buffer has been filled to maximum capacity.

Definition at line 128 of file FlatFileBuffer.cpp.

References bufferSize, contentSize, pBuffer, and pCurrent.

00129 {
00130     assert(pBuffer != NULL);
00131     return (pCurrent == pBuffer && contentSize == bufferSize);
00132 }

bool FlatFileBuffer::isComplete (  ) 

Returns whether entire file has been read.

Definition at line 134 of file FlatFileBuffer.cpp.

References pFile.

Referenced by isDone().

00135 {
00136     assert(pFile);
00137     return feof(pFile);
00138 }

bool FlatFileBuffer::isDone (  ) 

Returns whether the entire file has been read AND the read pointer is equal to (or past) the end pointer.

See also:
isComplete()

getReadPtr()

getEndPtr()

Definition at line 140 of file FlatFileBuffer.cpp.

References getEndPtr(), getReadPtr(), and isComplete().

00141 {
00142     return isComplete() && getReadPtr() >= getEndPtr();
00143 }

void FlatFileBuffer::setReadPtr ( char *  ptr  ) 

Consumes buffer contents up to pointer.

Definition at line 145 of file FlatFileBuffer.cpp.

References getEndPtr(), pBuffer, and pCurrent.

00146 {
00147     assert(pBuffer <= pCurrent && pCurrent <= ptr && ptr <= getEndPtr());
00148     pCurrent = ptr;
00149 }

bool ClosableObject::isClosed (  )  const [inline, inherited]

Returns:
whether the object has been closed

Definition at line 58 of file ClosableObject.h.

00059     {
00060         return !needsClose;
00061     }

void ClosableObject::close (  )  [inherited]

Closes this object, releasing any unallocated resources.

Reimplemented in CollectExecStream, CorrelationJoinExecStream, LcsClusterAppendExecStream, and LcsClusterReplaceExecStream.

Definition at line 39 of file ClosableObject.cpp.

References ClosableObject::closeImpl(), and ClosableObject::needsClose.

Referenced by CacheImpl< PageT, VictimPolicyT >::allocatePages(), LcsRowScanBaseExecStream::closeImpl(), ExecStreamGraphImpl::closeImpl(), open(), ClosableObjectDestructor::operator()(), and Segment::~Segment().

00040 {
00041     if (!needsClose) {
00042         return;
00043     }
00044     needsClose = false;
00045     closeImpl();
00046 }

void TraceSource::initTraceSource ( SharedTraceTarget  pTraceTarget,
std::string  name 
) [virtual, inherited]

For use when initialization has to be deferred until after construction.

Parameters:
pTraceTarget the TraceTarget to which messages will be sent
name the name of this source

Definition at line 46 of file TraceSource.cpp.

References TraceSource::isTracing(), TraceSource::minimumLevel, TraceSource::name, TraceSource::pTraceTarget, and TRACE_OFF.

Referenced by TestBase::beforeTestCase(), TestBase::TestBase(), and TraceSource::TraceSource().

00049 {
00050     assert(!pTraceTarget.get());
00051 
00052     pTraceTarget = pTraceTargetInit;
00053     name = nameInit;
00054     if (isTracing()) {
00055         minimumLevel = pTraceTarget->getSourceTraceLevel(name);
00056     } else {
00057         minimumLevel = TRACE_OFF;
00058     }
00059 }

void TraceSource::trace ( TraceLevel  level,
std::string  message 
) const [inherited]

Records a trace message.

Normally only called via FENNEL_TRACE.

Parameters:
level severity level of event being trace
message the text of the message

Definition at line 61 of file TraceSource.cpp.

References TraceSource::getTraceTarget(), TraceSource::isTracing(), TraceSource::name, and TraceTarget::notifyTrace().

Referenced by Calculator::exec(), and ExecStreamScheduler::traceStreamBufferContents().

00062 {
00063     if (isTracing()) {
00064         getTraceTarget().notifyTrace(name,level,message);
00065     }
00066 }

bool TraceSource::isTracing (  )  const [inline, inherited]

Returns:
true iff tracing is enabled for this source

Definition at line 88 of file TraceSource.h.

Referenced by TraceSource::initTraceSource(), CalcExecStream::prepare(), and TraceSource::trace().

00089     {
00090         return pTraceTarget.get() ? true : false;
00091     }

bool TraceSource::isTracingLevel ( TraceLevel  level  )  const [inline, inherited]

Determines whether a particular level is being traced.

Parameters:
level trace level to test
Returns:
true iff tracing is enabled for the given level

Definition at line 100 of file TraceSource.h.

Referenced by ExecStreamScheduler::addGraph(), SimpleExecStreamGovernor::assignCachePages(), SimpleExecStreamGovernor::distributeCachePages(), Calculator::exec(), ExecStreamScheduler::ExecStreamScheduler(), LcsClusterNodeWriter::getLastClusterPageForWrite(), LcsClusterNodeWriter::moveFromTempToIndex(), JavaSinkExecStream::stuffByteBuffer(), and ExecStreamScheduler::traceStreamBuffers().

00101     {
00102         return level >= minimumLevel;
00103     }

TraceTarget& TraceSource::getTraceTarget (  )  const [inline, inherited]

Returns:
the TraceTarget for this source

Definition at line 108 of file TraceSource.h.

Referenced by TraceSource::trace().

00109     {
00110         assert(isTracing());
00111         return *(pTraceTarget.get());
00112     }

SharedTraceTarget TraceSource::getSharedTraceTarget (  )  const [inline, inherited]

Returns:
the SharedTraceTarget for this source

Definition at line 117 of file TraceSource.h.

Referenced by Database::init(), LcsClusterAppendExecStream::initLoad(), and CalcExecStream::prepare().

00118     {
00119         return pTraceTarget;
00120     }

std::string TraceSource::getTraceSourceName (  )  const [inline, inherited]

Gets the name of this source.

Useful to construct nested names for subcomponents that are also TraceSources.

Returns:
the name

Definition at line 127 of file TraceSource.h.

Referenced by LcsClusterAppendExecStream::initLoad().

00128     {
00129         return name;
00130     }

void TraceSource::setTraceSourceName ( std::string const &  n  )  [inline, inherited]

Sets the name of this source.

Useful to construct dynamic names for fine-grained filtering.

Definition at line 136 of file TraceSource.h.

00137     {
00138         name = n;
00139     }

TraceLevel TraceSource::getMinimumTraceLevel (  )  const [inline, inherited]

Definition at line 141 of file TraceSource.h.

00142     {
00143         return minimumLevel;
00144     }

void TraceSource::disableTracing (  )  [inherited]

Definition at line 68 of file TraceSource.cpp.

References TraceSource::minimumLevel, TraceSource::pTraceTarget, and TRACE_OFF.

Referenced by TestBase::afterTestCase().

00069 {
00070     pTraceTarget.reset();
00071     minimumLevel = TRACE_OFF;
00072 }

Member Data Documentation

std::string FlatFileBuffer::path [private]

Definition at line 76 of file FlatFileBuffer.h.

Referenced by open(), and read().

FILE* FlatFileBuffer::pFile [private]

Definition at line 77 of file FlatFileBuffer.h.

Referenced by closeImpl(), FlatFileBuffer(), isComplete(), open(), and read().

FileSize FlatFileBuffer::filePosition [private]

Definition at line 78 of file FlatFileBuffer.h.

Referenced by open(), and read().

char* FlatFileBuffer::pBuffer [private]

Definition at line 80 of file FlatFileBuffer.h.

Referenced by FlatFileBuffer(), getEndPtr(), getReadPtr(), isFull(), read(), and setReadPtr().

uint FlatFileBuffer::bufferSize [private]

Definition at line 81 of file FlatFileBuffer.h.

Referenced by FlatFileBuffer(), isFull(), read(), and setStorage().

uint FlatFileBuffer::contentSize [private]

Definition at line 81 of file FlatFileBuffer.h.

Referenced by closeImpl(), FlatFileBuffer(), getEndPtr(), isFull(), open(), read(), and setStorage().

char* FlatFileBuffer::pCurrent [private]

Definition at line 82 of file FlatFileBuffer.h.

Referenced by closeImpl(), FlatFileBuffer(), getReadPtr(), isFull(), open(), read(), setReadPtr(), and setStorage().

bool ClosableObject::needsClose [protected, inherited]

Definition at line 44 of file ClosableObject.h.

Referenced by SegStreamAllocation::beginWrite(), ExecStreamGraphImpl::clear(), ClosableObject::ClosableObject(), ClosableObject::close(), open(), ExecStreamGraphImpl::open(), ExecStream::open(), and ClosableObject::~ClosableObject().

The documentation for this class was generated from the following files:
Generated on Mon Jun 22 04:00:32 2009 for Fennel by  doxygen 1.5.1