|
| SSMBase (Int aBucketSize=0, uInt aCacheSize=1) |
| Create a Standard storage manager with default name SSM. More...
|
|
| SSMBase (const String &aDataManName, Int aBucketSize=0, uInt aCacheSize=1) |
| Create a Standard storage manager with the given name. More...
|
|
| SSMBase (const String &aDataManName, const Record &spec) |
| Create a Standard storage manager with the given name. More...
|
|
| ~SSMBase () |
|
virtual DataManager * | clone () const |
| Clone this object. More...
|
|
virtual String | dataManagerType () const |
| Get the type name of the data manager (i.e. More...
|
|
virtual String | dataManagerName () const |
| Get the name given to the storage manager (in the constructor). More...
|
|
virtual Record | dataManagerSpec () const |
| Record a record containing data manager specifications. More...
|
|
virtual Record | getProperties () const |
| Get data manager properties that can be modified. More...
|
|
virtual void | setProperties (const Record &spec) |
| Modify data manager properties. More...
|
|
uInt | getVersion () const |
| Get the version of the class. More...
|
|
void | setCacheSize (uInt aCacheSize, Bool canExceedNrBuckets=True) |
| Set the cache size (in buckets). More...
|
|
uInt | getCacheSize () const |
| Get the current cache size (in buckets). More...
|
|
void | clearCache () |
| Clear the cache used by this storage manager. More...
|
|
virtual void | showCacheStatistics (ostream &anOs) const |
| Show the statistics of all caches used. More...
|
|
void | showIndexStatistics (ostream &anOs) const |
| Show statistics of all indices used. More...
|
|
void | showBaseStatistics (ostream &anOs) const |
| Show statistics of the Base offsets/index etc. More...
|
|
uInt | getBucketSize () const |
| Get the bucket size. More...
|
|
rownr_t | getNRow () const |
| Get the number of rows in this storage manager. More...
|
|
virtual Bool | canAddRow () const |
| The storage manager can add rows. More...
|
|
virtual Bool | canRemoveRow () const |
| The storage manager can delete rows. More...
|
|
virtual Bool | canAddColumn () const |
| The storage manager can add columns. More...
|
|
virtual Bool | canRemoveColumn () const |
| The storage manager can delete columns. More...
|
|
SSMColumn & | getColumn (uInt aColNr) |
| Get access to the given column. More...
|
|
SSMIndex & | getIndex (uInt anIdxNr) |
| Get access to the given Index. More...
|
|
void | setBucketDirty () |
| Make the current bucket in the cache dirty (i.e. More...
|
|
StManArrayFile * | openArrayFile (ByteIO::OpenOption anOpt) |
| Open (if needed) the file for indirect arrays with the given mode. More...
|
|
char * | find (rownr_t aRowNr, uInt aColNr, rownr_t &aStartRow, rownr_t &anEndRow, const String &colName) |
| Find the bucket containing the column and row and return the pointer to the beginning of the column data in that bucket. More...
|
|
uInt | getNewBucket () |
| Add a new bucket and get its bucket number. More...
|
|
char * | getBucket (uInt aBucketNr) |
| Read the bucket (if needed) and return the pointer to it. More...
|
|
void | removeBucket (uInt aBucketNr) |
| Remove a bucket from the bucket cache. More...
|
|
uInt | getRowsPerBucket (uInt aColumn) const |
| Get rows per bucket for the given column. More...
|
|
SSMStringHandler * | getStringHandler () |
| Return a pointer to the (one and only) StringHandler object. More...
|
|
| DataManager () |
| Default constructor. More...
|
|
virtual | ~DataManager () |
|
void | dataManagerInfo (Record &info) const |
| Add SEQNR and SPEC (the DataManagerSpec subrecord) to the info. More...
|
|
virtual Bool | isStorageManager () const |
| Is the data manager a storage manager? The default is yes. More...
|
|
virtual Bool | canReallocateColumns () const |
| Tell if the data manager wants to reallocate the data manager column objects. More...
|
|
virtual DataManagerColumn * | reallocateColumn (DataManagerColumn *column) |
| Reallocate the column object if it is part of this data manager. More...
|
|
uInt | sequenceNr () const |
| Get the (unique) sequence nr of this data manager. More...
|
|
uInt | ncolumn () const |
| Get the nr of columns in this data manager (can be zero). More...
|
|
Bool | asBigEndian () const |
| Have the data to be stored in big or little endian canonical format? More...
|
|
const TSMOption & | tsmOption () const |
| Get the TSM option. More...
|
|
MultiFileBase * | multiFile () |
| Get the MultiFile pointer (can be 0). More...
|
|
String | keywordName (const String &keyword) const |
| Compose a keyword name from the given keyword appended with the sequence number (e.g. More...
|
|
String | fileName () const |
| Compose a unique filename from the table name and sequence number. More...
|
|
ByteIO::OpenOption | fileOption () const |
| Get the AipsIO option of the underlying file. More...
|
|
virtual Bool | isRegular () const |
| Is this a regular storage manager? It is regular if it allows addition of rows and writing data in them. More...
|
|
Table & | table () const |
| Get the table this object is associated with. More...
|
|
virtual Bool | canRenameColumn () const |
| Does the data manager allow to rename columns? (default yes) More...
|
|
virtual void | setMaximumCacheSize (uInt nMiB) |
| Set the maximum cache size (in bytes) to be used by a storage manager. More...
|
|
virtual void | showCacheStatistics (std::ostream &) const |
| Show the data manager's IO statistics. More...
|
|
DataManagerColumn * | createScalarColumn (const String &columnName, int dataType, const String &dataTypeId) |
| Create a column in the data manager on behalf of a table column. More...
|
|
DataManagerColumn * | createDirArrColumn (const String &columnName, int dataType, const String &dataTypeId) |
| Create a direct array column. More...
|
|
DataManagerColumn * | createIndArrColumn (const String &columnName, int dataType, const String &dataTypeId) |
| Create an indirect array column. More...
|
|
DataManager * | getClone () const |
| Has the object already been cloned? More...
|
|
void | setClone (DataManager *clone) const |
| Set the pointer to the clone. More...
|
|
|
| SSMBase (const SSMBase &that) |
| Copy constructor (only meant for clone function). More...
|
|
SSMBase & | operator= (const SSMBase &that) |
| Assignment cannot be used. More...
|
|
void | recreate () |
| (Re)create the index, file, and cache object. More...
|
|
virtual Bool | hasMultiFileSupport () const |
| The data manager supports use of MultiFile. More...
|
|
virtual Bool | flush (AipsIO &, Bool doFsync) |
| Flush and optionally fsync the data. More...
|
|
virtual void | create64 (rownr_t aNrRows) |
| Let the storage manager create files as needed for a new table. More...
|
|
virtual rownr_t | open64 (rownr_t aRowNr, AipsIO &) |
| Open the storage manager file for an existing table, read in the data, and let the SSMColumn objects read their data. More...
|
|
virtual rownr_t | resync64 (rownr_t aRowNr) |
| Resync the storage manager with the new file contents. More...
|
|
virtual void | reopenRW () |
| Reopen the storage manager files for read/write. More...
|
|
virtual void | deleteManager () |
| The data manager will be deleted (because all its columns are requested to be deleted). More...
|
|
void | init () |
| Let the storage manager initialize itself (upon creation). More...
|
|
uInt | setBucketSize () |
| Determine and set the bucket size. More...
|
|
uInt | getNrIndices () const |
| Get the number of indices in use. More...
|
|
virtual void | addRow64 (rownr_t aNrRows) |
| Add rows to the storage manager. More...
|
|
virtual void | removeRow64 (rownr_t aRowNr) |
| Delete a row from all columns. More...
|
|
virtual void | addColumn (DataManagerColumn *) |
| Do the final addition of a column. More...
|
|
virtual void | removeColumn (DataManagerColumn *) |
| Remove a column from the data file. More...
|
|
virtual DataManagerColumn * | makeScalarColumn (const String &aName, int aDataType, const String &aDataTypeID) |
| Create a column in the storage manager on behalf of a table column. More...
|
|
virtual DataManagerColumn * | makeDirArrColumn (const String &aName, int aDataType, const String &aDataTypeID) |
| Create a direct array column. More...
|
|
virtual DataManagerColumn * | makeIndArrColumn (const String &aName, int aDataType, const String &aDataTypeID) |
| Create an indirect array column. More...
|
|
BucketCache & | getCache () |
| Get the cache object. More...
|
|
void | makeCache () |
| Construct the cache object (if not constructed yet). More...
|
|
void | readHeader () |
| Read the header. More...
|
|
void | readIndexBuckets () |
| Read the index from its buckets. More...
|
|
void | writeIndex () |
| Write the header and the indices. More...
|
|
Base class of the Standard Storage Manager
Intended use:
Internal
Review Status
- Reviewed By:
- UNKNOWN
- Date Reviewed:
- before2004/08/25
- Test programs:
- tStandardStMan
Prerequisite
Etymology
SSMBase is the base class of the Standard Storage Manager.
Synopsis
The global principles of this class are described in StandardStMan.
The Standard Storage Manager divides the data file in equally sized chunks called buckets. There are 3 types of buckets:
-
Data buckets containing the fixed length data (scalars and direct arrays of data type Int, Float, Bool, etc.). For variable shaped data (strings and indirect arrays) they contain references to the actual data position in the string buckets or in an external file.
-
String buckets containing strings and array of strings.
-
Index buckets containing the index info for the data buckets.
Bucket access is handled by class BucketCache. It also keeps a list of free buckets. A bucket is freed when it is not needed anymore (e.g. all data from it are deleted).
Data buckets form the main part of the SSM. The data can be viewed as a few streams of buckets, where each stream contains the data of a given number of columns. Each stream has an SSMIndex object describing the number of rows stored in each data bucket of the stream. The SSM starts with a single bucket stream (holding all columns), but when columns are added, new bucket streams might be created.
For example, we have an SSM with a bucket size of 100 bytes. There are 5 Int columns (A,B,C,D,E) each taking 4 bytes per row. Column A, B, C, and D are stored in bucket stream 1, while column E is stored in bucket stream 2. So in stream 1 each bucket can hold 6 rows, while in stream 2 each bucket can hold 25 rows. For a 100 row table it will result in 17+4 data buckets.
A few classes collaborate to make it work:
-
Each bucket stream has an SSMIndex object to map row number to bucket number. Note that in principle each bucket in a stream contains the same number of rows. However, when a row is deleted it is removed from its bucket shifting the remainder to the left. Data in the next buckets is not shifted, so that bucket has now one row less.
-
For each column SSMBase knows to which bucket stream it belongs and at which offset the column starts in a bucket. Note that column data in a bucket are adjacent, which is done to make it easier to use the ColumnCache object in SSMColumn and to be able to efficiently store Bool values as bits.
-
Each column has an SSMColumn object knowing how many bits each data cell takes in a bucket. The SSMColumn objects handle all access to data in the columns (using SSMBase and SSMIndex).
String buckets are used by class SSMStringHandler to store scalar strings and fixed and variable shaped arrays of strings. The bucketnr, offset, and length of such string (arrays) are stored in the data buckets.
Indirect arrays of other data types are also stored indirectly and their offset is stored in the data buckets. Such arrays are handled by class StIndArray which uses an extra file to store the arrays.
Index buckets are used by SSMBase to make the SSMIndex data persistent. It uses alternately 2 sets of index buckets. In that way there is always an index availanle in case the system crashes. If possible 2 halfs of a single bucket are used alternately, otherwise separate buckets are used.
Motivation
The public interface of SSMBase is quite large, because the other internal SSM classes need these functions. To have a class with a minimal interface for the normal user, class StandardStMan
is derived from it.
StandardStMan needs an isA- instead of hasA-relation to be able to bind columns to it in class SetupNewTable.
To Do
-
Remove AipsIO argument from open and close.
-
When only 1 bucket in use addcolumn can check if there's enough room to fit the new column (so rearange the bucket) in the free row space.
Definition at line 158 of file SSMBase.h.