Abstract base class to combine multiple logical files in a single one. More...

#include <MultiFileBase.h>

Public Member Functions
	MultiFileBase (const String &name, Int blockSize, Bool useODirect)
	Create a MultiFileBase object with the given name.

virtual	~MultiFileBase ()
	The destructor flushes dirty blocks and closes the container file.

	MultiFileBase (const MultiFileBase &)=delete
	Forbid copy constructor.

MultiFileBase &	operator= (const MultiFileBase &)=delete
	Forbid assignment.

virtual std::shared_ptr< MultiFileBase >	makeNested (const std::shared_ptr< MultiFileBase > &parent, const String &name, ByteIO::OpenOption, Int blockSize) const =0
	Make the correct MultiFileBase object for a nested file.

String	fileName () const
	Get the file name of the MultiFileBase container file.

Bool	isWritable () const
	Is the container file writable?

Int	openFile (const String &name)
	Open the given logical file and return its file id.

Int	createFile (const String &name, ByteIO::OpenOption=ByteIO::New)
	Create a new logical file and return its file id.

void	flushFile (Int fileId)
	Flush the possible dirty buffer of the given logical file.

void	closeFile (Int fileId)
	Close a logical file.

void	deleteFile (Int fileId)
	Delete a logical file.

Int64	fileSize (Int fileId) const
	Get the size of a logical file.

Int64	read (Int fileId, void *buffer, Int64 size, Int64 offset)
	Read a block at the given offset in the logical file.

Int64	write (Int fileId, const void *buffer, Int64 size, Int64 offset)
	Write a block at the given offset in the logical file.

void	truncate (Int fileId, Int64 size)
	Truncate the logical file to the given size.

virtual void	reopenRW ()=0
	Reopen the underlying file for read/write access.

void	flush ()
	Flush the file by writing all dirty data and all header info.

Int64	blockSize () const
	Get the block size used.

uInt	nfile () const
	Get the nr of logical files.

Int64	nblock () const
	Get the total nr of data blocks used.

const std::vector< MultiFileInfo > &	info () const
	Get the info object (for test purposes mainly).

const std::vector< Int64 > &	freeBlocks () const
	Get the free blocks (for test purposes mainly).

Int	fileId (const String &name, Bool throwExcp=True) const
	Return the file id of a file in the MultiFileBase object.

Bool	useODirect () const
	Is O_DIRECT used?

Static Public Member Functions
static std::shared_ptr< MultiFileBase >	openMF (const String &fileName)
	Open the correct MultiFileBase (as plain or HDF5).

Protected Member Functions
void	resync ()
	Resync with another process by clearing the buffers and rereading the header.

virtual void	fsync ()=0
	Fsync the file (i.e., force the data to be physically written).

void	setNewFile ()
	Set the flags and blockSize for a new MultiFile/HDF5.

Protected Attributes
String	itsName

Int64	itsBlockSize

Int64	itsNrBlock

Int64	itsHdrCounter

std::vector< MultiFileInfo >	itsInfo

std::shared_ptr< MultiFileBuffer >	itsBuffer

Bool	itsUseODirect

Bool	itsWritable

Bool	itsChanged

std::vector< Int64 >	itsFreeBlocks

Private Member Functions
void	writeDirty (MultiFileInfo &info)
	Write the dirty block and clear dirty flag.

Int	addFile (const String &name)
	Add a file to the MultiFileBase object.

virtual void	doOpenFile (MultiFileInfo &)=0
	Do the class-specific actions on opening a logical file.

virtual void	doCloseFile (MultiFileInfo &)=0
	Do the class-specific actions on closing a logical file.

virtual void	doAddFile (MultiFileInfo &)=0
	Do the class-specific actions on adding a logical file.

virtual void	doDeleteFile (MultiFileInfo &)=0
	Do the class-specific actions on deleting a logical file.

virtual void	doTruncateFile (MultiFileInfo &info, uInt64 nrblk)=0
	Truncate the container file to `nrblk` blocks.

virtual void	doFlushFile ()=0
	Flush the container file.

virtual void	close ()=0
	Flush and close the container file.

virtual void	writeHeader ()=0
	Write the header info.

virtual void	readHeader (Bool always=True)=0
	Read the header info.

virtual void	extend (MultiFileInfo &info, Int64 lastblk)=0
	Extend a logical file to fit lastblk.

virtual void	writeBlock (MultiFileInfo &info, Int64 blknr, const void *buffer)=0
	Write a data block of a logical file into the container file.

virtual void	readBlock (MultiFileInfo &info, Int64 blknr, void *buffer)=0
	Read a data block of a logical file from the container file.

Detailed Description

Abstract base class to combine multiple logical files in a single one.

Intended use:

Public interface

Review Status

Test programs:: tMultiFile

Synopsis

This class is the abstract base class for classes defining a container file holding multiple logical files. These classes are meant as a container files for the storage manager files of a table to reduce the number of files used (especially for Lustre) and to reduce the number of open files (especially when concatenating tables). The derived classes MultiFile and MultiHDF5 implement such container files using a regular file and HDF5, resp.

MultiFileBase implements several functions with common functionality for the derived classes.

A logical file is represented by an MFFileIO object, which is derived from ByteIO and as such part of the casacore IO framework. It makes it possible for applications to access a logical file in the same way as a regular file.

Definition at line 127 of file MultiFileBase.h.

Constructor & Destructor Documentation

◆ MultiFileBase() [1/2]

casacore::MultiFileBase::MultiFileBase	(	const String &	name,
		Int	blockSize,
		Bool	useODirect
	)

Create a MultiFileBase object with the given name.

Upon creation of the container file the block size can be given. If <=0, it uses the block size of the file system the file is on, but it will not be less than the absolute value of the given block size.
If useODIrect=True, it means that O_DIRECT is used. If the OS does not support it (as determined at configure time), the flag will always be set to False. If True, the data buffers will have a proper alignment and size (as needed by O_DIRECT).

◆ ~MultiFileBase()

virtual casacore::MultiFileBase::~MultiFileBase ( )

virtual

The destructor flushes dirty blocks and closes the container file.

◆ MultiFileBase() [2/2]

casacore::MultiFileBase::MultiFileBase ( const MultiFileBase & )

delete

Forbid copy constructor.

Member Function Documentation

◆ addFile()

Int casacore::MultiFileBase::addFile ( const String & name )

private

Add a file to the MultiFileBase object.

It returns the file id. Only the base name of the given file name is used. In this way the MultiFileBase container file can be moved.

◆ blockSize()

Int64 casacore::MultiFileBase::blockSize ( ) const

inline

Get the block size used.

Definition at line 212 of file MultiFileBase.h.

References itsBlockSize.

◆ close()

virtual void casacore::MultiFileBase::close ( )

privatepure virtual

Flush and close the container file.

Implemented in casacore::MultiHDF5.

◆ closeFile()

void casacore::MultiFileBase::closeFile ( Int fileId )

Close a logical file.

It flushes and deallocates its buffer.

◆ createFile()

Int casacore::MultiFileBase::createFile	(	const String &	name,
		ByteIO::OpenOption	= `ByteIO::New`
	)

Create a new logical file and return its file id.

Only the base name of the given file name is used. In this way the MultiFileBase container file can be moved. If the logical file already exists, it is deleted if ByteIO::New is given. Otherwise an exception is thrown. It allocates the internal buffer of the logical file.

◆ deleteFile()

void casacore::MultiFileBase::deleteFile ( Int fileId )

Delete a logical file.

It adds its blocks to the free block list.

◆ doAddFile()

virtual void casacore::MultiFileBase::doAddFile ( MultiFileInfo & )

privatepure virtual

Do the class-specific actions on adding a logical file.

Implemented in casacore::MultiHDF5.

◆ doCloseFile()

virtual void casacore::MultiFileBase::doCloseFile ( MultiFileInfo & )

privatepure virtual

Do the class-specific actions on closing a logical file.

Implemented in casacore::MultiHDF5.

◆ doDeleteFile()

virtual void casacore::MultiFileBase::doDeleteFile ( MultiFileInfo & )

privatepure virtual

Do the class-specific actions on deleting a logical file.

Implemented in casacore::MultiHDF5.

◆ doFlushFile()

virtual void casacore::MultiFileBase::doFlushFile ( )

privatepure virtual

Flush the container file.

Implemented in casacore::MultiHDF5.

◆ doOpenFile()

virtual void casacore::MultiFileBase::doOpenFile ( MultiFileInfo & )

privatepure virtual

Do the class-specific actions on opening a logical file.

Implemented in casacore::MultiHDF5.

◆ doTruncateFile()

virtual void casacore::MultiFileBase::doTruncateFile	(	MultiFileInfo &	info,
		uInt64	nrblk
	)

privatepure virtual

Truncate the container file to nrblk blocks.

Implemented in casacore::MultiHDF5.

◆ extend()

virtual void casacore::MultiFileBase::extend	(	MultiFileInfo &	info,
		Int64	lastblk
	)

privatepure virtual

Extend a logical file to fit lastblk.

Implemented in casacore::MultiHDF5.

◆ fileId()

Int casacore::MultiFileBase::fileId	(	const String &	name,
		Bool	throwExcp = `True`
	)		const

Return the file id of a file in the MultiFileBase object.

If the name is unknown, an exception is thrown if throwExcp is set. Otherwise it returns -1.

◆ fileName()

String casacore::MultiFileBase::fileName ( ) const

inline

Get the file name of the MultiFileBase container file.

Definition at line 158 of file MultiFileBase.h.

References itsName.

◆ fileSize()

Int64 casacore::MultiFileBase::fileSize ( Int fileId ) const

Get the size of a logical file.

◆ flush()

void casacore::MultiFileBase::flush ( )

Flush the file by writing all dirty data and all header info.

◆ flushFile()

void casacore::MultiFileBase::flushFile ( Int fileId )

Flush the possible dirty buffer of the given logical file.

◆ freeBlocks()

const std::vector< Int64 > & casacore::MultiFileBase::freeBlocks ( ) const

inline

Get the free blocks (for test purposes mainly).

Definition at line 227 of file MultiFileBase.h.

References itsFreeBlocks.

◆ fsync()

virtual void casacore::MultiFileBase::fsync ( )

protectedpure virtual

Fsync the file (i.e., force the data to be physically written).

Implemented in casacore::MultiFile, and casacore::MultiHDF5.

◆ info()

const std::vector< MultiFileInfo > & casacore::MultiFileBase::info ( ) const

inline

Get the info object (for test purposes mainly).

Definition at line 223 of file MultiFileBase.h.

References itsInfo.

Referenced by writeDirty().

◆ isWritable()

Bool casacore::MultiFileBase::isWritable ( ) const

inline

Is the container file writable?

Definition at line 162 of file MultiFileBase.h.

References itsWritable.

◆ makeNested()

virtual std::shared_ptr< MultiFileBase > casacore::MultiFileBase::makeNested	(	const std::shared_ptr< MultiFileBase > &	parent,
		const String &	name,
		ByteIO::OpenOption	,
		Int	blockSize
	)		const

pure virtual

Make the correct MultiFileBase object for a nested file.

Implemented in casacore::MultiFile, and casacore::MultiHDF5.

◆ nblock()

Int64 casacore::MultiFileBase::nblock ( ) const

inline

Get the total nr of data blocks used.

Definition at line 219 of file MultiFileBase.h.

References itsNrBlock.

◆ nfile()

uInt casacore::MultiFileBase::nfile ( ) const

Get the nr of logical files.

◆ openFile()

Int casacore::MultiFileBase::openFile ( const String & name )

Open the given logical file and return its file id.

If the name is unknown, an exception is thrown. It allocates the internal buffer of the logical file.

◆ openMF()

static std::shared_ptr< MultiFileBase > casacore::MultiFileBase::openMF ( const String & fileName )

static

Open the correct MultiFileBase (as plain or HDF5).

◆ operator=()

MultiFileBase & casacore::MultiFileBase::operator= ( const MultiFileBase & )

delete

Forbid assignment.

◆ read()

Int64 casacore::MultiFileBase::read	(	Int	fileId,
		void *	buffer,
		Int64	size,
		Int64	offset
	)

Read a block at the given offset in the logical file.

It returns the actual size read.

◆ readBlock()

virtual void casacore::MultiFileBase::readBlock	(	MultiFileInfo &	info,
		Int64	blknr,
		void *	buffer
	)

privatepure virtual

Read a data block of a logical file from the container file.

Implemented in casacore::MultiHDF5.

◆ readHeader()

virtual void casacore::MultiFileBase::readHeader ( Bool always = True )

privatepure virtual

Read the header info.

If always==False, the info is only read if the header counter has changed.

Implemented in casacore::MultiHDF5.

◆ reopenRW()

virtual void casacore::MultiFileBase::reopenRW ( )

pure virtual

Reopen the underlying file for read/write access.

Nothing will be done if the file is writable already. Otherwise it will be reopened and an exception will be thrown if it is not possible to reopen it for read/write access.

Implemented in casacore::MultiFile, and casacore::MultiHDF5.

◆ resync()

void casacore::MultiFileBase::resync ( )

protected

Resync with another process by clearing the buffers and rereading the header.

The header is only read if its counter has changed.

◆ setNewFile()

void casacore::MultiFileBase::setNewFile ( )

protected

Set the flags and blockSize for a new MultiFile/HDF5.

◆ truncate()

void casacore::MultiFileBase::truncate	(	Int	fileId,
		Int64	size
	)

Truncate the logical file to the given size.

◆ useODirect()

Bool casacore::MultiFileBase::useODirect ( ) const

inline

Is O_DIRECT used?

Definition at line 236 of file MultiFileBase.h.

References itsUseODirect.

◆ write()

Int64 casacore::MultiFileBase::write	(	Int	fileId,
		const void *	buffer,
		Int64	size,
		Int64	offset
	)

Write a block at the given offset in the logical file.

It returns the actual size written.

◆ writeBlock()

virtual void casacore::MultiFileBase::writeBlock	(	MultiFileInfo &	info,
		Int64	blknr,
		const void *	buffer
	)

privatepure virtual

Write a data block of a logical file into the container file.

Implemented in casacore::MultiHDF5.

Referenced by writeDirty().

◆ writeDirty()

void casacore::MultiFileBase::writeDirty ( MultiFileInfo & info )

inlineprivate

Write the dirty block and clear dirty flag.

Definition at line 249 of file MultiFileBase.h.

References casacore::False, info(), and writeBlock().

◆ writeHeader()

virtual void casacore::MultiFileBase::writeHeader ( )

privatepure virtual

Write the header info.

Implemented in casacore::MultiHDF5.

Member Data Documentation

◆ itsBlockSize

Int64 casacore::MultiFileBase::itsBlockSize

protected

Definition at line 294 of file MultiFileBase.h.

Referenced by blockSize().

◆ itsBuffer

std::shared_ptr<MultiFileBuffer> casacore::MultiFileBase::itsBuffer

protected

Definition at line 298 of file MultiFileBase.h.

◆ itsChanged

Bool casacore::MultiFileBase::itsChanged

protected

Definition at line 301 of file MultiFileBase.h.

◆ itsFreeBlocks

std::vector<Int64> casacore::MultiFileBase::itsFreeBlocks

protected

Definition at line 302 of file MultiFileBase.h.

Referenced by freeBlocks().

◆ itsHdrCounter

Int64 casacore::MultiFileBase::itsHdrCounter

protected

Definition at line 296 of file MultiFileBase.h.

◆ itsInfo

std::vector<MultiFileInfo> casacore::MultiFileBase::itsInfo

protected

Definition at line 297 of file MultiFileBase.h.

Referenced by info().

◆ itsName

String casacore::MultiFileBase::itsName

protected

Definition at line 293 of file MultiFileBase.h.

Referenced by fileName().

◆ itsNrBlock

Int64 casacore::MultiFileBase::itsNrBlock

protected

Definition at line 295 of file MultiFileBase.h.

Referenced by nblock().

◆ itsUseODirect

Bool casacore::MultiFileBase::itsUseODirect

protected

Definition at line 299 of file MultiFileBase.h.

Referenced by useODirect().

◆ itsWritable

Bool casacore::MultiFileBase::itsWritable

protected

Definition at line 300 of file MultiFileBase.h.

Referenced by isWritable().

The documentation for this class was generated from the following file:

casa/IO/MultiFileBase.h

Public Member Functions

Static Public Member Functions

Protected Member Functions

Protected Attributes

Private Member Functions

Detailed Description

Intended use:

Review Status

Synopsis

Constructor & Destructor Documentation

◆ MultiFileBase() [1/2]

◆ ~MultiFileBase()

◆ MultiFileBase() [2/2]

Member Function Documentation

◆ addFile()

◆ blockSize()

◆ close()

◆ closeFile()

◆ createFile()

◆ deleteFile()

◆ doAddFile()

◆ doCloseFile()

◆ doDeleteFile()

◆ doFlushFile()

◆ doOpenFile()

◆ doTruncateFile()

◆ extend()

◆ fileId()

◆ fileName()

◆ fileSize()

◆ flush()

◆ flushFile()

◆ freeBlocks()

◆ fsync()

◆ info()

◆ isWritable()

◆ makeNested()

◆ nblock()

◆ nfile()

◆ openFile()

◆ openMF()

◆ operator=()

◆ read()

◆ readBlock()

◆ readHeader()

◆ reopenRW()

◆ resync()

◆ setNewFile()

◆ truncate()

◆ useODirect()

◆ write()

◆ writeBlock()

◆ writeDirty()

◆ writeHeader()

Member Data Documentation

◆ itsBlockSize

◆ itsBuffer

◆ itsChanged

◆ itsFreeBlocks

◆ itsHdrCounter

◆ itsInfo

◆ itsName

◆ itsNrBlock

◆ itsUseODirect

◆ itsWritable