casacore
Classes | Public Types | Public Member Functions | Static Public Member Functions | Private Types | Private Member Functions | Static Private Member Functions | Private Attributes | List of all members
dyscostman::StochasticEncoder< ValueType > Class Template Reference

Lossy encoder for stochastic values. More...

#include <stochasticencoder.h>

Classes

class  Dictionary
 

Public Types

typedef unsigned symbol_t
 Unsigned integer type used for representing the encoded symbols. More...
 
typedef ValueType value_t
 Template type used for representing floating point values that are to be encoded. More...
 

Public Member Functions

 StochasticEncoder (size_t quantCount, ValueType stddev, bool gaussianMapping=true)
 Construct encoder for given dictionary size and Gaussian stddev. More...
 
symbol_t Encode (ValueType value) const
 Get the quantized symbol for the given floating point value. More...
 
symbol_t EncodeWithDithering (ValueType value, unsigned ditherValue) const
 Get the quantized symbol for the given floating point value. More...
 
value_t RightBoundary (symbol_t symbol) const
 Will return the right boundary of the given symbol. More...
 
ValueType Decode (symbol_t symbol) const
 Get the centroid value that belongs to the given symbol. More...
 
size_t QuantizationCount () const
 
ValueType MaxQuantity () const
 
ValueType MinQuantity () const
 

Static Public Member Functions

static StochasticEncoder StudentTEncoder (size_t quantCount, double nu, double rms)
 
static StochasticEncoder TruncatedGausEncoder (size_t quantCount, double trunc, double rms)
 
static std::uniform_int_distribution< unsigned > GetDitherDistribution ()
 

Private Types

typedef long double num_t
 

Private Member Functions

 StochasticEncoder (size_t quantCount)
 
void initializeStudentT (double nu, double rms)
 
void initializeTruncatedGaussian (double truncationValue, double rms)
 

Static Private Member Functions

static num_t cumulative (num_t x)
 
static num_t invCumulative (num_t c, num_t err=num_t(1e-13))
 

Private Attributes

Dictionary _encDictionary
 
Dictionary _decDictionary
 

Detailed Description

template<typename ValueType = float>
class dyscostman::StochasticEncoder< ValueType >

Lossy encoder for stochastic values.

This encoder can encode a numeric value represented by a floating point number (float, double, ...) into an integer value with a given limit. It can be made to be the least-square error quantization for some distributions.

Encoding and decoding have asymetric time complexity / speeds, as decoding is easier than encoding. Decoding is a single indexing into an array, thus extremely fast and with constant time complexity. Encoding is a binary search through the quantizaton values, thus takes O(log quantcount). Typical performance of encoding is 100 MB/s.

If the values are encoded into a number of bits which are not divisible by eight, the BytePacker class can be used to pack the values.

Author
André Offringa (offri.nosp@m.nga@.nosp@m.gmail.nosp@m..com)

Definition at line 32 of file stochasticencoder.h.

Member Typedef Documentation

◆ num_t

template<typename ValueType = float>
typedef long double dyscostman::StochasticEncoder< ValueType >::num_t
private

Definition at line 257 of file stochasticencoder.h.

◆ symbol_t

template<typename ValueType = float>
typedef unsigned dyscostman::StochasticEncoder< ValueType >::symbol_t

Unsigned integer type used for representing the encoded symbols.

Definition at line 65 of file stochasticencoder.h.

◆ value_t

template<typename ValueType = float>
typedef ValueType dyscostman::StochasticEncoder< ValueType >::value_t

Template type used for representing floating point values that are to be encoded.

Definition at line 71 of file stochasticencoder.h.

Constructor & Destructor Documentation

◆ StochasticEncoder() [1/2]

template<typename ValueType = float>
dyscostman::StochasticEncoder< ValueType >::StochasticEncoder ( size_t  quantCount,
ValueType  stddev,
bool  gaussianMapping = true 
)

Construct encoder for given dictionary size and Gaussian stddev.

This constructor initializes the lookup table, and is therefore quite slow.

Parameters
quantCountThe number of quantization levels, i.e., the dictionary size. Normally this is 2^bitcount. One quantity will be saved for storing non-finite values (NaN and infinites).
stddevThe standard deviation of the data. The closer this value is to the real stddev, the more accurate the encoder will be.
gaussianMappingUsed for testing with non-gaussian distributions.

◆ StochasticEncoder() [2/2]

template<typename ValueType = float>
dyscostman::StochasticEncoder< ValueType >::StochasticEncoder ( size_t  quantCount)
inlineexplicitprivate

Definition at line 154 of file stochasticencoder.h.

Member Function Documentation

◆ cumulative()

template<typename ValueType = float>
static num_t dyscostman::StochasticEncoder< ValueType >::cumulative ( num_t  x)
staticprivate

◆ Decode()

template<typename ValueType = float>
ValueType dyscostman::StochasticEncoder< ValueType >::Decode ( symbol_t  symbol) const
inline

Get the centroid value that belongs to the given symbol.

Parameters
symbolSymbol to be decoded
Returns
The best estimate of the original value.

Definition at line 143 of file stochasticencoder.h.

References dyscostman::StochasticEncoder< ValueType >::_decDictionary, and dyscostman::StochasticEncoder< ValueType >::Dictionary::value().

◆ Encode()

template<typename ValueType = float>
symbol_t dyscostman::StochasticEncoder< ValueType >::Encode ( ValueType  value) const
inline

Get the quantized symbol for the given floating point value.

This method is implemented with a binary search, so takes O(log N), with N the dictionary size (2^bitcount). Use Decode() on the returned symbol to get the decoded value.

Parameters
valueFloating point value to be encoded.

Definition at line 81 of file stochasticencoder.h.

References dyscostman::StochasticEncoder< ValueType >::_encDictionary, casacore::arrays_internal::isfinite(), dyscostman::StochasticEncoder< ValueType >::Dictionary::lower_bound(), dyscostman::StochasticEncoder< ValueType >::QuantizationCount(), dyscostman::StochasticEncoder< ValueType >::Dictionary::symbol(), and casacore::value().

◆ EncodeWithDithering()

template<typename ValueType = float>
symbol_t dyscostman::StochasticEncoder< ValueType >::EncodeWithDithering ( ValueType  value,
unsigned  ditherValue 
) const
inline

Get the quantized symbol for the given floating point value.

Dithering is applied, which will cause the average error to converge to zero, assuming the error is uniformly distributed. This method is implemented with a binary search, so takes O(log N), with N the dictionary size (2^bitcount). Use Decode() on the returned symbol to get the decoded value.

Parameters
valueFloating point value to be encoded.
ditherValueThe dithering value to apply.

Definition at line 103 of file stochasticencoder.h.

References dyscostman::StochasticEncoder< ValueType >::_decDictionary, dyscostman::StochasticEncoder< ValueType >::_encDictionary, dyscostman::StochasticEncoder< ValueType >::Dictionary::begin(), dyscostman::StochasticEncoder< ValueType >::Dictionary::end(), casacore::arrays_internal::isfinite(), dyscostman::StochasticEncoder< ValueType >::Dictionary::lower_bound(), dyscostman::StochasticEncoder< ValueType >::Dictionary::size(), dyscostman::StochasticEncoder< ValueType >::Dictionary::symbol(), casacore::value(), and dyscostman::StochasticEncoder< ValueType >::Dictionary::value().

◆ GetDitherDistribution()

template<typename ValueType = float>
static std::uniform_int_distribution<unsigned> dyscostman::StochasticEncoder< ValueType >::GetDitherDistribution ( )
inlinestatic

Definition at line 88 of file stochasticencoder.h.

◆ initializeStudentT()

template<typename ValueType = float>
void dyscostman::StochasticEncoder< ValueType >::initializeStudentT ( double  nu,
double  rms 
)
private

◆ initializeTruncatedGaussian()

template<typename ValueType = float>
void dyscostman::StochasticEncoder< ValueType >::initializeTruncatedGaussian ( double  truncationValue,
double  rms 
)
private

◆ invCumulative()

template<typename ValueType = float>
static num_t dyscostman::StochasticEncoder< ValueType >::invCumulative ( num_t  c,
num_t  err = num_t(1e-13) 
)
staticprivate

◆ MaxQuantity()

template<typename ValueType = float>
ValueType dyscostman::StochasticEncoder< ValueType >::MaxQuantity ( ) const
inline

◆ MinQuantity()

template<typename ValueType = float>
ValueType dyscostman::StochasticEncoder< ValueType >::MinQuantity ( ) const
inline

◆ QuantizationCount()

template<typename ValueType = float>
size_t dyscostman::StochasticEncoder< ValueType >::QuantizationCount ( ) const
inline

◆ RightBoundary()

template<typename ValueType = float>
value_t dyscostman::StochasticEncoder< ValueType >::RightBoundary ( symbol_t  symbol) const
inline

Will return the right boundary of the given symbol.

The right boundary is the smallest value that would not be quantized to the given symbol anymore. If no such boundary exists, 0.0 is returned.

Definition at line 131 of file stochasticencoder.h.

References dyscostman::StochasticEncoder< ValueType >::_encDictionary, dyscostman::StochasticEncoder< ValueType >::Dictionary::size(), and dyscostman::StochasticEncoder< ValueType >::Dictionary::value().

◆ StudentTEncoder()

template<typename ValueType = float>
static StochasticEncoder dyscostman::StochasticEncoder< ValueType >::StudentTEncoder ( size_t  quantCount,
double  nu,
double  rms 
)
inlinestatic

◆ TruncatedGausEncoder()

template<typename ValueType = float>
static StochasticEncoder dyscostman::StochasticEncoder< ValueType >::TruncatedGausEncoder ( size_t  quantCount,
double  trunc,
double  rms 
)
inlinestatic

Member Data Documentation

◆ _decDictionary

template<typename ValueType = float>
Dictionary dyscostman::StochasticEncoder< ValueType >::_decDictionary
private

◆ _encDictionary

template<typename ValueType = float>
Dictionary dyscostman::StochasticEncoder< ValueType >::_encDictionary
private

The documentation for this class was generated from the following file: