Lossy encoder for stochastic values. More...

#include <stochasticencoder.h>

Classes
class	Dictionary

Public Types
typedef unsigned	symbol_t
	Unsigned integer type used for representing the encoded symbols. More...

typedef ValueType	value_t
	Template type used for representing floating point values that are to be encoded. More...

Public Member Functions
	StochasticEncoder (size_t quantCount, ValueType stddev, bool gaussianMapping=true)
	Construct encoder for given dictionary size and Gaussian stddev. More...

symbol_t	Encode (ValueType value) const
	Get the quantized symbol for the given floating point value. More...

symbol_t	EncodeWithDithering (ValueType value, unsigned ditherValue) const
	Get the quantized symbol for the given floating point value. More...

value_t	RightBoundary (symbol_t symbol) const
	Will return the right boundary of the given symbol. More...

ValueType	Decode (symbol_t symbol) const
	Get the centroid value that belongs to the given symbol. More...

size_t	QuantizationCount () const

ValueType	MaxQuantity () const

ValueType	MinQuantity () const

Static Public Member Functions
static StochasticEncoder	StudentTEncoder (size_t quantCount, double nu, double rms)

static StochasticEncoder	TruncatedGausEncoder (size_t quantCount, double trunc, double rms)

static std::uniform_int_distribution< unsigned >	GetDitherDistribution ()

Private Types
typedef long double	num_t

Private Member Functions
	StochasticEncoder (size_t quantCount)

void	initializeStudentT (double nu, double rms)

void	initializeTruncatedGaussian (double truncationValue, double rms)

Static Private Member Functions
static num_t	cumulative (num_t x)

static num_t	invCumulative (num_t c, num_t err=num_t(1e-13))

Private Attributes
Dictionary	_encDictionary

Dictionary	_decDictionary

Detailed Description

template<typename ValueType = float>
class dyscostman::StochasticEncoder< ValueType >

Lossy encoder for stochastic values.

This encoder can encode a numeric value represented by a floating point number (float, double, ...) into an integer value with a given limit. It can be made to be the least-square error quantization for some distributions.

Encoding and decoding have asymetric time complexity / speeds, as decoding is easier than encoding. Decoding is a single indexing into an array, thus extremely fast and with constant time complexity. Encoding is a binary search through the quantizaton values, thus takes O(log quantcount). Typical performance of encoding is 100 MB/s.

If the values are encoded into a number of bits which are not divisible by eight, the BytePacker class can be used to pack the values.

Author: André Offringa (offri.nosp@m.nga@.nosp@m.gmail.nosp@m..com)

Definition at line 32 of file stochasticencoder.h.

Member Typedef Documentation

◆ num_t

template<typename ValueType = float>

typedef long double dyscostman::StochasticEncoder< ValueType >::num_t

private

Definition at line 257 of file stochasticencoder.h.

◆ symbol_t

template<typename ValueType = float>

typedef unsigned dyscostman::StochasticEncoder< ValueType >::symbol_t

Unsigned integer type used for representing the encoded symbols.

Definition at line 65 of file stochasticencoder.h.

◆ value_t

template<typename ValueType = float>

typedef ValueType dyscostman::StochasticEncoder< ValueType >::value_t

Template type used for representing floating point values that are to be encoded.

Definition at line 71 of file stochasticencoder.h.

Constructor & Destructor Documentation

◆ StochasticEncoder() [1/2]

template<typename ValueType = float>

dyscostman::StochasticEncoder< ValueType >::StochasticEncoder	(	size_t	quantCount,
		ValueType	stddev,
		bool	gaussianMapping = `true`
	)

Construct encoder for given dictionary size and Gaussian stddev.

This constructor initializes the lookup table, and is therefore quite slow.

Parameters

quantCount	The number of quantization levels, i.e., the dictionary size. Normally this is 2^bitcount. One quantity will be saved for storing non-finite values (NaN and infinites).
stddev	The standard deviation of the data. The closer this value is to the real stddev, the more accurate the encoder will be.
gaussianMapping	Used for testing with non-gaussian distributions.

◆ StochasticEncoder() [2/2]

template<typename ValueType = float>

dyscostman::StochasticEncoder< ValueType >::StochasticEncoder ( size_t quantCount )

inlineexplicitprivate

Definition at line 154 of file stochasticencoder.h.

Member Function Documentation

◆ cumulative()

template<typename ValueType = float>

static num_t dyscostman::StochasticEncoder< ValueType >::cumulative ( num_t x )

staticprivate

◆ Decode()

template<typename ValueType = float>

ValueType dyscostman::StochasticEncoder< ValueType >::Decode ( symbol_t symbol ) const

inline

Get the centroid value that belongs to the given symbol.

Parameters

symbol Symbol to be decoded

Returns: The best estimate of the original value.

Definition at line 143 of file stochasticencoder.h.

References dyscostman::StochasticEncoder< ValueType >::_decDictionary, and dyscostman::StochasticEncoder< ValueType >::Dictionary::value().

◆ Encode()

template<typename ValueType = float>

symbol_t dyscostman::StochasticEncoder< ValueType >::Encode ( ValueType value ) const

inline

Get the quantized symbol for the given floating point value.

This method is implemented with a binary search, so takes O(log N), with N the dictionary size (2^bitcount). Use Decode() on the returned symbol to get the decoded value.

Parameters

value Floating point value to be encoded.

Definition at line 81 of file stochasticencoder.h.

References dyscostman::StochasticEncoder< ValueType >::_encDictionary, casacore::arrays_internal::isfinite(), dyscostman::StochasticEncoder< ValueType >::Dictionary::lower_bound(), dyscostman::StochasticEncoder< ValueType >::QuantizationCount(), dyscostman::StochasticEncoder< ValueType >::Dictionary::symbol(), and casacore::value().

◆ EncodeWithDithering()

template<typename ValueType = float>

symbol_t dyscostman::StochasticEncoder< ValueType >::EncodeWithDithering	(	ValueType	value,
		unsigned	ditherValue
	)		const

inline

Get the quantized symbol for the given floating point value.

Dithering is applied, which will cause the average error to converge to zero, assuming the error is uniformly distributed. This method is implemented with a binary search, so takes O(log N), with N the dictionary size (2^bitcount). Use Decode() on the returned symbol to get the decoded value.

Parameters

value	Floating point value to be encoded.
ditherValue	The dithering value to apply.

Definition at line 103 of file stochasticencoder.h.

◆ GetDitherDistribution()

template<typename ValueType = float>

static std::uniform_int_distribution<unsigned> dyscostman::StochasticEncoder< ValueType >::GetDitherDistribution ( )

inlinestatic

Definition at line 88 of file stochasticencoder.h.

◆ initializeStudentT()

template<typename ValueType = float>

void dyscostman::StochasticEncoder< ValueType >::initializeStudentT	(	double	nu,
		double	rms
	)

private

Referenced by dyscostman::StochasticEncoder< ValueType >::StudentTEncoder().

◆ initializeTruncatedGaussian()

template<typename ValueType = float>

void dyscostman::StochasticEncoder< ValueType >::initializeTruncatedGaussian	(	double	truncationValue,
		double	rms
	)

private

Referenced by dyscostman::StochasticEncoder< ValueType >::TruncatedGausEncoder().

◆ invCumulative()

template<typename ValueType = float>

static num_t dyscostman::StochasticEncoder< ValueType >::invCumulative	(	num_t	c,
		num_t	err = `num_t(1e-13)`
	)

staticprivate

◆ MaxQuantity()

template<typename ValueType = float>

ValueType dyscostman::StochasticEncoder< ValueType >::MaxQuantity ( ) const

inline

Definition at line 149 of file stochasticencoder.h.

References dyscostman::StochasticEncoder< ValueType >::_decDictionary, and dyscostman::StochasticEncoder< ValueType >::Dictionary::largest_value().

◆ MinQuantity()

template<typename ValueType = float>

ValueType dyscostman::StochasticEncoder< ValueType >::MinQuantity ( ) const

inline

Definition at line 151 of file stochasticencoder.h.

References dyscostman::StochasticEncoder< ValueType >::_decDictionary, and dyscostman::StochasticEncoder< ValueType >::Dictionary::smallest_value().

◆ QuantizationCount()

template<typename ValueType = float>

size_t dyscostman::StochasticEncoder< ValueType >::QuantizationCount ( ) const

inline

Definition at line 147 of file stochasticencoder.h.

References dyscostman::StochasticEncoder< ValueType >::_decDictionary, and dyscostman::StochasticEncoder< ValueType >::Dictionary::size().

Referenced by dyscostman::StochasticEncoder< ValueType >::Encode().

◆ RightBoundary()

template<typename ValueType = float>

value_t dyscostman::StochasticEncoder< ValueType >::RightBoundary ( symbol_t symbol ) const

inline

Will return the right boundary of the given symbol.

The right boundary is the smallest value that would not be quantized to the given symbol anymore. If no such boundary exists, 0.0 is returned.

Definition at line 131 of file stochasticencoder.h.

References dyscostman::StochasticEncoder< ValueType >::_encDictionary, dyscostman::StochasticEncoder< ValueType >::Dictionary::size(), and dyscostman::StochasticEncoder< ValueType >::Dictionary::value().

◆ StudentTEncoder()

template<typename ValueType = float>

static StochasticEncoder dyscostman::StochasticEncoder< ValueType >::StudentTEncoder	(	size_t	quantCount,
		double	nu,
		double	rms
	)

inlinestatic

Definition at line 48 of file stochasticencoder.h.

References dyscostman::StochasticEncoder< ValueType >::initializeStudentT(), and casacore::rms().

◆ TruncatedGausEncoder()

template<typename ValueType = float>

static StochasticEncoder dyscostman::StochasticEncoder< ValueType >::TruncatedGausEncoder	(	size_t	quantCount,
		double	trunc,
		double	rms
	)

inlinestatic

Definition at line 55 of file stochasticencoder.h.

References dyscostman::StochasticEncoder< ValueType >::initializeTruncatedGaussian(), and casacore::rms().

Member Data Documentation

◆ _decDictionary

template<typename ValueType = float>

Dictionary dyscostman::StochasticEncoder< ValueType >::_decDictionary

private

Definition at line 263 of file stochasticencoder.h.

Referenced by dyscostman::StochasticEncoder< ValueType >::Decode(), dyscostman::StochasticEncoder< ValueType >::EncodeWithDithering(), dyscostman::StochasticEncoder< ValueType >::MaxQuantity(), dyscostman::StochasticEncoder< ValueType >::MinQuantity(), and dyscostman::StochasticEncoder< ValueType >::QuantizationCount().

◆ _encDictionary

template<typename ValueType = float>

Dictionary dyscostman::StochasticEncoder< ValueType >::_encDictionary

private

Definition at line 262 of file stochasticencoder.h.

Referenced by dyscostman::StochasticEncoder< ValueType >::Encode(), dyscostman::StochasticEncoder< ValueType >::EncodeWithDithering(), and dyscostman::StochasticEncoder< ValueType >::RightBoundary().

The documentation for this class was generated from the following file:

tables/Dysco/stochasticencoder.h

Classes

Public Types

Public Member Functions

Static Public Member Functions

Private Types

Private Member Functions

Static Private Member Functions

Private Attributes

Detailed Description

template<typename ValueType = float> class dyscostman::StochasticEncoder< ValueType >

Member Typedef Documentation

◆ num_t

◆ symbol_t

◆ value_t

Constructor & Destructor Documentation

◆ StochasticEncoder() [1/2]

◆ StochasticEncoder() [2/2]

Member Function Documentation

◆ cumulative()

◆ Decode()

◆ Encode()

◆ EncodeWithDithering()

◆ GetDitherDistribution()

◆ initializeStudentT()

◆ initializeTruncatedGaussian()

◆ invCumulative()

◆ MaxQuantity()

◆ MinQuantity()

◆ QuantizationCount()

◆ RightBoundary()

◆ StudentTEncoder()

◆ TruncatedGausEncoder()

Member Data Documentation

◆ _decDictionary

◆ _encDictionary

template<typename ValueType = float>
class dyscostman::StochasticEncoder< ValueType >