casacore
Public Member Functions | Static Public Attributes | Protected Member Functions | Private Member Functions | Private Attributes | List of all members
casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator > Class Template Reference

Class to calculate statistics using the so-called fit to half algorithm. More...

#include <FitToHalfStatistics.h>

Public Member Functions

 FitToHalfStatistics (FitToHalfStatisticsData::CENTER center=FitToHalfStatisticsData::CMEAN, FitToHalfStatisticsData::USE_DATA useData=FitToHalfStatisticsData::LE_CENTER, AccumType value=0)
 value is only used if center=CVALUE More...
 
 FitToHalfStatistics (const FitToHalfStatistics< CASA_STATP > &other)
 copy semantics More...
 
virtual ~FitToHalfStatistics ()
 
FitToHalfStatistics< CASA_STATP > & operator= (const FitToHalfStatistics< CASA_STATP > &other)
 copy semantics More...
 
virtual StatisticsAlgorithm< CASA_STATP > * clone () const
 Clone this instance. More...
 
virtual StatisticsData::ALGORITHM algorithm () const
 get the algorithm that this object uses for computing stats More...
 
AccumType getMedian (CountedPtr< uInt64 > knownNpts=nullptr, CountedPtr< AccumType > knownMin=nullptr, CountedPtr< AccumType > knownMax=nullptr, uInt binningThreshholdSizeBytes=4096 *4096, Bool persistSortedArray=False, uInt nBins=10000)
 The median is just the center value, so none of the parameters to this method are used. More...
 
AccumType getMedianAndQuantiles (std::map< Double, AccumType > &quantiles, const std::set< Double > &fractions, CountedPtr< uInt64 > knownNpts=nullptr, CountedPtr< AccumType > knownMin=nullptr, CountedPtr< AccumType > knownMax=nullptr, uInt binningThreshholdSizeBytes=4096 *4096, Bool persistSortedArray=False, uInt nBins=10000)
 In the following group of methods, if the size of the composite dataset is smaller than binningThreshholdSizeBytes, the composite dataset will be (perhaps partially) sorted and persisted in memory during the call. More...
 
AccumType getMedianAbsDevMed (CountedPtr< uInt64 > knownNpts=nullptr, CountedPtr< AccumType > knownMin=nullptr, CountedPtr< AccumType > knownMax=nullptr, uInt binningThreshholdSizeBytes=4096 *4096, Bool persistSortedArray=False, uInt nBins=10000)
 get the median of the absolute deviation about the median of the data. More...
 
std::map< Double, AccumType > getQuantiles (const std::set< Double > &fractions, CountedPtr< uInt64 > knownNpts=nullptr, CountedPtr< AccumType > knownMin=nullptr, CountedPtr< AccumType > knownMax=nullptr, uInt binningThreshholdSizeBytes=4096 *4096, Bool persistSortedArray=False, uInt nBins=10000)
 Get the specified quantiles. More...
 
virtual void getMinMax (AccumType &mymin, AccumType &mymax)
 scan the dataset(s) that have been added, and find the min and max. More...
 
uInt64 getNPts ()
 scan the dataset(s) that have been added, and find the number of good points. More...
 
virtual void reset ()
 reset object to initial state. More...
 
void setCalculateAsAdded (Bool c)
 This class does not allow statistics to be calculated as datasets are added, so an exception will be thrown if c is True. More...
 
virtual void setStatsToCalculate (std::set< StatisticsData::STATS > &stats)
 Override base class method by requiring mean to be computed in addition to what is added in stats if the requested center value is CMEAN. More...
 
- Public Member Functions inherited from casacore::ConstrainedRangeStatistics< CASA_STATP >
 ConstrainedRangeStatistics ()=delete
 
virtual ~ConstrainedRangeStatistics ()
 
virtual CASA_STATP getMedian (CountedPtr< uInt64 > knownNpts=nullptr, CountedPtr< CASA_STATP > knownMin=nullptr, CountedPtr< CASA_STATP > knownMax=nullptr, uInt binningThreshholdSizeBytes=4096 *4096, Bool persistSortedArray=False, uInt nBins=10000)
 In the following group of methods, if the size of the composite dataset is smaller than binningThreshholdSizeBytes, the composite dataset will be (perhaps partially) sorted and persisted in memory during the call. More...
 
virtual CASA_STATP getMedianAbsDevMed (CountedPtr< uInt64 > knownNpts=nullptr, CountedPtr< CASA_STATP > knownMin=nullptr, CountedPtr< CASA_STATP > knownMax=nullptr, uInt binningThreshholdSizeBytes=4096 *4096, Bool persistSortedArray=False, uInt nBins=10000)
 get the median of the absolute deviation about the median of the data. More...
 
virtual CASA_STATP getMedianAndQuantiles (std::map< Double, CASA_STATP > &quantileToValue, const std::set< Double > &quantiles, CountedPtr< uInt64 > knownNpts=nullptr, CountedPtr< CASA_STATP > knownMin=nullptr, CountedPtr< CASA_STATP > knownMax=nullptr, uInt binningThreshholdSizeBytes=4096 *4096, Bool persistSortedArray=False, uInt nBins=10000)
 If one needs to compute both the median and quantile values, it is better to call getMedianAndQuantiles() rather than getMedian() and getQuantiles() separately, as the first will scan large data sets fewer times than calling the seperate methods. More...
 
virtual std::map< Double, CASA_STATPgetQuantiles (const std::set< Double > &quantiles, CountedPtr< uInt64 > knownNpts=nullptr, CountedPtr< CASA_STATP > knownMin=nullptr, CountedPtr< CASA_STATP > knownMax=NULL, uInt binningThreshholdSizeBytes=4096 *4096, Bool persistSortedArray=False, uInt nBins=10000)
 Get the specified quantiles. More...
 
virtual void getMinMax (CASA_STATP &mymin, CASA_STATP &mymax)
 get the min and max of the data set More...
 
virtual LocationType getStatisticIndex (StatisticsData::STATS stat)
 see base class description More...
 
- Public Member Functions inherited from casacore::ClassicalStatistics< CASA_STATP >
 ClassicalStatistics ()
 
 ClassicalStatistics (const ClassicalStatistics &cs)
 copy semantics More...
 
virtual ~ClassicalStatistics ()
 
ClassicalStatisticsoperator= (const ClassicalStatistics &other)
 copy semantics More...
 
virtual void getMinMaxNpts (uInt64 &npts, CASA_STATP &mymin, CASA_STATP &mymax)
 
virtual void setDataProvider (StatsDataProvider< CASA_STATP > *dataProvider)
 An exception will be thrown if setCalculateAsAdded(True) has been called. More...
 
void setQuantileComputer (CountedPtr< ClassicalQuantileComputer< CASA_STATP >> qc)
 Allow derived objects to set the quantile computer object. More...
 
- Public Member Functions inherited from casacore::StatisticsAlgorithm< CASA_STATP >
virtual ~StatisticsAlgorithm ()
 
void addData (const DataIterator &first, uInt nr, uInt dataStride=1, Bool nrAccountsForStride=False)
 Add a dataset to an existing set of datasets on which statistics are to be calculated. More...
 
void addData (const DataIterator &first, uInt nr, const DataRanges &dataRanges, Bool isInclude=True, uInt dataStride=1, Bool nrAccountsForStride=False)
 
void addData (const DataIterator &first, const const Bool * &maskFirst, uInt nr, uInt dataStride=1, Bool nrAccountsForStride=False, uInt maskStride=1)
 
void addData (const DataIterator &first, const const Bool * &maskFirst, uInt nr, const DataRanges &dataRanges, Bool isInclude=True, uInt dataStride=1, Bool nrAccountsForStride=False, uInt maskStride=1)
 
void addData (const DataIterator &first, const DataIterator &weightFirst, uInt nr, uInt dataStride=1, Bool nrAccountsForStride=False)
 
void addData (const DataIterator &first, const DataIterator &weightFirst, uInt nr, const DataRanges &dataRanges, Bool isInclude=True, uInt dataStride=1, Bool nrAccountsForStride=False)
 
void addData (const DataIterator &first, const DataIterator &weightFirst, const const Bool * &maskFirst, uInt nr, uInt dataStride=1, Bool nrAccountsForStride=False, uInt maskStride=1)
 
void addData (const DataIterator &first, const DataIterator &weightFirst, const const Bool * &maskFirst, uInt nr, const DataRanges &dataRanges, Bool isInclude=True, uInt dataStride=1, Bool nrAccountsForStride=False, uInt maskStride=1)
 
CASA_STATP getQuantile (Double quantile, CountedPtr< uInt64 > knownNpts=nullptr, CountedPtr< CASA_STATP > knownMin=nullptr, CountedPtr< CASA_STATP > knownMax=nullptr, uInt binningThreshholdSizeBytes=4096 *4096, Bool persistSortedArray=False, uInt nBins=10000)
 Purposefully not virtual. More...
 
CASA_STATP getStatistic (StatisticsData::STATS stat)
 get the value of the specified statistic. More...
 
StatsData< CASA_STATPgetStatistics ()
 Return statistics. More...
 
void setData (const DataIterator &first, uInt nr, uInt dataStride=1, Bool nrAccountsForStride=False)
 setdata() clears any current datasets or data provider and then adds the specified data set as the first dataset in the (possibly new) set of data sets for which statistics are to be calculated. More...
 
void setData (const DataIterator &first, uInt nr, const DataRanges &dataRanges, Bool isInclude=True, uInt dataStride=1, Bool nrAccountsForStride=False)
 
void setData (const DataIterator &first, const const Bool * &maskFirst, uInt nr, uInt dataStride=1, Bool nrAccountsForStride=False, uInt maskStride=1)
 
void setData (const DataIterator &first, const const Bool * &maskFirst, uInt nr, const DataRanges &dataRanges, Bool isInclude=True, uInt dataStride=1, Bool nrAccountsForStride=False, uInt maskStride=1)
 
void setData (const DataIterator &first, const DataIterator &weightFirst, uInt nr, uInt dataStride=1, Bool nrAccountsForStride=False)
 
void setData (const DataIterator &first, const DataIterator &weightFirst, uInt nr, const DataRanges &dataRanges, Bool isInclude=True, uInt dataStride=1, Bool nrAccountsForStride=False)
 
void setData (const DataIterator &first, const DataIterator &weightFirst, const const Bool * &maskFirst, uInt nr, uInt dataStride=1, Bool nrAccountsForStride=False, uInt maskStride=1)
 
void setData (const DataIterator &first, const DataIterator &weightFirst, const const Bool * &maskFirst, uInt nr, const DataRanges &dataRanges, Bool isInclude=True, uInt dataStride=1, Bool nrAccountsForStride=False, uInt maskStride=1)
 

Static Public Attributes

static const AccumType TWO
 

Protected Member Functions

virtual StatsData< AccumType > _getInitialStats () const
 
virtual StatsData< AccumType > _getStatistics ()
 
StatsData< AccumType > & _getStatsData ()
 Retrieve stats structure. More...
 
const StatsData< AccumType > & _getStatsData () const
 
void _unweightedStats (StatsData< AccumType > &stats, uInt64 &ngood, LocationType &location, const DataIterator &dataBegin, uInt64 nr, uInt dataStride)
 no weights, no mask, no ranges More...
 
void _unweightedStats (StatsData< AccumType > &stats, uInt64 &ngood, LocationType &location, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude)
 no weights, no mask More...
 
void _unweightedStats (StatsData< AccumType > &stats, uInt64 &ngood, LocationType &location, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride)
 
void _unweightedStats (StatsData< AccumType > &stats, uInt64 &ngood, LocationType &location, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude)
 
void _updateDataProviderMaxMin (const StatsData< AccumType > &threadStats)
 
void _weightedStats (StatsData< AccumType > &stats, LocationType &location, const DataIterator &dataBegin, const WeightsIterator &weightsBegin, uInt64 nr, uInt dataStride)
 has weights, but no mask, no ranges More...
 
void _weightedStats (StatsData< AccumType > &stats, LocationType &location, const DataIterator &dataBegin, const WeightsIterator &weightsBegin, uInt64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude)
 
void _weightedStats (StatsData< AccumType > &stats, LocationType &location, const DataIterator &dataBegin, const WeightsIterator &weightBegin, uInt64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride)
 
void _weightedStats (StatsData< AccumType > &stats, LocationType &location, const DataIterator &dataBegin, const WeightsIterator &weightBegin, uInt64 nr, uInt dataStride, const MaskIterator &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude)
 
- Protected Member Functions inherited from casacore::ConstrainedRangeStatistics< CASA_STATP >
 ConstrainedRangeStatistics (CountedPtr< ConstrainedRangeQuantileComputer< CASA_STATP >> qc)
 Concrete derived classes are responsible for providing an appropriate QuantileComputer object to the constructor, which is ultimately passed up the instantiation hierarchy and stored at the StatisticsAlgorithm level. More...
 
 ConstrainedRangeStatistics (const ConstrainedRangeStatistics< CASA_STATP > &other)
 copy semantics More...
 
ConstrainedRangeStatistics< CASA_STATP > & operator= (const ConstrainedRangeStatistics< CASA_STATP > &other)
 copy semantics More...
 
virtual void _accumNpts (uInt64 &npts, const DataIterator &dataStart, uInt64 nr, uInt dataStride) const
 scan through the data set to determine the number of good (unmasked, weight > 0, within range) points. More...
 
virtual void _accumNpts (uInt64 &npts, const DataIterator &dataStart, uInt64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude) const
 
virtual void _accumNpts (uInt64 &npts, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const const Bool * &maskBegin, uInt maskStride) const
 
virtual void _accumNpts (uInt64 &npts, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const const Bool * &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude) const
 
virtual void _accumNpts (uInt64 &npts, const DataIterator &dataBegin, const DataIterator &weightsBegin, uInt64 nr, uInt dataStride) const
 
virtual void _accumNpts (uInt64 &npts, const DataIterator &dataBegin, const DataIterator &weightsBegin, uInt64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude) const
 
virtual void _accumNpts (uInt64 &npts, const DataIterator &dataBegin, const DataIterator &weightsBegin, uInt64 nr, uInt dataStride, const const Bool * &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude) const
 
virtual void _accumNpts (uInt64 &npts, const DataIterator &dataBegin, const DataIterator &weightBegin, uInt64 nr, uInt dataStride, const const Bool * &maskBegin, uInt maskStride) const
 
virtual CASA_STATP _getStatistic (StatisticsData::STATS stat)
 
virtual void _minMax (CountedPtr< CASA_STATP > &mymin, CountedPtr< CASA_STATP > &mymax, const DataIterator &dataBegin, uInt64 nr, uInt dataStride) const
 
virtual void _minMax (CountedPtr< CASA_STATP > &mymin, CountedPtr< CASA_STATP > &mymax, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude) const
 
virtual void _minMax (CountedPtr< CASA_STATP > &mymin, CountedPtr< CASA_STATP > &mymax, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const const Bool * &maskBegin, uInt maskStride) const
 
virtual void _minMax (CountedPtr< CASA_STATP > &mymin, CountedPtr< CASA_STATP > &mymax, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const const Bool * &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude) const
 
virtual void _minMax (CountedPtr< CASA_STATP > &mymin, CountedPtr< CASA_STATP > &mymax, const DataIterator &dataBegin, const DataIterator &weightsBegin, uInt64 nr, uInt dataStride) const
 
virtual void _minMax (CountedPtr< CASA_STATP > &mymin, CountedPtr< CASA_STATP > &mymax, const DataIterator &dataBegin, const DataIterator &weightsBegin, uInt64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude) const
 
virtual void _minMax (CountedPtr< CASA_STATP > &mymin, CountedPtr< CASA_STATP > &mymax, const DataIterator &dataBegin, const DataIterator &weightsBegin, uInt64 nr, uInt dataStride, const const Bool * &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude) const
 
virtual void _minMax (CountedPtr< CASA_STATP > &mymin, CountedPtr< CASA_STATP > &mymax, const DataIterator &dataBegin, const DataIterator &weightBegin, uInt64 nr, uInt dataStride, const const Bool * &maskBegin, uInt maskStride) const
 
virtual void _minMaxNpts (uInt64 &npts, CountedPtr< CASA_STATP > &mymin, CountedPtr< CASA_STATP > &mymax, const DataIterator &dataBegin, uInt64 nr, uInt dataStride) const
 Sometimes we want the min, max, and npts all in one scan. More...
 
virtual void _minMaxNpts (uInt64 &npts, CountedPtr< CASA_STATP > &mymin, CountedPtr< CASA_STATP > &mymax, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude) const
 
virtual void _minMaxNpts (uInt64 &npts, CountedPtr< CASA_STATP > &mymin, CountedPtr< CASA_STATP > &mymax, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const const Bool * &maskBegin, uInt maskStride) const
 
virtual void _minMaxNpts (uInt64 &npts, CountedPtr< CASA_STATP > &mymin, CountedPtr< CASA_STATP > &mymax, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const const Bool * &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude) const
 
virtual void _minMaxNpts (uInt64 &npts, CountedPtr< CASA_STATP > &mymin, CountedPtr< CASA_STATP > &mymax, const DataIterator &dataBegin, const DataIterator &weightsBegin, uInt64 nr, uInt dataStride) const
 
virtual void _minMaxNpts (uInt64 &npts, CountedPtr< CASA_STATP > &mymin, CountedPtr< CASA_STATP > &mymax, const DataIterator &dataBegin, const DataIterator &weightsBegin, uInt64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude) const
 
virtual void _minMaxNpts (uInt64 &npts, CountedPtr< CASA_STATP > &mymin, CountedPtr< CASA_STATP > &mymax, const DataIterator &dataBegin, const DataIterator &weightsBegin, uInt64 nr, uInt dataStride, const const Bool * &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude) const
 
virtual void _minMaxNpts (uInt64 &npts, CountedPtr< CASA_STATP > &mymin, CountedPtr< CASA_STATP > &mymax, const DataIterator &dataBegin, const DataIterator &weightBegin, uInt64 nr, uInt dataStride, const const Bool * &maskBegin, uInt maskStride) const
 
void _setRange (CountedPtr< std::pair< CASA_STATP, CASA_STATP > > r)
 This method is purposefully non-virtual. More...
 
virtual void _unweightedStats (StatsData< CASA_STATP > &stats, uInt64 &ngood, LocationType &location, const DataIterator &dataBegin, uInt64 nr, uInt dataStride)
 no weights, no mask, no ranges More...
 
virtual void _unweightedStats (StatsData< CASA_STATP > &stats, uInt64 &ngood, LocationType &location, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude)
 no weights, no mask More...
 
virtual void _unweightedStats (StatsData< CASA_STATP > &stats, uInt64 &ngood, LocationType &location, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const const Bool * &maskBegin, uInt maskStride)
 
virtual void _unweightedStats (StatsData< CASA_STATP > &stats, uInt64 &ngood, LocationType &location, const DataIterator &dataBegin, uInt64 nr, uInt dataStride, const const Bool * &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude)
 
virtual void _weightedStats (StatsData< CASA_STATP > &stats, LocationType &location, const DataIterator &dataBegin, const DataIterator &weightsBegin, uInt64 nr, uInt dataStride)
 has weights, but no mask, no ranges More...
 
virtual void _weightedStats (StatsData< CASA_STATP > &stats, LocationType &location, const DataIterator &dataBegin, const DataIterator &weightsBegin, uInt64 nr, uInt dataStride, const DataRanges &ranges, Bool isInclude)
 
virtual void _weightedStats (StatsData< CASA_STATP > &stats, LocationType &location, const DataIterator &dataBegin, const DataIterator &weightBegin, uInt64 nr, uInt dataStride, const const Bool * &maskBegin, uInt maskStride)
 
virtual void _weightedStats (StatsData< CASA_STATP > &stats, LocationType &location, const DataIterator &dataBegin, const DataIterator &weightBegin, uInt64 nr, uInt dataStride, const const Bool * &maskBegin, uInt maskStride, const DataRanges &ranges, Bool isInclude)
 
- Protected Member Functions inherited from casacore::ClassicalStatistics< CASA_STATP >
 ClassicalStatistics (CountedPtr< ClassicalQuantileComputer< CASA_STATP > > qc)
 This constructor should be used by derived objects in order to set the proper quantile computer object. More...
 
void _accumulate (StatsData< CASA_STATP > &stats, const CASA_STATP &datum, const LocationType &location)
 
void _accumulate (StatsData< CASA_STATP > &stats, const CASA_STATP &datum, const CASA_STATP &weight, const LocationType &location)
 
void _addData ()
 Allows derived classes to do things after data is set or added. More...
 
void _clearStats ()
 
Bool _getDoMaxMin () const
 
CountedPtr< StatisticsAlgorithmQuantileComputer< CASA_STATP > > _getQuantileComputer ()
 
virtual void _updateDataProviderMaxMin (const StatsData< CASA_STATP > &threadStats)
 
- Protected Member Functions inherited from casacore::StatisticsAlgorithm< CASA_STATP >
 StatisticsAlgorithm ()
 
 StatisticsAlgorithm (const StatisticsAlgorithm &other)
 use copy semantics, except for the data provider which uses reference semantics More...
 
StatisticsAlgorithmoperator= (const StatisticsAlgorithm &other)
 use copy semantics, except for the data provider which uses reference semantics More...
 
const StatisticsDataset< CASA_STATP > & _getDataset () const
 These methods are purposefully not virtual. More...
 
StatisticsDataset< CASA_STATP > & _getDataset ()
 
const std::set< StatisticsData::STATS_getStatsToCalculate () const
 
virtual const std::set< StatisticsData::STATS > & _getUnsupportedStatistics () const
 
void _setUnsupportedStatistics (const std::set< StatisticsData::STATS > &stats)
 Derived classes should normally call this in their constructors, if applicable. More...
 

Private Member Functions

void _getMinMax (CountedPtr< AccumType > &realMin, CountedPtr< AccumType > &realMax, CountedPtr< AccumType > knownMin, CountedPtr< AccumType > knownMax)
 get the min max of the entire (real + virtual) data set. More...
 
void _getRealMinMax (AccumType &realMin, AccumType &realMax)
 get the min/max of the real portion only of the dataset More...
 
void _setRange ()
 derived classes need to implement how to set their respective range More...
 

Private Attributes

FitToHalfStatisticsData::CENTER _centerType
 
Bool _useLower
 
AccumType _centerValue
 
StatsData< AccumType > _statsData
 
Bool _doMedAbsDevMed
 
Bool _rangeIsSet
 
CountedPtr< AccumType > _realMax
 these are the max and min for the real portion of the dataset More...
 
CountedPtr< AccumType > _realMin
 
Bool _isNullSet
 
CountedPtr< std::pair< AccumType, AccumType > > _range
 This is used for convenience and performance. More...
 

Detailed Description

template<class AccumType, class DataIterator, class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
class casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >

Class to calculate statistics using the so-called fit to half algorithm.

In this algorithm, a center value is specified, and only points greater or equal or less or equal this value are included. Furthermore, each of the included points is reflected about the center value, and these virtual points are added to the included points and the union of sets of included real points and virtual points are used for computing statistics. The specified center point is therefore the mean and median of the resulting distribution, and the total number of points is exactly twice the number of real data points that are included.

This class uses a ConstrainedRangeQuantileComputer object for computing quantile-like statistics. See class documentation for StatisticsAlgorithm for details regarding QuantileComputer classes.

Definition at line 54 of file FitToHalfStatistics.h.

Constructor & Destructor Documentation

◆ FitToHalfStatistics() [1/2]

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::FitToHalfStatistics ( FitToHalfStatisticsData::CENTER  center = FitToHalfStatisticsData::CMEAN,
FitToHalfStatisticsData::USE_DATA  useData = FitToHalfStatisticsData::LE_CENTER,
AccumType  value = 0 
)

value is only used if center=CVALUE

◆ FitToHalfStatistics() [2/2]

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::FitToHalfStatistics ( const FitToHalfStatistics< CASA_STATP > &  other)

copy semantics

◆ ~FitToHalfStatistics()

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
virtual casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::~FitToHalfStatistics ( )
virtual

Member Function Documentation

◆ _getInitialStats()

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
virtual StatsData<AccumType> casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_getInitialStats ( ) const
protectedvirtual

◆ _getMinMax()

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
void casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_getMinMax ( CountedPtr< AccumType > &  realMin,
CountedPtr< AccumType > &  realMax,
CountedPtr< AccumType >  knownMin,
CountedPtr< AccumType >  knownMax 
)
private

get the min max of the entire (real + virtual) data set.

Only used for quantile computation

◆ _getRealMinMax()

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
void casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_getRealMinMax ( AccumType &  realMin,
AccumType &  realMax 
)
private

get the min/max of the real portion only of the dataset

◆ _getStatistics()

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
virtual StatsData<AccumType> casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_getStatistics ( )
protectedvirtual

◆ _getStatsData() [1/2]

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
StatsData<AccumType>& casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_getStatsData ( )
inlineprotectedvirtual

Retrieve stats structure.

Allows derived classes to maintain their own StatsData structs.

Reimplemented from casacore::ClassicalStatistics< CASA_STATP >.

Definition at line 187 of file FitToHalfStatistics.h.

References casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_statsData.

◆ _getStatsData() [2/2]

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
const StatsData<AccumType>& casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_getStatsData ( ) const
inlineprotectedvirtual

◆ _setRange()

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
void casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_setRange ( )
privatevirtual

derived classes need to implement how to set their respective range

Implements casacore::ConstrainedRangeStatistics< CASA_STATP >.

◆ _unweightedStats() [1/4]

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
void casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_unweightedStats ( StatsData< AccumType > &  stats,
uInt64 ngood,
LocationType location,
const DataIterator &  dataBegin,
uInt64  nr,
uInt  dataStride 
)
protected

no weights, no mask, no ranges

◆ _unweightedStats() [2/4]

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
void casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_unweightedStats ( StatsData< AccumType > &  stats,
uInt64 ngood,
LocationType location,
const DataIterator &  dataBegin,
uInt64  nr,
uInt  dataStride,
const DataRanges ranges,
Bool  isInclude 
)
protected

no weights, no mask

◆ _unweightedStats() [3/4]

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
void casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_unweightedStats ( StatsData< AccumType > &  stats,
uInt64 ngood,
LocationType location,
const DataIterator &  dataBegin,
uInt64  nr,
uInt  dataStride,
const MaskIterator &  maskBegin,
uInt  maskStride 
)
protected

◆ _unweightedStats() [4/4]

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
void casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_unweightedStats ( StatsData< AccumType > &  stats,
uInt64 ngood,
LocationType location,
const DataIterator &  dataBegin,
uInt64  nr,
uInt  dataStride,
const MaskIterator &  maskBegin,
uInt  maskStride,
const DataRanges ranges,
Bool  isInclude 
)
protected

◆ _updateDataProviderMaxMin()

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
void casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_updateDataProviderMaxMin ( const StatsData< AccumType > &  threadStats)
protected

◆ _weightedStats() [1/4]

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
void casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_weightedStats ( StatsData< AccumType > &  stats,
LocationType location,
const DataIterator &  dataBegin,
const WeightsIterator &  weightBegin,
uInt64  nr,
uInt  dataStride,
const MaskIterator &  maskBegin,
uInt  maskStride 
)
protected

◆ _weightedStats() [2/4]

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
void casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_weightedStats ( StatsData< AccumType > &  stats,
LocationType location,
const DataIterator &  dataBegin,
const WeightsIterator &  weightBegin,
uInt64  nr,
uInt  dataStride,
const MaskIterator &  maskBegin,
uInt  maskStride,
const DataRanges ranges,
Bool  isInclude 
)
protected

◆ _weightedStats() [3/4]

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
void casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_weightedStats ( StatsData< AccumType > &  stats,
LocationType location,
const DataIterator &  dataBegin,
const WeightsIterator &  weightsBegin,
uInt64  nr,
uInt  dataStride 
)
protected

has weights, but no mask, no ranges

◆ _weightedStats() [4/4]

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
void casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_weightedStats ( StatsData< AccumType > &  stats,
LocationType location,
const DataIterator &  dataBegin,
const WeightsIterator &  weightsBegin,
uInt64  nr,
uInt  dataStride,
const DataRanges ranges,
Bool  isInclude 
)
protected

◆ algorithm()

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
virtual StatisticsData::ALGORITHM casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::algorithm ( ) const
inlinevirtual

get the algorithm that this object uses for computing stats

Reimplemented from casacore::ClassicalStatistics< CASA_STATP >.

Definition at line 82 of file FitToHalfStatistics.h.

References casacore::StatisticsData::FITTOHALF.

◆ clone()

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
virtual StatisticsAlgorithm<CASA_STATP>* casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::clone ( ) const
virtual

Clone this instance.

Caller is responsible for deleting the returned pointer.

Reimplemented from casacore::ClassicalStatistics< CASA_STATP >.

◆ getMedian()

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
AccumType casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::getMedian ( CountedPtr< uInt64 knownNpts = nullptr,
CountedPtr< AccumType >  knownMin = nullptr,
CountedPtr< AccumType >  knownMax = nullptr,
uInt  binningThreshholdSizeBytes = 4096 *4096,
Bool  persistSortedArray = False,
uInt  nBins = 10000 
)

The median is just the center value, so none of the parameters to this method are used.

◆ getMedianAbsDevMed()

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
AccumType casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::getMedianAbsDevMed ( CountedPtr< uInt64 knownNpts = nullptr,
CountedPtr< AccumType >  knownMin = nullptr,
CountedPtr< AccumType >  knownMax = nullptr,
uInt  binningThreshholdSizeBytes = 4096 *4096,
Bool  persistSortedArray = False,
uInt  nBins = 10000 
)

get the median of the absolute deviation about the median of the data.

◆ getMedianAndQuantiles()

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
AccumType casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::getMedianAndQuantiles ( std::map< Double, AccumType > &  quantiles,
const std::set< Double > &  fractions,
CountedPtr< uInt64 knownNpts = nullptr,
CountedPtr< AccumType >  knownMin = nullptr,
CountedPtr< AccumType >  knownMax = nullptr,
uInt  binningThreshholdSizeBytes = 4096 *4096,
Bool  persistSortedArray = False,
uInt  nBins = 10000 
)

In the following group of methods, if the size of the composite dataset is smaller than binningThreshholdSizeBytes, the composite dataset will be (perhaps partially) sorted and persisted in memory during the call.

In that case, and if persistSortedArray is True, this sorted array will remain in memory after the call and will be used on subsequent calls of this method when binningThreshholdSizeBytes is greater than the size of the composite dataset. If persistSortedArray is False, the sorted array will not be stored after this call completes and so any subsequent calls for which the dataset size is less than binningThreshholdSizeBytes, the dataset will be sorted from scratch. Values which are not included due to non-unity strides, are not included in any specified ranges, are masked, or have associated weights of zero are not considered as dataset members for quantile computations. If one has a priori information regarding the number of points (npts) and/or the minimum and maximum values of the data set, these can be supplied to improve performance. Note however, that if these values are not correct, the resulting median and/or quantile values will also not be correct (although see the following notes regarding max/min). Note that if this object has already had getStatistics() called, and the min and max were calculated, there is no need to pass these values in as they have been stored internally and will be used (although passing them explicitly shouldn't hurt anything). If provided, npts, the number of points falling in the specified ranges which are not masked and have weights > 0, should be correct. min can be less than the true minimum, and max can be greater than the True maximum, but for best performance, these should be as close to the actual min and max as possible (and ideally the actual min/max values of the data set).

◆ getMinMax()

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
virtual void casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::getMinMax ( AccumType &  mymin,
AccumType &  mymax 
)
virtual

scan the dataset(s) that have been added, and find the min and max.

This method may be called even if setStatsToCaclulate has been called and MAX and MIN has been excluded.

◆ getNPts()

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
uInt64 casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::getNPts ( )
virtual

scan the dataset(s) that have been added, and find the number of good points.

This method may be called even if setStatsToCaclulate has been called and NPTS has been excluded. If setCalculateAsAdded(True) has previously been called after this object has been (re)initialized, an exception will be thrown.

Reimplemented from casacore::ConstrainedRangeStatistics< CASA_STATP >.

◆ getQuantiles()

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
std::map<Double, AccumType> casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::getQuantiles ( const std::set< Double > &  fractions,
CountedPtr< uInt64 knownNpts = nullptr,
CountedPtr< AccumType >  knownMin = nullptr,
CountedPtr< AccumType >  knownMax = nullptr,
uInt  binningThreshholdSizeBytes = 4096 *4096,
Bool  persistSortedArray = False,
uInt  nBins = 10000 
)

Get the specified quantiles.

fractions must be between 0 and 1, noninclusive.

◆ operator=()

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
FitToHalfStatistics<CASA_STATP>& casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::operator= ( const FitToHalfStatistics< CASA_STATP > &  other)

copy semantics

◆ reset()

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
virtual void casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::reset ( )
virtual

reset object to initial state.

Clears all private fields including data, accumulators, global range. It does not affect the center type, center value, or which "side" to use which were set at construction.

Reimplemented from casacore::ConstrainedRangeStatistics< CASA_STATP >.

◆ setCalculateAsAdded()

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
void casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::setCalculateAsAdded ( Bool  c)
virtual

This class does not allow statistics to be calculated as datasets are added, so an exception will be thrown if c is True.

Reimplemented from casacore::ClassicalStatistics< CASA_STATP >.

◆ setStatsToCalculate()

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
virtual void casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::setStatsToCalculate ( std::set< StatisticsData::STATS > &  stats)
virtual

Override base class method by requiring mean to be computed in addition to what is added in stats if the requested center value is CMEAN.

Reimplemented from casacore::ClassicalStatistics< CASA_STATP >.

Member Data Documentation

◆ _centerType

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
FitToHalfStatisticsData::CENTER casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_centerType
private

Definition at line 253 of file FitToHalfStatistics.h.

◆ _centerValue

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
AccumType casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_centerValue
private

Definition at line 255 of file FitToHalfStatistics.h.

◆ _doMedAbsDevMed

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
Bool casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_doMedAbsDevMed
private

Definition at line 257 of file FitToHalfStatistics.h.

◆ _isNullSet

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
Bool casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_isNullSet
private

Definition at line 260 of file FitToHalfStatistics.h.

◆ _range

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
CountedPtr<std::pair<AccumType, AccumType> > casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_range
private

This is used for convenience and performance.

It should always be the same as the _range value used in the base ConstrainedRangeStatistics object

Definition at line 264 of file FitToHalfStatistics.h.

◆ _rangeIsSet

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
Bool casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_rangeIsSet
private

Definition at line 257 of file FitToHalfStatistics.h.

◆ _realMax

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
CountedPtr<AccumType> casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_realMax
private

these are the max and min for the real portion of the dataset

Definition at line 259 of file FitToHalfStatistics.h.

◆ _realMin

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
CountedPtr<AccumType> casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_realMin
private

Definition at line 259 of file FitToHalfStatistics.h.

◆ _statsData

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
StatsData<AccumType> casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_statsData
private

◆ _useLower

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
Bool casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::_useLower
private

Definition at line 254 of file FitToHalfStatistics.h.

◆ TWO

template<class AccumType , class DataIterator , class MaskIterator = const Bool *, class WeightsIterator = DataIterator>
const AccumType casacore::FitToHalfStatistics< AccumType, DataIterator, MaskIterator, WeightsIterator >::TWO
static

Definition at line 58 of file FitToHalfStatistics.h.


The documentation for this class was generated from the following file: