casys.computation.diagnostics

Diagnostics representation and computation sub-module.

Classes

BaseDiagnostic(name[, store_params])

Abstract class defining the common structure of a diagnostic: computation, results, storage and schema.

Binned2dDiag(name[, store_params, field_y, ...])

Class representing a non-geographical box statistic with two dimensions.

BinnedDiag(name[, store_params, field_y, ...])

Class representing a non-geographical box statistic.

BinnedDiagBase(name[, store_params, ...])

Abstract class that represents a binned diagnostic.

BinnedResult([data, count, index, ...])

Binned diagnostic results.

CountDiag(name[, store_params, field_y, ...])

Count related diagnostics.

CountResult([data, count, index, internal_stats])

Count diagnostic results.

CrossoverDiag(name[, store_params, field_y, ...])

Class representing a crossover statistic.

CrossoverResult([delta, geobox, temporal, ...])

Crossover diagnostic results.

CrossoversBuilderContainer()

Shared CrossoversBuilder for all crossovers of a NadirData instance.

Diagnostic(name[, store_params, field_y, ...])

Abstract class defining the common structure of a diagnostic: computation, results, storage, schema and visualization, and related attributes.

DiagnosticsGroup(name)

Group of diagnostics that can be computed together.

DiagResult()

Diagnostics results base class.

EditingDiag(name[, store_params, field_y, ...])

Class representing an editing.

EmptyResult()

Results containing nothing.

GeoBoxDiag(name[, store_params, field_y, ...])

Class representing a geographical box statistic.

HistogramDiag(name[, store_params, field_y, ...])

Class representing a histogram statistic.

MissingPointsDiag(name[, store_params, ...])

Class representing missing points data and statistic.

MissingPointsResult(missing[, geobox, ...])

Missing points diagnostic results.

PeriodogramDiag(name[, store_params])

Container for the periodogram diagnostic.

RatioDiag(name[, store_params])

Container for the ratio diagnostic.

RawComparisonDiag(name[, store_params, ...])

Class representing a raw comparison data.

RawComparison3DDiag(name[, store_params, ...])

Class representing a raw comparison data.

SpectralAnalysisResult(data, segments, time, ...)

Spectral analysis diagnostic results.

SpectralAnalysisSwathResult(data)

Spectral diagnostic Swath results.

RawDataResult(index)

Raw data diagnostic results.

RawDiag(name[, store_params, field_y, ...])

Class representing a raw data associated to its time, longitude and latitude.

RawGenerator(name[, store_params, field_y, ...])

Abstract class that represents a raw data generator.

ScatterDiag(name[, store_params, field_y, ...])

Class representing a scatter statistic.

SectionAnalysesDiag(name[, store_params, ...])

Class representing section analyses data.

SpectralAnalysisDiag(name[, store_params, ...])

Spectral analysis diagnostic.

StatDiagResult()

Common result base for diagnostics computing statistics.

StatisticDiag(name[, store_params, field_y, ...])

Abstract class defining the common structure of a statistical diagnostic: computation, results, storage, schema and visualization, and related methods and attributes.

TemporalDiag(name[, store_params, field_y, ...])

Class representing a time statistic.

Binned2dSwathDiag(name[, store_params, ...])

Class representing a non-geographical box statistic with two dimensions.

BinnedSwathDiag(name[, store_params, ...])

Class representing a non-geographical box statistic.

CrossoverSwathDiag(name[, store_params, ...])

Class representing a Swath crossover statistic.

GeoBoxSwathDiag(name[, store_params, ...])

Class representing a geographical box statistic.

HistogramSwathDiag(name[, store_params, ...])

Class representing a histogram statistic.

RawComparisonSwathDiag(name[, store_params, ...])

Class representing a 2D (x,y) raw comparison data.

RawComparison3DSwathDiag(name[, ...])

Class representing a 3D (x,y,z) raw comparison data.

ScatterSwathDiag(name[, store_params, ...])

Class representing a scatter statistic.

SpectralAnalysisSwathDiag(name[, ...])

Swath spectral analysis diagnostic.

TemporalSwathDiag(name[, store_params, ...])

Class representing a time statistic.

RawSwathDiag(name[, store_params, field_y, ...])

Class representing a Swath raw data associated to its time, longitude and latitude.

SpectralConfResult(data, wn, individual, ...)

Spectral analysis diagnostic sub results for spectral curves.
class casys.computation.diagnostics.BaseDiagnostic(name, store_params=None)

Bases: Generic[DiagResultType, StoragePropertiesType], ABC

Abstract class defining the common structure of a diagnostic: computation, results, storage and schema.

Parameters:

  • name (str) – Name of the data.

  • computed – True if the data have been computed, false otherwise.

  • is_post_diag – Whether this diagnostic is a post computation diagnostic or not.

  • store_params (dict[str, Any] | None)

RES_TYPE: ClassVar[type[DiagResult]] = None
SCHEMA: ClassVar[type[Schema]] = None
computed: bool = False
dask_full_compatible: bool = False
abstract html_elem(style='')

HTML representation of the data container.

Parameters:

style (str) – Style to add to the element.

Return type:

str

Returns:

Html representation of elements of this object.

abstract html_header(span)

HTML representation header of the data container.

Parameters:

span (int) – Span size.

Return type:

str

Returns:

Object html header.

property identity: DiagnosticType
is_autonomous(dask)

Check whether this diagnostic is autonomous with or without dask enabled.

Parameters:

dask (bool) – Whether dask is enabled or not.

Returns:

Whether this is an autonomous diagnostic or not.

is_post_diag: bool = False
metadata(internal=False)

Diagnostic serializable configuration.

Parameters:

internal (bool) – Whether this diagnostic is an internal diagnostic or not.

Return type:

DiagnosticMeta

Returns:

Diagnostic metadata.

name: str
results: TypeVar(DiagResultType, bound= DiagResult)
storage: TypeVar(StoragePropertiesType, bound= StorageProperties) = None
store(store, group, analyse_type, analyse_date, start, end, mode, fields)

Write the diagnostic results into the provided store.

Parameters:

  • store (DiagnosticStore) – DiagnosticStore used for the storage.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the computed data.

  • end (datetime64) – Ending date for the computed data.

  • mode (StorageMode) – Storage mode to use when writing data.

  • fields (list[str]) – List of all fields name used in the group of this statistic.

store_configuration(analyse_type, analyse_date, group=None)

Returns the list of storage group parameters required to store this diagnostic.

Parameters:

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date representing the set of data used in this analyse. It’s used to determine at which timestamp to store non-temporal diagnostics.

  • group (str | None) – Name of the group to use for storage.

Return type:

list[StorageGroupParams]

Returns:

List of storage group parameters.

store_params: dict[str, Any] | None = None
class casys.computation.diagnostics.Binned2dDiag(name, store_params=None, *, data_index, data_time, start, end, field_x, field_y=None, indexer=None, time_extension=False, stats=<factory>, stat_selection=None, field_z, res_x, res_y)

Bases: BinnedDiagBase[BinnedResult, Binned2dStorageProperties]

Class representing a non-geographical box statistic with two dimensions.

Parameters:

  • name (str) – Name of the data.

  • computed – True if the data have been computed, false otherwise.

  • is_post_diag – Whether this diagnostic is a post computation diagnostic or not.

  • data_index (str) – Name of the field on which the data is indexed.

  • start (np.datetime64) – Starting date of the input dataset.

  • end (np.datetime64) – Ending date of the input dataset.

  • field_x (Field) – Field used for x axes.

  • field_y (Field | None) – Field used for y axes.

  • stats (list[StatType]) – Statistics included in this Diagnostic.

  • indexer (ong_indexer_t.PassIndexer | None) – Indexer used to read data.

  • time_extension (bool) – Whether time extension is enabled or not.

  • ext_start – Extended starting date required by the computation.

  • ext_end – Ending date for the data required by the computation.

  • group – the name of the group of this diagnostic.

  • stat_selection (ong_clip_t.Clip | None) – Selection clip used to invalidate some bins.

  • field_z (Field) – Field used for z axes.

  • res_x (DataResolution) – Min, max and width for the x-axis.

  • res_y (DataResolution) – Min, max and width for the y-axis.

  • store_params (dict[str, Any] | None)

  • data_time (str)

INDEX_USED: ClassVar[str] = 'INDEX_USED'
MERGEABLE_STAT: ClassVar[list[StatType]] = [MIN, MAX, COUNT, MEAN, VAR, STD, PROD, SUM]
RES_TYPE

alias of BinnedResult

SCHEMA: ClassVar[type[mm.Schema]] = None
STORABLE: ClassVar[bool] = True
apply_stats_selection(diagnostic, fields)

Filter diagnostic by applying the statistic selection clip and removing unwanted statistics.

Parameters:

  • diagnostic (TypeVar(BinnedResultType, bound= BinnedResult)) – Diagnostic to filter.

  • fields (list[str]) – List of diagnostic field’s names.

Return type:

TypeVar(BinnedResultType, bound= BinnedResult)

check_stat(stat)

Check whether this data is computed and provided stat is valid. Determines the stat if None is provided.

Parameters:

stat (StatType | None) – Statistic to check.

Return type:

StatType

Returns:

Checked statistic.

Raises:

AltiDataError – If data with the provided name were not computed. If the stat parameter is invalid.

clear_data()

Clear stored data from this Diagnostic data.

compute(data, start, end, fields, diags)

Compute the diagnostic on the provided data.

Parameters:

  • data (Dataset) – The dataset with data to read.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • diags (list[StatisticDiag]) – List of diagnostics included in this computation.

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Dictionary containing the statistic type with its associated data array.

compute_from_store(store, group, analyse_type, analyse_date, start, end, fields, delayed)

Compute diagnostic using stored data. The computation can be done in a delayed way allowing to only load requested data when needed.

Parameters:

  • store (DiagnosticStore) – Store containing diagnostics data.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • delayed (bool) – Flag indicating whether to load from store during computation or at first data request (plot/get_data).

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Computation results.

computed: bool = False
dask_full_compatible: bool = False
data_index: str
data_time: str
diag_data(**kwargs)

Diagnostic data as a dataset.

Parameters:

kwargs – Additional parameters required to retrieve diagnostic data.

Return type:

Dataset

Returns:

Dataset containing this diagnostic data.

end: np.datetime64
property ext_end: datetime64
property ext_start: datetime64
property field: Field

Field of the main variable represented by this data.

field_x: Field
field_y: Field | None = None
field_z: Field
property fields: dict[str, Field]

Dictionary of fields used to compute this data.

fill_count_with_nan()

Whether count statistics have to be filled with nan or not.

Return type:

bool

Returns:

True or False.

finalize(result)

Correctly format the diagnostic result.

Parameters:

result (TypeVar(BinnedResultType, bound= BinnedResult)) – Result to finalize.

Return type:

TypeVar(BinnedResultType, bound= BinnedResult)

Returns:

Finalized result.

static grid_2d(res_x, res_y)

Generate a Grid2D from provided parameters.

Parameters:
Return type:

Grid2D

Returns:

Grid2D with provided resolutions.

group: str = None
group_base: str = None
html_elem(style='')

HTML representation of the data container.

Parameters:

style (str) – Style to add to the element.

Return type:

str

Returns:

Html representation of elements of this object.

html_header(span)

HTML representation header of the data container.

Parameters:

span (int) – Span size.

Returns:

Object html header.

property identity: DiagnosticType
indexer: ong_indexer_t.PassIndexer | None = None
initialize()

Diagnostic initialization.

At least the following properties have to be initialized:

  • group_base

  • storage_properties

is_autonomous(dask)

Check whether this diagnostic is autonomous with or without dask enabled.

Parameters:

dask (bool) – Whether dask is enabled or not.

Returns:

Whether this is an autonomous diagnostic or not.

is_dask_compatible(freq)

Check whether this diagnostic can be computed with dask or not.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

Return type:

bool

is_post_diag: bool = False
load_diagnostic(**kwargs)

Ensure requested diagnostic data are loaded.

make_dask_compatible(freq)

Set this container, so it can be used with dask.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

merge(diagnostics)

Merge provided diagnostics.

Parameters:

diagnostics (list[TypeVar(CountResultType, bound= CountResult)]) – Diagnostics to merge.

Return type:

TypeVar(CountResultType, bound= CountResult)

Returns:

Merged diagnostic.

metadata(internal=False)

Diagnostic serializable configuration.

Parameters:

internal (bool) – Whether this diagnostic is an internal diagnostic or not.

Return type:

DiagnosticMeta

Returns:

Diagnostic metadata.

name: str
new_algorithm()

Instantiate the algorithm.

Return type:

BinnedStat

Returns:

Instantiated algorithm.

plot_container(template=None, stat=None, plot=None, axis=None)

Create a PlotContainer for the provided stat data.

Parameters:

  • template (PlotTemplate | None) – Parameters template to use.

  • stat (StatType | None) – Statistic included in this data.

  • plot (str | None) – Plot type to display (“BOX”, “CURVE”, “3D” or “BOX3D”).

  • axis (str | None) – Axis to plot the curves along, only for plot=”CURVE” case.

Return type:

PlotContainer

Returns:

PlotContainer containing the requested data.

res_x: DataResolution
res_y: DataResolution
results: DiagResultType
run_algorithm(data, fields)

Run the algorithm on the provided dataset.

Parameters:

  • data (Dataset) – Data on which to apply the algorithm.

  • fields (list[str]) – List of fields for which we want results.

Return type:

TypeVar(BinnedResultType, bound= BinnedResult)

Returns:

Computation result.

start: np.datetime64
stat_selection: ong_clip_t.Clip | None = None
stats: list[StatType]
storage: StoragePropertiesType = None
store(store, group, analyse_type, analyse_date, start, end, mode, fields)

Write the diagnostic results into the provided store.

Parameters:

  • store (DiagnosticStore) – DiagnosticStore used for the storage.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the computed data.

  • end (datetime64) – Ending date for the computed data.

  • mode (StorageMode) – Storage mode to use when writing data.

  • fields (list[str]) – List of all fields name used in the group of this statistic.

store_configuration(analyse_type, analyse_date, group=None)

Returns the list of storage group parameters required to store this diagnostic.

Parameters:

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date representing the set of data used in this analyse. It’s used to determine at which timestamp to store non-temporal diagnostics.

  • group (str | None) – Name of the group to use for storage.

Return type:

list[StorageGroupParams]

Returns:

List of storage group parameters.

store_params: dict[str, Any] | None = None
time_extension: bool = False
update_properties(diag)

Update diagnostic properties with the provided one.

Parameters:

diag (StatisticDiag) – Diagnostic from which to get properties.

property variable: str

Name of the main variable represented by this data.

class casys.computation.diagnostics.Binned2dSwathDiag(name, store_params=None, *, data_index, data_time, start, end, field_x, field_y=None, indexer=None, time_extension=False, stats=<factory>, stat_selection=None, field_z, res_x, res_y)

Bases: Binned2dDiag

Class representing a non-geographical box statistic with two dimensions.

Parameters:

  • name (str) – Name of the data.

  • computed – True if the data have been computed, false otherwise.

  • is_post_diag – Whether this diagnostic is a post computation diagnostic or not.

  • data_index (str) – Name of the field on which the data is indexed.

  • start (np.datetime64) – Starting date of the input dataset.

  • end (np.datetime64) – Ending date of the input dataset.

  • field_x (Field) – Field used for x axes.

  • field_y (Field | None) – Field used for y axes.

  • stats (list[StatType]) – Statistics included in this Diagnostic.

  • indexer (ong_indexer_t.PassIndexer | None) – Indexer used to read data.

  • time_extension (bool) – Whether time extension is enabled or not.

  • ext_start – Extended starting date required by the computation.

  • ext_end – Ending date for the data required by the computation.

  • group – the name of the group of this diagnostic.

  • stat_selection (ong_clip_t.Clip | None) – Selection clip used to invalidate some bins.

  • field_z (Field) – Field used for z axes.

  • res_x (DataResolution) – Min, max and width for the x-axis.

  • res_y (DataResolution) – Min, max and width for the y-axis.

  • store_params (dict[str, Any] | None)

  • data_time (str)

INDEX_USED: ClassVar[str] = 'INDEX_USED'
MERGEABLE_STAT: ClassVar[list[StatType]] = [MIN, MAX, COUNT, MEAN, VAR, STD, PROD, SUM]
RES_TYPE

alias of BinnedResult

SCHEMA: ClassVar[type[mm.Schema]] = None
STORABLE: ClassVar[bool] = True
apply_stats_selection(diagnostic, fields)

Filter diagnostic by applying the statistic selection clip and removing unwanted statistics.

Parameters:

  • diagnostic (TypeVar(BinnedResultType, bound= BinnedResult)) – Diagnostic to filter.

  • fields (list[str]) – List of diagnostic field’s names.

Return type:

TypeVar(BinnedResultType, bound= BinnedResult)

check_stat(stat)

Check whether this data is computed and provided stat is valid. Determines the stat if None is provided.

Parameters:

stat (StatType | None) – Statistic to check.

Return type:

StatType

Returns:

Checked statistic.

Raises:

AltiDataError – If data with the provided name were not computed. If the stat parameter is invalid.

clear_data()

Clear stored data from this Diagnostic data.

compute(data, start, end, fields, diags)

Compute the diagnostic on the provided data.

Parameters:

  • data (Dataset) – The dataset with data to read.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • diags (list[StatisticDiag]) – List of diagnostics included in this computation.

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Dictionary containing the statistic type with its associated data array.

compute_from_store(store, group, analyse_type, analyse_date, start, end, fields, delayed)

Compute diagnostic using stored data. The computation can be done in a delayed way allowing to only load requested data when needed.

Parameters:

  • store (DiagnosticStore) – Store containing diagnostics data.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • delayed (bool) – Flag indicating whether to load from store during computation or at first data request (plot/get_data).

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Computation results.

computed: bool = False
dask_full_compatible: bool = False
data_index: str
data_time: str
diag_data(**kwargs)

Diagnostic data as a dataset.

Parameters:

kwargs – Additional parameters required to retrieve diagnostic data.

Return type:

Dataset

Returns:

Dataset containing this diagnostic data.

end: np.datetime64
property ext_end: datetime64
property ext_start: datetime64
property field: Field

Field of the main variable represented by this data.

field_x: Field
field_y: Field | None = None
field_z: Field
property fields: dict[str, Field]

Dictionary of fields used to compute this data.

fill_count_with_nan()

Whether count statistics have to be filled with nan or not.

Return type:

bool

Returns:

True or False.

finalize(result)

Correctly format the diagnostic result.

Parameters:

result (TypeVar(BinnedResultType, bound= BinnedResult)) – Result to finalize.

Return type:

TypeVar(BinnedResultType, bound= BinnedResult)

Returns:

Finalized result.

static grid_2d(res_x, res_y)

Generate a Grid2D from provided parameters.

Parameters:
Return type:

Grid2D

Returns:

Grid2D with provided resolutions.

group: str = None
group_base: str = None
html_elem(style='')

HTML representation of the data container.

Parameters:

style (str) – Style to add to the element.

Return type:

str

Returns:

Html representation of elements of this object.

html_header(span)

HTML representation header of the data container.

Parameters:

span (int) – Span size.

Returns:

Object html header.

property identity: DiagnosticType
indexer: ong_indexer_t.PassIndexer | None = None
initialize()

Diagnostic initialization.

At least the following properties have to be initialized:

  • group_base

  • storage_properties

is_autonomous(dask)

Check whether this diagnostic is autonomous with or without dask enabled.

Parameters:

dask (bool) – Whether dask is enabled or not.

Returns:

Whether this is an autonomous diagnostic or not.

is_dask_compatible(freq)

Check whether this diagnostic can be computed with dask or not.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

Return type:

bool

is_post_diag: bool = False
load_diagnostic(**kwargs)

Ensure requested diagnostic data are loaded.

make_dask_compatible(freq)

Set this container, so it can be used with dask.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

merge(diagnostics)

Merge provided diagnostics.

Parameters:

diagnostics (list[TypeVar(CountResultType, bound= CountResult)]) – Diagnostics to merge.

Return type:

TypeVar(CountResultType, bound= CountResult)

Returns:

Merged diagnostic.

metadata(internal=False)

Diagnostic serializable configuration.

Parameters:

internal (bool) – Whether this diagnostic is an internal diagnostic or not.

Return type:

DiagnosticMeta

Returns:

Diagnostic metadata.

name: str
new_algorithm()

Instantiate the algorithm.

Return type:

BinnedStat

Returns:

Instantiated algorithm.

plot_container(template=None, stat=None, plot=None, axis=None)

Create a PlotContainer for the provided stat data.

Parameters:

  • template (PlotTemplate | None) – Parameters template to use.

  • stat (StatType | None) – Statistic included in this data.

  • plot (str | None) – Plot type to display (“BOX”, “CURVE”, “3D” or “BOX3D”).

  • axis (str | None) – Axis to plot the curves along, only for plot=”CURVE” case.

Return type:

PlotContainer

Returns:

PlotContainer containing the requested data.

res_x: DataResolution
res_y: DataResolution
results: DiagResultType
run_algorithm(data, fields)

Run the algorithm on the provided dataset.

Parameters:

  • data (Dataset) – Data on which to apply the algorithm.

  • fields (list[str]) – List of fields for which we want results.

Return type:

TypeVar(BinnedResultType, bound= BinnedResult)

Returns:

Computation result.

start: np.datetime64
stat_selection: ong_clip_t.Clip | None = None
stats: list[StatType]
storage: StoragePropertiesType = None
store(store, group, analyse_type, analyse_date, start, end, mode, fields)

Write the diagnostic results into the provided store.

Parameters:

  • store (DiagnosticStore) – DiagnosticStore used for the storage.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the computed data.

  • end (datetime64) – Ending date for the computed data.

  • mode (StorageMode) – Storage mode to use when writing data.

  • fields (list[str]) – List of all fields name used in the group of this statistic.

store_configuration(analyse_type, analyse_date, group=None)

Returns the list of storage group parameters required to store this diagnostic.

Parameters:

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date representing the set of data used in this analyse. It’s used to determine at which timestamp to store non-temporal diagnostics.

  • group (str | None) – Name of the group to use for storage.

Return type:

list[StorageGroupParams]

Returns:

List of storage group parameters.

store_params: dict[str, Any] | None = None
time_extension: bool = False
update_properties(diag)

Update diagnostic properties with the provided one.

Parameters:

diag (StatisticDiag) – Diagnostic from which to get properties.

property variable: str

Name of the main variable represented by this data.

class casys.computation.diagnostics.BinnedDiag(name, store_params=None, *, data_index, data_time, start, end, field_x, field_y=None, indexer=None, time_extension=False, stats=<factory>, stat_selection=None, res_x)

Bases: BinnedDiagBase[BinnedResult, BinnedStorageProperties]

Class representing a non-geographical box statistic.

Parameters:

  • name (str) – Name of the data.

  • computed – True if the data have been computed, false otherwise.

  • is_post_diag – Whether this diagnostic is a post computation diagnostic or not.

  • data_index (str) – Name of the field on which the data is indexed.

  • start (np.datetime64) – Starting date of the input dataset.

  • end (np.datetime64) – Ending date of the input dataset.

  • field_x (Field) – Field used for x axes.

  • field_y (Field | None) – Field used for y axes.

  • stats (list[StatType]) – Statistics included in this Diagnostic.

  • indexer (ong_indexer_t.PassIndexer | None) – Indexer used to read data.

  • time_extension (bool) – Whether time extension is enabled or not.

  • ext_start – Extended starting date required by the computation.

  • ext_end – Ending date for the data required by the computation.

  • group – the name of the group of this diagnostic.

  • stat_selection (ong_clip_t.Clip | None) – Selection clip used to invalidate some bins.

  • res_x (DataResolution) – Min, max and width for the x-axis.

  • store_params (dict[str, Any] | None)

  • data_time (str)

INDEX_USED: ClassVar[str] = 'INDEX_USED'
MERGEABLE_STAT: ClassVar[list[StatType]] = [MIN, MAX, COUNT, MEAN, VAR, STD, PROD, SUM]
RES_TYPE

alias of BinnedResult

SCHEMA: ClassVar[type[mm.Schema]] = None
STORABLE: ClassVar[bool] = True
apply_stats_selection(diagnostic, fields)

Filter diagnostic by applying the statistic selection clip and removing unwanted statistics.

Parameters:

  • diagnostic (TypeVar(BinnedResultType, bound= BinnedResult)) – Diagnostic to filter.

  • fields (list[str]) – List of diagnostic field’s names.

Return type:

TypeVar(BinnedResultType, bound= BinnedResult)

check_stat(stat)

Check whether this data is computed and provided stat is valid. Determines the stat if None is provided.

Parameters:

stat (StatType | None) – Statistic to check.

Return type:

StatType

Returns:

Checked statistic.

Raises:

AltiDataError – If data with the provided name were not computed. If the stat parameter is invalid.

clear_data()

Clear stored data from this Diagnostic data.

compute(data, start, end, fields, diags)

Compute the diagnostic on the provided data.

Parameters:

  • data (Dataset) – The dataset with data to read.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • diags (list[StatisticDiag]) – List of diagnostics included in this computation.

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Dictionary containing the statistic type with its associated data array.

compute_from_store(store, group, analyse_type, analyse_date, start, end, fields, delayed)

Compute diagnostic using stored data. The computation can be done in a delayed way allowing to only load requested data when needed.

Parameters:

  • store (DiagnosticStore) – Store containing diagnostics data.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • delayed (bool) – Flag indicating whether to load from store during computation or at first data request (plot/get_data).

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Computation results.

computed: bool = False
dask_full_compatible: bool = False
data_index: str
data_time: str
diag_data(**kwargs)

Diagnostic data as a dataset.

Parameters:

kwargs – Additional parameters required to retrieve diagnostic data.

Return type:

Dataset

Returns:

Dataset containing this diagnostic data.

end: np.datetime64
property ext_end: datetime64
property ext_start: datetime64
property field: Field

Field of the main variable represented by this data.

field_x: Field
field_y: Field | None = None
property fields: dict[str, Field]

Dictionary of fields used to compute this data.

fill_count_with_nan()

Whether count statistics have to be filled with nan or not.

Return type:

bool

Returns:

True or False.

finalize(result)

Correctly format the diagnostic result.

Parameters:

result (TypeVar(BinnedResultType, bound= BinnedResult)) – Result to finalize.

Return type:

TypeVar(BinnedResultType, bound= BinnedResult)

Returns:

Finalized result.

static grid_2d(res_x, res_y)

Generate a Grid2D from provided parameters.

Parameters:
Return type:

Grid2D

Returns:

Grid2D with provided resolutions.

group: str = None
group_base: str = None
html_elem(style='')

HTML representation of the data container.

Parameters:

style (str) – Style to add to the element.

Return type:

str

Returns:

Html representation of elements of this object.

html_header(span)

HTML representation header of the data container.

Parameters:

span (int) – Span size.

Returns:

Object html header.

property identity: DiagnosticType
indexer: ong_indexer_t.PassIndexer | None = None
initialize()

Diagnostic initialization.

At least the following properties have to be initialized:

  • group_base

  • storage_properties

is_autonomous(dask)

Check whether this diagnostic is autonomous with or without dask enabled.

Parameters:

dask (bool) – Whether dask is enabled or not.

Returns:

Whether this is an autonomous diagnostic or not.

is_dask_compatible(freq)

Check whether this diagnostic can be computed with dask or not.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

Return type:

bool

is_post_diag: bool = False
load_diagnostic(**kwargs)

Ensure requested diagnostic data are loaded.

make_dask_compatible(freq)

Set this container, so it can be used with dask.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

merge(diagnostics)

Merge provided diagnostics.

Parameters:

diagnostics (list[TypeVar(CountResultType, bound= CountResult)]) – Diagnostics to merge.

Return type:

TypeVar(CountResultType, bound= CountResult)

Returns:

Merged diagnostic.

property mergeable: bool
metadata(internal=False)

Diagnostic serializable configuration.

Parameters:

internal (bool) – Whether this diagnostic is an internal diagnostic or not.

Return type:

DiagnosticMeta

Returns:

Diagnostic metadata.

name: str
new_algorithm()

Instantiate the algorithm.

Return type:

BinnedStat

Returns:

Instantiated algorithm.

plot_container(template=None, stat=None)

Create a PlotContainer for the provided stat data.

Parameters:
Return type:

PlotContainer

Returns:

PlotContainer containing the requested data.

res_x: DataResolution
results: DiagResultType
run_algorithm(data, fields)

Run the algorithm on the provided dataset.

Parameters:

  • data (Dataset) – Data on which to apply the algorithm.

  • fields (list[str]) – List of fields for which we want results.

Return type:

TypeVar(BinnedResultType, bound= BinnedResult)

Returns:

Computation result.

start: np.datetime64
stat_selection: ong_clip_t.Clip | None = None
stats: list[StatType]
storage: StoragePropertiesType = None
store(store, group, analyse_type, analyse_date, start, end, mode, fields)

Write the diagnostic results into the provided store.

Parameters:

  • store (DiagnosticStore) – DiagnosticStore used for the storage.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the computed data.

  • end (datetime64) – Ending date for the computed data.

  • mode (StorageMode) – Storage mode to use when writing data.

  • fields (list[str]) – List of all fields name used in the group of this statistic.

store_configuration(analyse_type, analyse_date, group=None)

Returns the list of storage group parameters required to store this diagnostic.

Parameters:

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date representing the set of data used in this analyse. It’s used to determine at which timestamp to store non-temporal diagnostics.

  • group (str | None) – Name of the group to use for storage.

Return type:

list[StorageGroupParams]

Returns:

List of storage group parameters.

store_params: dict[str, Any] | None = None
time_extension: bool = False
update_properties(diag)

Update diagnostic properties with the provided one.

Parameters:

diag (StatisticDiag) – Diagnostic from which to get properties.

property variable: str

Name of the main variable represented by this data.

class casys.computation.diagnostics.BinnedDiagBase(name, store_params=None, *, data_index, data_time, start, end, field_x, field_y=None, indexer=None, time_extension=False, stats=<factory>, stat_selection=None)

Bases: CountDiag, Generic[BinnedResultType, StoragePropertiesType], ABC

Abstract class that represents a binned diagnostic.

Parameters:

  • name (str) – Name of the data.

  • computed – True if the data have been computed, false otherwise.

  • is_post_diag – Whether this diagnostic is a post computation diagnostic or not.

  • data_index (str) – Name of the field on which the data is indexed.

  • start (np.datetime64) – Starting date of the input dataset.

  • end (np.datetime64) – Ending date of the input dataset.

  • field_x (Field) – Field used for x axes.

  • field_y (Field | None) – Field used for y axes.

  • stats (list[StatType]) – Statistics included in this Diagnostic.

  • indexer (ong_indexer_t.PassIndexer | None) – Indexer used to read data.

  • time_extension (bool) – Whether time extension is enabled or not.

  • ext_start – Extended starting date required by the computation.

  • ext_end – Ending date for the data required by the computation.

  • group – the name of the group of this diagnostic.

  • stat_selection (ong_clip_t.Clip | None) – Selection clip used to invalidate some bins.

  • store_params (dict[str, Any] | None)

  • data_time (str)

INDEX_USED: ClassVar[str] = 'INDEX_USED'
MERGEABLE_STAT: ClassVar[list[StatType]] = [MIN, MAX, COUNT, MEAN, VAR, STD, PROD, SUM]
RES_TYPE

alias of BinnedResult

SCHEMA: ClassVar[type[mm.Schema]] = None
STORABLE: ClassVar[bool] = True
apply_stats_selection(diagnostic, fields)

Filter diagnostic by applying the statistic selection clip and removing unwanted statistics.

Parameters:

  • diagnostic (TypeVar(BinnedResultType, bound= BinnedResult)) – Diagnostic to filter.

  • fields (list[str]) – List of diagnostic field’s names.

Return type:

TypeVar(BinnedResultType, bound= BinnedResult)

check_stat(stat)

Check whether this data is computed and provided stat is valid. Determines the stat if None is provided.

Parameters:

stat (StatType | None) – Statistic to check.

Return type:

StatType

Returns:

Checked statistic.

Raises:

AltiDataError – If data with the provided name were not computed. If the stat parameter is invalid.

clear_data()

Clear stored data from this Diagnostic data.

compute(data, start, end, fields, diags)

Compute the diagnostic on the provided data.

Parameters:

  • data (Dataset) – The dataset with data to read.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • diags (list[StatisticDiag]) – List of diagnostics included in this computation.

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Dictionary containing the statistic type with its associated data array.

compute_from_store(store, group, analyse_type, analyse_date, start, end, fields, delayed)

Compute diagnostic using stored data. The computation can be done in a delayed way allowing to only load requested data when needed.

Parameters:

  • store (DiagnosticStore) – Store containing diagnostics data.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • delayed (bool) – Flag indicating whether to load from store during computation or at first data request (plot/get_data).

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Computation results.

computed: bool = False
dask_full_compatible: bool = False
data_index: str
data_time: str
diag_data(**kwargs)

Diagnostic data as a dataset.

Parameters:

kwargs – Additional parameters required to retrieve diagnostic data.

Return type:

Dataset

Returns:

Dataset containing this diagnostic data.

end: np.datetime64
property ext_end: datetime64
property ext_start: datetime64
abstract property field: Field

Field of the main variable represented by this data.

field_x: Field
field_y: Field | None = None
abstract property fields: dict[str, Field]

Dictionary of fields used to compute this data.

fill_count_with_nan()

Whether count statistics have to be filled with nan or not.

Return type:

bool

Returns:

True or False.

finalize(result)

Correctly format the diagnostic result.

Parameters:

result (TypeVar(BinnedResultType, bound= BinnedResult)) – Result to finalize.

Return type:

TypeVar(BinnedResultType, bound= BinnedResult)

Returns:

Finalized result.

static grid_2d(res_x, res_y)

Generate a Grid2D from provided parameters.

Parameters:
Return type:

Grid2D

Returns:

Grid2D with provided resolutions.

group: str = None
group_base: str = None
abstract html_elem(style='')

HTML representation of the data container.

Parameters:

style (str) – Style to add to the element.

Return type:

str

Returns:

Html representation of elements of this object.

abstract html_header(span)

HTML representation header of the data container.

Parameters:

span (int) – Span size.

Return type:

str

Returns:

Object html header.

property identity: DiagnosticType
indexer: ong_indexer_t.PassIndexer | None = None
abstract initialize()

Diagnostic initialization.

At least the following properties have to be initialized:

  • group_base

  • storage_properties

is_autonomous(dask)

Check whether this diagnostic is autonomous with or without dask enabled.

Parameters:

dask (bool) – Whether dask is enabled or not.

Returns:

Whether this is an autonomous diagnostic or not.

is_dask_compatible(freq)

Check whether this diagnostic can be computed with dask or not.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

Return type:

bool

is_post_diag: bool = False
load_diagnostic(**kwargs)

Ensure requested diagnostic data are loaded.

make_dask_compatible(freq)

Set this container, so it can be used with dask.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

merge(diagnostics)

Merge provided diagnostics.

Parameters:

diagnostics (list[TypeVar(CountResultType, bound= CountResult)]) – Diagnostics to merge.

Return type:

TypeVar(CountResultType, bound= CountResult)

Returns:

Merged diagnostic.

metadata(internal=False)

Diagnostic serializable configuration.

Parameters:

internal (bool) – Whether this diagnostic is an internal diagnostic or not.

Return type:

DiagnosticMeta

Returns:

Diagnostic metadata.

name: str
abstract new_algorithm()

Instantiate the algorithm.

Return type:

BinnedStat

Returns:

Instantiated algorithm.

abstract plot_container(template=None, **kwargs)

Create a PlotContainer for the provided stat data.

Parameters:

  • template (PlotTemplate | None) – Parameters template to use.

  • kwargs – Additional parameters required to generate the container.

Return type:

PlotContainer

Returns:

PlotContainer containing the requested data.

results: DiagResultType
run_algorithm(data, fields)

Run the algorithm on the provided dataset.

Parameters:

  • data (Dataset) – Data on which to apply the algorithm.

  • fields (list[str]) – List of fields for which we want results.

Return type:

TypeVar(BinnedResultType, bound= BinnedResult)

Returns:

Computation result.

start: np.datetime64
stat_selection: ong_clip_t.Clip | None = None
stats: list[StatType]
storage: StoragePropertiesType = None
store(store, group, analyse_type, analyse_date, start, end, mode, fields)

Write the diagnostic results into the provided store.

Parameters:

  • store (DiagnosticStore) – DiagnosticStore used for the storage.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the computed data.

  • end (datetime64) – Ending date for the computed data.

  • mode (StorageMode) – Storage mode to use when writing data.

  • fields (list[str]) – List of all fields name used in the group of this statistic.

store_configuration(analyse_type, analyse_date, group=None)

Returns the list of storage group parameters required to store this diagnostic.

Parameters:

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date representing the set of data used in this analyse. It’s used to determine at which timestamp to store non-temporal diagnostics.

  • group (str | None) – Name of the group to use for storage.

Return type:

list[StorageGroupParams]

Returns:

List of storage group parameters.

store_params: dict[str, Any] | None = None
time_extension: bool = False
update_properties(diag)

Update diagnostic properties with the provided one.

Parameters:

diag (StatisticDiag) – Diagnostic from which to get properties.

property variable: str

Name of the main variable represented by this data.

class casys.computation.diagnostics.BinnedResult(data=<factory>, count=None, index=None, internal_stats=<factory>, stats=<factory>)

Bases: CountResult

Binned diagnostic results.

Parameters:
count: dict[str, ndarray] | None = None
data: dict[StatType, Dataset] = None
filter_count(field, selection)

Filter the count statistic of a field on value where the selection’s field count is greater than zero.

Parameters:

  • field (str) – Field to filter.

  • selection (str) – Field to use as selection.

have_data()

Whether this result contains data or not.

Return type:

bool

Returns:

True if this result has data, false otherwise.

index: ndarray | None = None
internal_stats: dict[str, int | float]
merge_data(result)

Merge provided result’s data into this one. These results are in a finalized state.

Parameters:

result (BinnedResult | None) – Result to merge.

merge_stats(results, simple=False, **kwargs)

Merge provided results into this one. These results are in a non finalized state.

Parameters:
remove_data(field=None, stats=None)

Remove results’ data from the stat of the given field.

Parameters:

  • field (str) – Name of the field

  • stats (list[StatType]) – Statistics to remove.

remove_fields(fields)

Remove provided fields from statistics.

Parameters:

fields (list[str]) – Fields to remove.

select_data(selection, fields, stats)
Parameters:
statistics()

List of statistics computed for this diagnostic.

Return type:

list[StatType]

stats: dict[str, dict[str, ndarray]]
class casys.computation.diagnostics.BinnedSwathDiag(name, store_params=None, *, data_index, data_time, start, end, field_x, field_y=None, indexer=None, time_extension=False, stats=<factory>, stat_selection=None, res_x)

Bases: BinnedDiag

Class representing a non-geographical box statistic.

Parameters:

  • name (str) – Name of the data.

  • computed – True if the data have been computed, false otherwise.

  • is_post_diag – Whether this diagnostic is a post computation diagnostic or not.

  • data_index (str) – Name of the field on which the data is indexed.

  • start (np.datetime64) – Starting date of the input dataset.

  • end (np.datetime64) – Ending date of the input dataset.

  • field_x (Field) – Field used for x axes.

  • field_y (Field | None) – Field used for y axes.

  • stats (list[StatType]) – Statistics included in this Diagnostic.

  • indexer (ong_indexer_t.PassIndexer | None) – Indexer used to read data.

  • time_extension (bool) – Whether time extension is enabled or not.

  • ext_start – Extended starting date required by the computation.

  • ext_end – Ending date for the data required by the computation.

  • group – the name of the group of this diagnostic.

  • stat_selection (ong_clip_t.Clip | None) – Selection clip used to invalidate some bins.

  • res_x (DataResolution) – Min, max and width for the x-axis.

  • store_params (dict[str, Any] | None)

  • data_time (str)

INDEX_USED: ClassVar[str] = 'INDEX_USED'
MERGEABLE_STAT: ClassVar[list[StatType]] = [MIN, MAX, COUNT, MEAN, VAR, STD, PROD, SUM]
RES_TYPE

alias of BinnedResult

SCHEMA: ClassVar[type[mm.Schema]] = None
STORABLE: ClassVar[bool] = True
apply_stats_selection(diagnostic, fields)

Filter diagnostic by applying the statistic selection clip and removing unwanted statistics.

Parameters:

  • diagnostic (TypeVar(BinnedResultType, bound= BinnedResult)) – Diagnostic to filter.

  • fields (list[str]) – List of diagnostic field’s names.

Return type:

TypeVar(BinnedResultType, bound= BinnedResult)

check_stat(stat)

Check whether this data is computed and provided stat is valid. Determines the stat if None is provided.

Parameters:

stat (StatType | None) – Statistic to check.

Return type:

StatType

Returns:

Checked statistic.

Raises:

AltiDataError – If data with the provided name were not computed. If the stat parameter is invalid.

clear_data()

Clear stored data from this Diagnostic data.

compute(data, start, end, fields, diags)

Compute the diagnostic on the provided data.

Parameters:

  • data (Dataset) – The dataset with data to read.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • diags (list[StatisticDiag]) – List of diagnostics included in this computation.

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Dictionary containing the statistic type with its associated data array.

compute_from_store(store, group, analyse_type, analyse_date, start, end, fields, delayed)

Compute diagnostic using stored data. The computation can be done in a delayed way allowing to only load requested data when needed.

Parameters:

  • store (DiagnosticStore) – Store containing diagnostics data.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • delayed (bool) – Flag indicating whether to load from store during computation or at first data request (plot/get_data).

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Computation results.

computed: bool = False
dask_full_compatible: bool = False
data_index: str
data_time: str
diag_data(**kwargs)

Diagnostic data as a dataset.

Parameters:

kwargs – Additional parameters required to retrieve diagnostic data.

Return type:

Dataset

Returns:

Dataset containing this diagnostic data.

end: np.datetime64
property ext_end: datetime64
property ext_start: datetime64
property field: Field

Field of the main variable represented by this data.

field_x: Field
field_y: Field | None = None
property fields: dict[str, Field]

Dictionary of fields used to compute this data.

fill_count_with_nan()

Whether count statistics have to be filled with nan or not.

Return type:

bool

Returns:

True or False.

finalize(result)

Correctly format the diagnostic result.

Parameters:

result (TypeVar(BinnedResultType, bound= BinnedResult)) – Result to finalize.

Return type:

TypeVar(BinnedResultType, bound= BinnedResult)

Returns:

Finalized result.

static grid_2d(res_x, res_y)

Generate a Grid2D from provided parameters.

Parameters:
Return type:

Grid2D

Returns:

Grid2D with provided resolutions.

group: str = None
group_base: str = None
html_elem(style='')

HTML representation of the data container.

Parameters:

style (str) – Style to add to the element.

Return type:

str

Returns:

Html representation of elements of this object.

html_header(span)

HTML representation header of the data container.

Parameters:

span (int) – Span size.

Returns:

Object html header.

property identity: DiagnosticType
indexer: ong_indexer_t.PassIndexer | None = None
initialize()

Diagnostic initialization.

At least the following properties have to be initialized:

  • group_base

  • storage_properties

is_autonomous(dask)

Check whether this diagnostic is autonomous with or without dask enabled.

Parameters:

dask (bool) – Whether dask is enabled or not.

Returns:

Whether this is an autonomous diagnostic or not.

is_dask_compatible(freq)

Check whether this diagnostic can be computed with dask or not.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

Return type:

bool

is_post_diag: bool = False
load_diagnostic(**kwargs)

Ensure requested diagnostic data are loaded.

make_dask_compatible(freq)

Set this container, so it can be used with dask.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

merge(diagnostics)

Merge provided diagnostics.

Parameters:

diagnostics (list[TypeVar(CountResultType, bound= CountResult)]) – Diagnostics to merge.

Return type:

TypeVar(CountResultType, bound= CountResult)

Returns:

Merged diagnostic.

property mergeable: bool
metadata(internal=False)

Diagnostic serializable configuration.

Parameters:

internal (bool) – Whether this diagnostic is an internal diagnostic or not.

Return type:

DiagnosticMeta

Returns:

Diagnostic metadata.

name: str
new_algorithm()

Instantiate the algorithm.

Return type:

BinnedStat

Returns:

Instantiated algorithm.

plot_container(template=None, stat=None)

Create a PlotContainer for the provided stat data.

Parameters:
Return type:

PlotContainer

Returns:

PlotContainer containing the requested data.

res_x: DataResolution
results: DiagResultType
run_algorithm(data, fields)

Run the algorithm on the provided dataset.

Parameters:

  • data (Dataset) – Data on which to apply the algorithm.

  • fields (list[str]) – List of fields for which we want results.

Return type:

TypeVar(BinnedResultType, bound= BinnedResult)

Returns:

Computation result.

start: np.datetime64
stat_selection: ong_clip_t.Clip | None = None
stats: list[StatType]
storage: StoragePropertiesType = None
store(store, group, analyse_type, analyse_date, start, end, mode, fields)

Write the diagnostic results into the provided store.

Parameters:

  • store (DiagnosticStore) – DiagnosticStore used for the storage.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the computed data.

  • end (datetime64) – Ending date for the computed data.

  • mode (StorageMode) – Storage mode to use when writing data.

  • fields (list[str]) – List of all fields name used in the group of this statistic.

store_configuration(analyse_type, analyse_date, group=None)

Returns the list of storage group parameters required to store this diagnostic.

Parameters:

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date representing the set of data used in this analyse. It’s used to determine at which timestamp to store non-temporal diagnostics.

  • group (str | None) – Name of the group to use for storage.

Return type:

list[StorageGroupParams]

Returns:

List of storage group parameters.

store_params: dict[str, Any] | None = None
time_extension: bool = False
update_properties(diag)

Update diagnostic properties with the provided one.

Parameters:

diag (StatisticDiag) – Diagnostic from which to get properties.

property variable: str

Name of the main variable represented by this data.

class casys.computation.diagnostics.CountDiag(name, store_params=None, *, data_index, data_time, start, end, field_x, field_y=None, indexer=None, time_extension=False, stats=<factory>)

Bases: StatisticDiag, Generic[CountResultType, StoragePropertiesType], ABC

Count related diagnostics.

Parameters:

  • name (str) – Name of the data.

  • computed – True if the data have been computed, false otherwise.

  • is_post_diag – Whether this diagnostic is a post computation diagnostic or not.

  • data_index (str) – Name of the field on which the data is indexed.

  • start (np.datetime64) – Starting date of the input dataset.

  • end (np.datetime64) – Ending date of the input dataset.

  • field_x (Field) – Field used for x axes.

  • field_y (Field | None) – Field used for y axes.

  • stats (list[StatType]) – Statistics included in this Diagnostic.

  • indexer (ong_indexer_t.PassIndexer | None) – Indexer used to read data.

  • time_extension (bool) – Whether time extension is enabled or not.

  • ext_start – Extended starting date required by the computation.

  • ext_end – Ending date for the data required by the computation.

  • group – the name of the group of this diagnostic.

  • store_params (dict[str, Any] | None)

  • data_time (str)

MERGEABLE_STAT: ClassVar[list[StatType]] = [MIN, MAX, COUNT, MEAN, VAR, STD, PROD, SUM]
RES_TYPE

alias of CountResult

SCHEMA: ClassVar[type[mm.Schema]] = None
STORABLE: ClassVar[bool] = True
apply_stats_selection(diagnostic, fields)

Filter diagnostic by applying the statistic selection clip and removing unwanted statistics.

Parameters:

  • diagnostic (TypeVar(DiagResultType, bound= DiagResult)) – Diagnostic to filter.

  • fields (list[str]) – List of diagnostic field’s names.

Return type:

TypeVar(DiagResultType, bound= DiagResult)

check_stat(stat)

Check whether this data is computed and provided stat is valid. Determines the stat if None is provided.

Parameters:

stat (StatType | None) – Statistic to check.

Return type:

StatType

Returns:

Checked statistic.

Raises:

AltiDataError – If data with the provided name were not computed. If the stat parameter is invalid.

clear_data()

Clear stored data from this Diagnostic data.

compute(data, start, end, fields, diags)

Compute the diagnostic on the provided data.

Parameters:

  • data (Dataset) – The dataset with data to read.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • diags (list[StatisticDiag]) – List of diagnostics included in this computation.

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Dictionary containing the statistic type with its associated data array.

compute_from_store(store, group, analyse_type, analyse_date, start, end, fields, delayed)

Compute diagnostic using stored data. The computation can be done in a delayed way allowing to only load requested data when needed.

Parameters:

  • store (DiagnosticStore) – Store containing diagnostics data.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • delayed (bool) – Flag indicating whether to load from store during computation or at first data request (plot/get_data).

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Computation results.

computed: bool = False
dask_full_compatible: bool = False
data_index: str
data_time: str
diag_data(**kwargs)

Diagnostic data as a dataset.

Parameters:

kwargs – Additional parameters required to retrieve diagnostic data.

Return type:

Dataset

Returns:

Dataset containing this diagnostic data.

end: np.datetime64
property ext_end: datetime64
property ext_start: datetime64
abstract property field: Field

Field of the main variable represented by this data.

field_x: Field
field_y: Field | None = None
abstract property fields: dict[str, Field]

Dictionary of fields used to compute this data.

finalize(result)

Correctly format the diagnostic result.

Parameters:

result (TypeVar(CountResultType, bound= CountResult)) – Result to finalize.

Return type:

TypeVar(CountResultType, bound= CountResult)

Returns:

Finalized result.

static grid_2d(res_x, res_y)

Generate a Grid2D from provided parameters.

Parameters:
Return type:

Grid2D

Returns:

Grid2D with provided resolutions.

group: str = None
group_base: str = None
abstract html_elem(style='')

HTML representation of the data container.

Parameters:

style (str) – Style to add to the element.

Return type:

str

Returns:

Html representation of elements of this object.

abstract html_header(span)

HTML representation header of the data container.

Parameters:

span (int) – Span size.

Return type:

str

Returns:

Object html header.

property identity: DiagnosticType
indexer: ong_indexer_t.PassIndexer | None = None
abstract initialize()

Diagnostic initialization.

At least the following properties have to be initialized:

  • group_base

  • storage_properties

is_autonomous(dask)

Check whether this diagnostic is autonomous with or without dask enabled.

Parameters:

dask (bool) – Whether dask is enabled or not.

Returns:

Whether this is an autonomous diagnostic or not.

is_dask_compatible(freq)

Check whether this diagnostic can be computed with dask or not.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

Return type:

bool

is_post_diag: bool = False
load_diagnostic(**kwargs)

Ensure requested diagnostic data are loaded.

make_dask_compatible(freq)

Set this container, so it can be used with dask.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

merge(diagnostics)

Merge provided diagnostics.

Parameters:

diagnostics (list[TypeVar(CountResultType, bound= CountResult)]) – Diagnostics to merge.

Return type:

TypeVar(CountResultType, bound= CountResult)

Returns:

Merged diagnostic.

metadata(internal=False)

Diagnostic serializable configuration.

Parameters:

internal (bool) – Whether this diagnostic is an internal diagnostic or not.

Return type:

DiagnosticMeta

Returns:

Diagnostic metadata.

name: str
abstract new_algorithm()

Instantiate the algorithm.

Return type:

Dispersion

Returns:

Instantiated algorithm.

abstract plot_container(template=None, **kwargs)

Create a PlotContainer for the provided stat data.

Parameters:

  • template (PlotTemplate | None) – Parameters template to use.

  • kwargs – Additional parameters required to generate the container.

Return type:

PlotContainer

Returns:

PlotContainer containing the requested data.

results: DiagResultType
start: np.datetime64
stats: list[StatType]
storage: StoragePropertiesType = None
store(store, group, analyse_type, analyse_date, start, end, mode, fields)

Write the diagnostic results into the provided store.

Parameters:

  • store (DiagnosticStore) – DiagnosticStore used for the storage.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the computed data.

  • end (datetime64) – Ending date for the computed data.

  • mode (StorageMode) – Storage mode to use when writing data.

  • fields (list[str]) – List of all fields name used in the group of this statistic.

store_configuration(analyse_type, analyse_date, group=None)

Returns the list of storage group parameters required to store this diagnostic.

Parameters:

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date representing the set of data used in this analyse. It’s used to determine at which timestamp to store non-temporal diagnostics.

  • group (str | None) – Name of the group to use for storage.

Return type:

list[StorageGroupParams]

Returns:

List of storage group parameters.

store_params: dict[str, Any] | None = None
time_extension: bool = False
update_properties(diag)

Update diagnostic properties with the provided one.

Parameters:

diag (StatisticDiag) – Diagnostic from which to get properties.

property variable: str

Name of the main variable represented by this data.

class casys.computation.diagnostics.CountResult(data=None, count=None, index=None, internal_stats=<factory>)

Bases: StatDiagResult

Count diagnostic results.

Parameters:
count: dict[str, ndarray] | None = None
data: Dataset | None = None
have_data()

Whether this result contains data or not.

Return type:

bool

Returns:

True if this result has data, false otherwise.

index: dict[str, ndarray] | None = None
internal_stats: dict[str, int | float]
merge_data(result)

Merge provided result’s data into this one. These results are in a finalized state.

Parameters:

result (CountResult | None) – Result to merge.

merge_stats(results, **kwargs)

Merge provided results into this one. These results are in a non finalized state.

Parameters:

results (list[CountResult]) – Results to merge.

remove_data(field=None, stats=None)

Remove results’ data from the stat of the given field.

Parameters:

  • field (str) – Name of the field

  • stats (list[StatType]) – Statistics to remove.

statistics()

List of statistics computed for this diagnostic.

Return type:

list[StatType]

class casys.computation.diagnostics.CrossoverDiag(name, store_params=None, *, data_index, data_time, start, end, field_x, field_y=None, indexer=None, time_extension=False, stats=<factory>, field_z, time, max_time_difference, interp_mode=None, spline_half_window_size=None, jump_threshold=None, data, res_lon, res_lat, box_selection, geobox_stats, temporal_freq, temporal_stats, reader, cro_builder=None, computation_split=None)

Bases: StatisticDiag[CrossoverResult, CrossoverStorageProperties]

Class representing a crossover statistic.

Parameters:

  • name (str) – Name of the data.

  • computed – True if the data have been computed, false otherwise.

  • is_post_diag – Whether this diagnostic is a post computation diagnostic or not.

  • data_index (str) – Name of the field on which the data is indexed.

  • start (np.datetime64) – Starting date of the input dataset.

  • end (np.datetime64) – Ending date of the input dataset.

  • field_x (Field) – Field used for x axes.

  • field_y (Field | None) – Field used for y axes.

  • stats (list[StatType]) – Statistics included in this Diagnostic.

  • indexer (ong_indexer_t.PassIndexer | None) – Indexer used to read data.

  • time_extension (bool) – Whether time extension is enabled or not.

  • ext_start – Extended starting date required by the computation.

  • ext_end – Ending date for the data required by the computation.

  • group – the name of the group of this diagnostic.

  • field_z (Field) – Field used for z data.

  • time (Field) – Field used for the time.

  • data (casys_t.NadirData | None) – External data (CommonData) to compute crossovers with. This option is used to compute multi-missions crossovers.

  • max_time_difference (np.timedelta64) – Maximum delta of time between the two arcs.

  • interp_mode (str | None) – Interpolation mode used to compute the field value at the crossover position.

  • spline_half_window_size (int | None) – Half window size of the spline.

  • jump_threshold (float | None) – This parameter sets the tolerance level of the jumps (holes) in the input data.

  • res_lon (DataResolution) – Min, max and width for the longitude axis.

  • res_lat (DataResolution) – Min, max and width for the latitude axis.

  • box_selection (Field | None) – Field used as selection for computation of the count statistic. Box in which the box_selection field does not contain any data will be set to NaN instead of 0.

  • geobox_stats (list[StatType]) – Statistics included in the geobox diagnostic.

  • temporal_freq (list[FrequencyHandler]) – List of temporal statistics frequencies to compute.

  • temporal_stats (list[StatType]) – Statistics included in the temporal diagnostic.

  • store_params (dict[str, Any] | None)

  • data_time (str)

  • reader (CasysReader)

  • cro_builder (CrossoversBuilderContainer | None)

  • computation_split (FrequencyHandler | None)

MERGEABLE_STAT: ClassVar[list[StatType]] = [MIN, MAX, COUNT, MEAN, VAR, STD, PROD, SUM]
RES_TYPE

alias of CrossoverResult

SCHEMA: ClassVar[type[mm.Schema]] = None
STORABLE: ClassVar[bool] = True
apply_stats_selection(diagnostic, fields)

Filter diagnostic by applying the statistic selection clip and removing unwanted statistics.

Parameters:

  • diagnostic (TypeVar(DiagResultType, bound= DiagResult)) – Diagnostic to filter.

  • fields (list[str]) – List of diagnostic field’s names.

Return type:

TypeVar(DiagResultType, bound= DiagResult)

box_diag: GeoBoxDiag
box_selection: Field | None
check_stat(stat)

Check whether this data is computed and provided stat is valid. Determines the stat if None is provided.

Parameters:

stat (StatType | None) – Statistic to check.

Return type:

StatType

Returns:

Checked statistic.

Raises:

AltiDataError – If data with the provided name were not computed. If the stat parameter is invalid.

clear_data()

Clear stored data from this Diagnostic data.

computation_split: FrequencyHandler | None = None
compute(data, start, end, fields, diags)

Compute the diagnostic on the provided data.

Parameters:

  • data (Dataset) – The dataset with data to read.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • diags (list[StatisticDiag]) – List of diagnostics included in this computation.

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Dictionary containing the statistic type with its associated data array.

compute_from_store(store, group, analyse_type, analyse_date, start, end, fields, delayed)

Compute diagnostic using stored data. The computation can be done in a delayed way allowing to only load requested data when needed.

Parameters:

  • store (DiagnosticStore) – Store containing diagnostics data.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • delayed (bool) – Flag indicating whether to load from store during computation or at first data request (plot/get_data).

Return type:

CrossoverResult

Returns:

Computation results.

computed: bool = False
cro_builder: CrossoversBuilderContainer | None = None
dask_full_compatible: bool = False
data: casys_t.NadirData | None
data_index: str
data_time: str
property delta_time: str
diag_data(**kwargs)

Diagnostic data as a dataset.

Parameters:

kwargs – Additional parameters required to retrieve diagnostic data.

Return type:

Dataset

Returns:

Dataset containing this diagnostic data.

end: np.datetime64
property ext_end: datetime64
property ext_start: datetime64
property field: Field

Field of the main variable represented by this data.

field_x: Field
field_y: Field | None = None
field_z: Field
property fields: dict[str, Field]

Dictionary of fields used to compute this data.

finalize(result)

Correctly format the diagnostic result.

Parameters:

result (CrossoverResult) – Result to finalize.

Return type:

CrossoverResult

Returns:

Finalized result.

geobox_stats: list[StatType]
group: str = None
group_base: str = None
html_elem(style='')

HTML representation of the data container.

Parameters:

style (str) – Style to add to the element.

Return type:

str

Returns:

Html representation of elements of this object.

html_header(span)

HTML representation header of the data container.

Parameters:

span (int) – Span size.

Returns:

Object html header.

property identity: DiagnosticType
indexer: ong_indexer_t.PassIndexer | None = None
initialize()

Diagnostic initialization.

At least the following properties have to be initialized:

  • group_base

  • storage_properties

interp_mode: str | None = None
is_autonomous(dask)

Check whether this diagnostic is autonomous with or without dask enabled.

Parameters:

dask (bool) – Whether dask is enabled or not.

Returns:

Whether this is an autonomous diagnostic or not.

is_dask_compatible(freq)

Check whether this diagnostic can be computed with dask or not.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

Return type:

bool

is_post_diag: bool = False
jump_threshold: float | None = None
load_diagnostic(**kwargs)

Ensure requested diagnostic data are loaded.

make_dask_compatible(freq)

Set this container, so it can be used with dask.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

max_time_difference: np.timedelta64
merge(diagnostics)

Merge provided diagnostics.

Parameters:

diagnostics (list[CrossoverResult]) – Diagnostics to merge.

Return type:

CrossoverResult

Returns:

Merged diagnostic.

metadata(internal=False)

Diagnostic serializable configuration.

Parameters:

internal (bool) – Whether this diagnostic is an internal diagnostic or not.

Return type:

DiagnosticMeta

Returns:

Diagnostic metadata.

name: str
plot_container(stat=None, template=None, delta=None, freq=None)

Create a PlotContainer for the provided stat data.

Parameters:

  • template (PlotTemplate | None) – Parameters template to use.

  • stat (StatType | None) – Statistic included in this data.

  • delta (str | None) – Difference data to access: “field” or “time”.

  • freq (str | FreqType | None) – Frequency of the requested statistic.

Return type:

PlotContainer

Returns:

PlotContainer containing the requested data.

reader: CasysReader
res_lat: DataResolution
res_lon: DataResolution
results: DiagResultType
set_cro_builder(data)

Set the crossovers builder and compute intersections.

Parameters:

data (Dataset) – Data for which to compute intersections.

spline_half_window_size: int | None = None
start: np.datetime64
stats: list[StatType]
storage: StoragePropertiesType = None
store(store, group, analyse_type, analyse_date, start, end, mode, fields)

Write the diagnostic results into the provided store.

Parameters:

  • store (DiagnosticStore) – DiagnosticStore used for the storage.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the computed data.

  • end (datetime64) – Ending date for the computed data.

  • mode (StorageMode) – Storage mode to use when writing data.

  • fields (list[str]) – List of all fields name used in the group of this statistic.

store_configuration(analyse_type, analyse_date, group=None)

Returns the list of storage group parameters required to store this diagnostic.

Parameters:

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date representing the set of data used in this analyse. It’s used to determine at which timestamp to store non-temporal diagnostics.

  • group (str | None) – Name of the group to use for storage.

Return type:

list[StorageGroupParams]

Returns:

List of storage group parameters.

store_params: dict[str, Any] | None = None
temp_diags: dict[str, TemporalDiag]
temporal_freq: list[FrequencyHandler]
temporal_stats: list[StatType]
time: Field
time_extension: bool = False
update_properties(diag)

Update diagnostic properties with the provided one.

Parameters:

diag (StatisticDiag) – Diagnostic from which to get properties.

property variable: str

Name of the main variable represented by this data.

class casys.computation.diagnostics.CrossoverResult(delta=None, geobox=None, temporal=<factory>, sections=<factory>, locations=None, tmp_geobox=None, tmp_temporal=<factory>)

Bases: StatDiagResult

Crossover diagnostic results.

Parameters:
property data: dict[DiagnosticType, Dataset] | dict[DiagnosticType, dict[StatType, Dataset]] | dict[DiagnosticType, dict[str, dict[StatType, Dataset]]]
delta: Dataset | None = None
geobox: BinnedResult | None = None
have_data()

Whether this result contains data or not.

Return type:

bool

Returns:

True if this result has data, false otherwise.

locations: ndarray | None = None
merge_data(result)

Merge provided result’s data into this one. These results are in a finalized state.

Parameters:

result (CountResult | None) – Result to merge.

merge_stats(results, **kwargs)

Merge provided results into this one. These results are in a non finalized state.

Parameters:

results (list[DiagResult]) – Results to merge.

remove_data(field=None, stats=None)

Remove results’ data from the stat of the given field.

Parameters:

  • field (str) – Name of the field

  • stats (list[StatType]) – Statistics to remove.

remove_fields(fields)

Remove provided fields from statistics.

Parameters:

fields (list[str]) – Fields to remove.

sections: dict[str, Analysis]
set_geobox(results)
Parameters:

results (BinnedResult)

set_temporal(freq, results)
Parameters:
statistics()

List of statistics computed for this diagnostic.

Return type:

list[StatType]

temporal: dict[str, BinnedResult]
tmp_geobox: BinnedResult | None = None
tmp_temporal: dict[str, BinnedResult]
class casys.computation.diagnostics.CrossoverSwathDiag(name, store_params=None, *, data_index, data_time, start, end, field_x, field_y=None, indexer=None, time_extension=False, stats=<factory>, field_z, time, max_time_difference, interp_mode=None, spline_half_window_size=None, jump_threshold=None, data, res_lon, res_lat, box_selection, geobox_stats, temporal_freq, temporal_stats, reader, cro_builder=None, computation_split=None, zdata, pass_multi_intersect, cartesian_plane, crossover_table, diamond_relocation, diamond_reduction)

Bases: CrossoverDiag

Class representing a Swath crossover statistic.

Parameters:

  • name (str) – Name of the data.

  • computed – True if the data have been computed, false otherwise.

  • is_post_diag – Whether this diagnostic is a post computation diagnostic or not.

  • data_index (str) – Name of the field on which the data is indexed.

  • start (np.datetime64) – Starting date of the input dataset.

  • end (np.datetime64) – Ending date of the input dataset.

  • field_x (Field) – Field used for x axes.

  • field_y (Field | None) – Field used for y axes.

  • stats (list[StatType]) – Statistics included in this Diagnostic.

  • indexer (ong_indexer_t.PassIndexer | None) – Indexer used to read data.

  • time_extension (bool) – Whether time extension is enabled or not.

  • ext_start – Extended starting date required by the computation.

  • ext_end – Ending date for the data required by the computation.

  • group – the name of the group of this diagnostic.

  • field_z (Field) – Field used for z data.

  • time (Field) – Field used for the time.

  • data (casys_t.NadirData | None) – External data (CommonData) to compute crossovers with. This option is used to compute multi-missions crossovers.

  • max_time_difference (np.timedelta64) – Maximum delta of time between the two arcs.

  • res_lon (DataResolution) – Min, max and width for the longitude axis.

  • res_lat (DataResolution) – Min, max and width for the latitude axis.

  • box_selection (Field | None) – Field used as selection for computation of the count statistic. Box in which the box_selection field does not contain any data will be set to NaN instead of 0.

  • geobox_stats (list[StatType]) – Statistics included in the geobox diagnostic.

  • temporal_freq (list[FrequencyHandler]) – List of temporal statistics frequencies to compute.

  • geobox_stats – Statistics included in the temporal diagnostic.

  • zdata (zc_t.Dataset) – Data (zcollection dataset) used to compute crossovers.

  • pass_multi_intersect (bool) – Whether to look for multiple intersections between a set of 2 passes or not.

  • cartesian_plane (bool) – Flag determining the plane used for crossovers computation. If True, the crossover is calculated in the cartesian plane. If False, the crossover is calculated in the spherical plane. Defaults to True.

  • crossover_table (set[tuple[int, int]]) – The table of possible combinations of crossovers between the different passes. If this table is not defined, all crossovers between odd and even passes will be calculated.

  • diamond_relocation (bool) – Flag determining data relocation to nadir crossover coordinates for the statistics computation. If True, data relocation is performed.

  • diamond_reduction (StatType) – Statistic type used to reduce the data on the crossover diamond.

  • store_params (dict[str, Any] | None)

  • data_time (str)

  • interp_mode (str | None)

  • spline_half_window_size (int | None)

  • jump_threshold (float | None)

  • temporal_stats (list[StatType])

  • reader (CasysReader)

  • cro_builder (CrossoversBuilderContainer | None)

  • computation_split (FrequencyHandler | None)

MERGEABLE_STAT: ClassVar[list[StatType]] = [MIN, MAX, COUNT, MEAN, VAR, STD, PROD, SUM]
RES_TYPE

alias of CrossoverResult

SCHEMA: ClassVar[type[mm.Schema]] = None
STORABLE: ClassVar[bool] = True
apply_stats_selection(diagnostic, fields)

Filter diagnostic by applying the statistic selection clip and removing unwanted statistics.

Parameters:

  • diagnostic (TypeVar(DiagResultType, bound= DiagResult)) – Diagnostic to filter.

  • fields (list[str]) – List of diagnostic field’s names.

Return type:

TypeVar(DiagResultType, bound= DiagResult)

box_diag: GeoBoxDiag
box_selection: Field | None
cartesian_plane: bool
check_stat(stat)

Check whether this data is computed and provided stat is valid. Determines the stat if None is provided.

Parameters:

stat (StatType | None) – Statistic to check.

Return type:

StatType

Returns:

Checked statistic.

Raises:

AltiDataError – If data with the provided name were not computed. If the stat parameter is invalid.

clear_data()

Clear stored data from this Diagnostic data.

computation_split: FrequencyHandler | None = None
compute(data, start, end, fields, diags)

Compute the diagnostic on the provided data.

Parameters:

  • data (Dataset) – The dataset with data to read.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • diags (list[StatisticDiag]) – List of diagnostics included in this computation.

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Dictionary containing the statistic type with its associated data array.

compute_from_store(store, group, analyse_type, analyse_date, start, end, fields, delayed)

Compute diagnostic using stored data. The computation can be done in a delayed way allowing to only load requested data when needed.

Parameters:

  • store (DiagnosticStore) – Store containing diagnostics data.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • delayed (bool) – Flag indicating whether to load from store during computation or at first data request (plot/get_data).

Return type:

CrossoverResult

Returns:

Computation results.

computed: bool = False
cro_builder: CrossoversBuilderContainer | None = None
crossover_table: set[tuple[int, int]]
dask_full_compatible: bool = False
data: casys_t.NadirData | None
data_index: str
data_time: str
property delta_time: str
diag_data(**kwargs)

Diagnostic data as a dataset.

Parameters:

kwargs – Additional parameters required to retrieve diagnostic data.

Return type:

Dataset

Returns:

Dataset containing this diagnostic data.

diamond_reduction: StatType
diamond_relocation: bool
end: np.datetime64
property ext_end: datetime64
property ext_start: datetime64
property field: Field

Field of the main variable represented by this data.

field_x: Field
field_y: Field | None = None
field_z: Field
property fields: dict[str, Field]

Dictionary of fields used to compute this data.

finalize(result)

Correctly format the diagnostic result.

Parameters:

result (CrossoverResult) – Result to finalize.

Return type:

CrossoverResult

Returns:

Finalized result.

geobox_stats: list[StatType]
group: str = None
group_base: str = None
html_elem(style='')

HTML representation of the data container.

Parameters:

style (str) – Style to add to the element.

Return type:

str

Returns:

Html representation of elements of this object.

html_header(span)

HTML representation header of the data container.

Parameters:

span (int) – Span size.

Returns:

Object html header.

property identity: DiagnosticType
indexer: ong_indexer_t.PassIndexer | None = None
initialize()

Diagnostic initialization.

At least the following properties have to be initialized:

  • group_base

  • storage_properties

interp_mode: str | None = None
is_autonomous(dask)

Check whether this diagnostic is autonomous with or without dask enabled.

Parameters:

dask (bool) – Whether dask is enabled or not.

Returns:

Whether this is an autonomous diagnostic or not.

is_dask_compatible(freq)

Check whether this diagnostic can be computed with dask or not.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

Return type:

bool

is_post_diag: bool = False
jump_threshold: float | None = None
load_diagnostic(**kwargs)

Ensure requested diagnostic data are loaded.

make_dask_compatible(freq)

Set this container, so it can be used with dask.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

max_time_difference: np.timedelta64
merge(diagnostics)

Merge provided diagnostics.

Parameters:

diagnostics (list[CrossoverResult]) – Diagnostics to merge.

Return type:

CrossoverResult

Returns:

Merged diagnostic.

metadata(internal=False)

Diagnostic serializable configuration.

Parameters:

internal (bool) – Whether this diagnostic is an internal diagnostic or not.

Return type:

DiagnosticMeta

Returns:

Diagnostic metadata.

name: str
pass_multi_intersect: bool
plot_container(stat=None, template=None, delta=None, freq=None)

Create a PlotContainer for the provided stat data.

Parameters:

  • template (PlotTemplate | None) – Parameters template to use.

  • stat (StatType | None) – Statistic included in this data.

  • delta (str | None) – Difference data to access: “field” or “time”.

  • freq (str | FreqType | None) – Frequency of the requested statistic.

Return type:

PlotContainer

Returns:

PlotContainer containing the requested data.

reader: CasysReader
res_lat: DataResolution
res_lon: DataResolution
results: DiagResultType
set_cro_builder(data)

Set the crossovers builder and compute intersections.

Parameters:

data (Dataset) – Data for which to compute intersections.

spline_half_window_size: int | None = None
start: np.datetime64
stats: list[StatType]
storage: StoragePropertiesType = None
store(store, group, analyse_type, analyse_date, start, end, mode, fields)

Write the diagnostic results into the provided store.

Parameters:

  • store (DiagnosticStore) – DiagnosticStore used for the storage.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the computed data.

  • end (datetime64) – Ending date for the computed data.

  • mode (StorageMode) – Storage mode to use when writing data.

  • fields (list[str]) – List of all fields name used in the group of this statistic.

store_configuration(analyse_type, analyse_date, group=None)

Returns the list of storage group parameters required to store this diagnostic.

Parameters:

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date representing the set of data used in this analyse. It’s used to determine at which timestamp to store non-temporal diagnostics.

  • group (str | None) – Name of the group to use for storage.

Return type:

list[StorageGroupParams]

Returns:

List of storage group parameters.

store_params: dict[str, Any] | None = None
temp_diags: dict[str, TemporalDiag]
temporal_freq: list[FrequencyHandler]
temporal_stats: list[StatType]
time: Field
time_extension: bool = False
update_properties(diag)

Update diagnostic properties with the provided one.

Parameters:

diag (StatisticDiag) – Diagnostic from which to get properties.

property variable: str

Name of the main variable represented by this data.

zdata: zc_t.Dataset
class casys.computation.diagnostics.CrossoversBuilderContainer

Bases: object

Shared CrossoversBuilder for all crossovers of a NadirData instance.

builder: CrossoversBuilder | None
is_set()

Whether this builder is set or not.

Return type:

bool

class casys.computation.diagnostics.DiagResult

Bases: ABC

Diagnostics results base class.

abstract have_data()

Whether this result contains data or not.

Return type:

bool

Returns:

True if this result has data, false otherwise.

abstract remove_data(field=None, stats=None)

Remove results’ data from the stat of the given field.

Parameters:

  • field (str) – Name of the field

  • stats (list[StatType]) – Statistics to remove.

abstract statistics()

List of statistics computed for this diagnostic.

Return type:

list[StatType]

class casys.computation.diagnostics.Diagnostic(name, store_params=None, *, data_index, data_time, start, end, field_x, field_y=None, indexer=None, time_extension=False)

Bases: BaseDiagnostic[DiagResultType, StoragePropertiesType], ABC

Abstract class defining the common structure of a diagnostic: computation, results, storage, schema and visualization, and related attributes.

Parameters:

  • name (str) – Name of the data.

  • computed – True if the data have been computed, false otherwise.

  • is_post_diag – Whether this diagnostic is a post computation diagnostic or not.

  • data_index (str) – Name of the field on which the data is indexed.

  • start (datetime64) – Starting date of the input dataset.

  • end (datetime64) – Ending date of the input dataset.

  • field_x (Field) – Field used for x axes.

  • field_y (Field | None) – Field used for y axes.

  • stats – Statistics included in this Diagnostic.

  • indexer (PassIndexer | None) – Indexer used to read data.

  • time_extension (bool) – Whether time extension is enabled or not.

  • ext_start – Extended starting date required by the computation.

  • ext_end – Ending date for the data required by the computation.

  • store_params (dict[str, Any] | None)

  • data_time (str)

RES_TYPE: ClassVar[type[DiagResult]] = None
SCHEMA: ClassVar[type[Schema]] = None
STORABLE: ClassVar[bool]
check_stat(stat)

Check whether this data is computed and provided stat is valid. Determines the stat if None is provided.

Parameters:

stat (StatType | None) – Statistic to check.

Return type:

StatType

Returns:

Checked statistic.

Raises:

AltiDataError – If data with the provided name were not computed. If the stat parameter is invalid.

clear_data()

Clear stored data from this Diagnostic data.

compute_from_store(store, group, analyse_type, analyse_date, start, end, fields, delayed)

Compute diagnostic using stored data. The computation can be done in a delayed way allowing to only load requested data when needed.

Parameters:

  • store (DiagnosticStore) – Store containing diagnostics data.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • delayed (bool) – Flag indicating whether to load from store during computation or at first data request (plot/get_data).

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Computation results.

computed: bool = False
dask_full_compatible: bool = False
data_index: str
data_time: str
diag_data(**kwargs)

Diagnostic data as a dataset.

Parameters:

kwargs – Additional parameters required to retrieve diagnostic data.

Return type:

Dataset

Returns:

Dataset containing this diagnostic data.

end: datetime64
property ext_end: datetime64
property ext_start: datetime64
abstract property field: Field

Field of the main variable represented by this data.

field_x: Field
field_y: Field | None = None
abstract property fields: dict[str, Field]

Dictionary of fields used to compute this data.

group: str = None
group_base: str = None
abstract html_elem(style='')

HTML representation of the data container.

Parameters:

style (str) – Style to add to the element.

Return type:

str

Returns:

Html representation of elements of this object.

abstract html_header(span)

HTML representation header of the data container.

Parameters:

span (int) – Span size.

Return type:

str

Returns:

Object html header.

property identity: DiagnosticType
indexer: PassIndexer | None = None
abstract initialize()

Diagnostic initialization.

At least the following properties have to be initialized:

  • group_base

  • storage_properties

is_autonomous(dask)

Check whether this diagnostic is autonomous with or without dask enabled.

Parameters:

dask (bool) – Whether dask is enabled or not.

Returns:

Whether this is an autonomous diagnostic or not.

abstract is_dask_compatible(freq)

Check whether this diagnostic can be computed with dask or not.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

Return type:

bool

is_post_diag: bool = False
load_diagnostic(**kwargs)

Ensure requested diagnostic data are loaded.

make_dask_compatible(freq)

Set this container, so it can be used with dask.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

metadata(internal=False)

Diagnostic serializable configuration.

Parameters:

internal (bool) – Whether this diagnostic is an internal diagnostic or not.

Return type:

DiagnosticMeta

Returns:

Diagnostic metadata.

name: str
abstract plot_container(template=None, **kwargs)

Create a PlotContainer for the provided stat data.

Parameters:

  • template (PlotTemplate | None) – Parameters template to use.

  • kwargs – Additional parameters required to generate the container.

Return type:

PlotContainer

Returns:

PlotContainer containing the requested data.

results: TypeVar(DiagResultType, bound= DiagResult)
start: datetime64
stats: list[StatType]
storage: TypeVar(StoragePropertiesType, bound= StorageProperties) = None
store(store, group, analyse_type, analyse_date, start, end, mode, fields)

Write the diagnostic results into the provided store.

Parameters:

  • store (DiagnosticStore) – DiagnosticStore used for the storage.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the computed data.

  • end (datetime64) – Ending date for the computed data.

  • mode (StorageMode) – Storage mode to use when writing data.

  • fields (list[str]) – List of all fields name used in the group of this statistic.

store_configuration(analyse_type, analyse_date, group=None)

Returns the list of storage group parameters required to store this diagnostic.

Parameters:

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date representing the set of data used in this analyse. It’s used to determine at which timestamp to store non-temporal diagnostics.

  • group (str | None) – Name of the group to use for storage.

Return type:

list[StorageGroupParams]

Returns:

List of storage group parameters.

store_params: dict[str, Any] | None = None
time_extension: bool = False
property variable: str

Name of the main variable represented by this data.

class casys.computation.diagnostics.DiagnosticsGroup(name)

Bases: object

Group of diagnostics that can be computed together.

Parameters:

  • name (str) – Group name.

  • diags – List of the diagnostics.

  • fields – Diagnostics fields.

add_diagnostic(diag)

Add a diagnostic to this group.

Parameters:

diag (StatisticDiag) – Diagnostic to add.

compute_from_store(store, analyse_type, analyse_date, start, end, group, delayed)

Compute diagnostics from this group using stored data.

Parameters:

  • store (DiagnosticStore) – Store with the diagnostic data to read.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • delayed (bool) – Flag indicating whether to load from store during computation or at first data request (plot/get_data).

compute_group(data, start, end)

Compute diagnostics from this group.

Parameters:

  • data (Dataset) – Input data to use for computation.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

diags: list[StatisticDiag]
fields: dict[str, Field]
finalize_computation(diagnostics, diagnostic)

Finalize group’s computation:

  • Merge provided group’s data

  • Filter statistics based on their stat_selection criteria

  • Remove unwanted statistics from results

  • Merge new results with existing ones

  • Set group’s diagnostics status

Parameters:
property main_diagnostic: StatisticDiag | None

First diagnostic of the group.

name: str
store(store, analyse_type, analyse_date, start, end, mode)

Write diagnostics’ from this group into the provided store.

Parameters:

  • store (DiagnosticStore | str) – DiagnosticStore used for the storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (Union[datetime64, Timestamp, datetime, str, DateHandler]) – Date used to reference stored data.

  • start (datetime64) – Starting date for the computed data.

  • end (datetime64) – Ending date for the computed data.

  • mode (StorageMode) – Storage mode to use when writing data.

update_diags(diag)

Update current diagnostics group properties with the provided one.

Parameters:

diag (StatisticDiag) – Diagnostic from which to get properties.

class casys.computation.diagnostics.EditingDiag(name, store_params=None, *, data_index, data_time, start, end, field_x, field_y=None, indexer=None, time_extension=False, field_z, time, params)

Bases: RawGenerator

Class representing an editing.

Parameters:

  • name (str) – Name of the data.

  • computed – True if the data have been computed, false otherwise.

  • is_post_diag – Whether this diagnostic is a post computation diagnostic or not.

  • data_index (str) – Name of the field on which the data is indexed.

  • start (np.datetime64) – Starting date of the input dataset.

  • end (np.datetime64) – Ending date of the input dataset.

  • field_x (Field) – Field used for x axes.

  • field_y (Field | None) – Field used for y axes.

  • stats – Statistics included in this Diagnostic.

  • indexer (ong_indexer_t.PassIndexer | None) – Indexer used to read data.

  • time_extension (bool) – Whether time extension is enabled or not.

  • ext_start – Extended starting date required by the computation.

  • ext_end – Ending date for the data required by the computation.

  • field_z (Field) – Field used for z axes.

  • time (Field) – Field used for the time.

  • params (list[ong_edit_t.EditingComponent]) – Editing components.

  • store_params (dict[str, Any] | None)

  • data_time (str)

RES_TYPE

alias of RawDataResult

SCHEMA: ClassVar[type[mm.Schema]] = None
STORABLE: ClassVar[bool] = False
check_stat(stat)

Check whether this data is computed and provided stat is valid. Determines the stat if None is provided.

Parameters:

stat (StatType | None) – Statistic to check.

Return type:

StatType

Returns:

Checked statistic.

Raises:

AltiDataError – If data with the provided name were not computed. If the stat parameter is invalid.

clear_data()

Clear stored data from this Diagnostic data.

compute_data(data)

Read data and compute statistics.

Parameters:

data (Dataset) – The dataset with data to read.

Return type:

Dataset

Returns:

Dataset including the newly computed data.

compute_from_store(**kwargs)

Compute diagnostic using stored data. The computation can be done in a delayed way allowing to only load requested data when needed.

Parameters:

  • store – Store containing diagnostics data.

  • group – Name of the group to use for storage.

  • analyse_type – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date – Date used to reference stored data.

  • start – Starting date for the data to compute.

  • end – Ending date for the data to compute.

  • fields – List of all fields to compute with this diagnostic properties.

  • delayed – Flag indicating whether to load from store during computation or at first data request (plot/get_data).

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Computation results.

computed: bool = False
dask_full_compatible: bool = False
data_index: str
data_time: str
diag_data(**kwargs)

Diagnostic data as a dataset.

Parameters:

kwargs – Additional parameters required to retrieve diagnostic data.

Return type:

Dataset

Returns:

Dataset containing this diagnostic data.

end: np.datetime64
property ext_end: datetime64
property ext_start: datetime64
property field: Field

Field of the main variable represented by this data.

field_x: Field
field_y: Field | None = None
field_z: Field
property fields: dict[str, Field]

Dictionary of fields used to compute this data.

group: str = None
group_base: str = None
html_elem(style='')

HTML representation of the data container.

Parameters:

style (str) – Style to add to the element.

Return type:

str

Returns:

Html representation of elements of this object.

html_header(span)

HTML representation header of the data container.

Parameters:

span (int) – Span size.

Returns:

Object html header.

property identity: DiagnosticType
indexer: ong_indexer_t.PassIndexer | None = None
initialize()

Diagnostic initialization.

At least the following properties have to be initialized:

  • group_base

  • storage_properties

is_autonomous(dask)

Check whether this diagnostic is autonomous with or without dask enabled.

Parameters:

dask (bool) – Whether dask is enabled or not.

Returns:

Whether this is an autonomous diagnostic or not.

is_dask_compatible(freq)

Check whether this diagnostic can be computed with dask or not.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

Return type:

bool

is_post_diag: bool = False
load_diagnostic(**kwargs)

Ensure requested diagnostic data are loaded.

make_dask_compatible(freq)

Set this container, so it can be used with dask.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

metadata(internal=False)

Diagnostic serializable configuration.

Parameters:

internal (bool) – Whether this diagnostic is an internal diagnostic or not.

Return type:

DiagnosticMeta

Returns:

Diagnostic metadata.

name: str
params: list[ong_edit_t.EditingComponent]
plot_container(template=None, stat=None, plot=None, group=None)

Create a PlotContainer for the provided stat data.

Parameters:

  • template (PlotTemplate | None) – Parameters template to use.

  • stat (StatType | None) – Statistic included in this data.

  • plot (str | None) – Plot type to display (“TIME”, “MAP” or “3D”).

  • group (str | None) – Data group to get.

Return type:

PlotContainer

Returns:

PlotContainer containing the requested data.

results: DiagResultType
start: np.datetime64
stats: list[StatType]
storage: StoragePropertiesType = None
store(store, group, analyse_type, analyse_date, start, end, mode, fields)

Write the diagnostic results into the provided store.

Parameters:

  • store (DiagnosticStore) – DiagnosticStore used for the storage.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the computed data.

  • end (datetime64) – Ending date for the computed data.

  • mode (StorageMode) – Storage mode to use when writing data.

  • fields (list[str]) – List of all fields name used in the group of this statistic.

store_configuration(analyse_type, analyse_date, group=None)

Returns the list of storage group parameters required to store this diagnostic.

Parameters:

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date representing the set of data used in this analyse. It’s used to determine at which timestamp to store non-temporal diagnostics.

  • group (str | None) – Name of the group to use for storage.

Return type:

list[StorageGroupParams]

Returns:

List of storage group parameters.

store_params: dict[str, Any] | None = None
time: Field
time_extension: bool = False
property variable: str

Name of the main variable represented by this data.

class casys.computation.diagnostics.EmptyResult

Bases: DiagResult

Results containing nothing.

Used for diagnostics not computed or without any result.

have_data()

Whether this result contains data or not.

Return type:

bool

Returns:

True if this result has data, false otherwise.

remove_data(field=None, stats=None)

Remove results’ data from the stat of the given field.

Parameters:

  • field (str) – Name of the field

  • stats (list[StatType]) – Statistics to remove.

statistics()

List of statistics computed for this diagnostic.

Return type:

list[StatType]

class casys.computation.diagnostics.GeoBoxDiag(name, store_params=None, *, data_index, data_time, start, end, field_x, field_y=None, indexer=None, time_extension=False, stats=<factory>, stat_selection=None, field_z, res_lon, res_lat, box_selection=None, projection=None)

Bases: BinnedDiagBase[BinnedResult, GeoBoxStorageProperties]

Class representing a geographical box statistic.

Parameters:

  • name (str) – Name of the data.

  • computed – True if the data have been computed, false otherwise.

  • is_post_diag – Whether this diagnostic is a post computation diagnostic or not.

  • data_index (str) – Name of the field on which the data is indexed.

  • start (np.datetime64) – Starting date of the input dataset.

  • end (np.datetime64) – Ending date of the input dataset.

  • field_x (Field) – Field used for x axes.

  • field_y (Field | None) – Field used for y axes.

  • stats (list[StatType]) – Statistics included in this Diagnostic.

  • indexer (ong_indexer_t.PassIndexer | None) – Indexer used to read data.

  • time_extension (bool) – Whether time extension is enabled or not.

  • ext_start – Extended starting date required by the computation.

  • ext_end – Ending date for the data required by the computation.

  • group – the name of the group of this diagnostic.

  • stat_selection (ong_clip_t.Clip | None) – Selection clip used to invalidate some bins.

  • field_z (Field) – Field used for z axes.

  • res_lon (DataResolution) – Min, max and width for the longitude axis.

  • res_lat (DataResolution) – Min, max and width for the latitude axis.

  • box_selection (Field | None) – Field used as selection for computation of the count statistic. Box in which the box_selection field does not contain any data will be set to NaN instead of 0.

  • store_params (dict[str, Any] | None)

  • data_time (str)

  • projection (pyproj_t.Proj | None)

INDEX_USED: ClassVar[str] = 'INDEX_USED'
MERGEABLE_STAT: ClassVar[list[StatType]] = [MIN, MAX, COUNT, MEAN, VAR, STD, PROD, SUM]
RES_TYPE

alias of BinnedResult

SCHEMA: ClassVar[type[mm.Schema]] = None
STORABLE: ClassVar[bool] = True
apply_stats_selection(diagnostic, fields)

Filter diagnostic by applying the statistic selection clip and removing unwanted statistics.

Parameters:

  • diagnostic (TypeVar(BinnedResultType, bound= BinnedResult)) – Diagnostic to filter.

  • fields (list[str]) – List of diagnostic field’s names.

Return type:

TypeVar(BinnedResultType, bound= BinnedResult)

box_selection: Field | None = None
classmethod check_box_selection(box_selection, stats)

Check and normalize the field used as box_selection parameter.

Parameters:

  • box_selection (Field | str | None) – Value of the box_selection field.

  • stats (list[StatType]) – Available stats in the current diagnostic.

Returns:

Normalized box_selection.

check_stat(stat)

Check whether this data is computed and provided stat is valid. Determines the stat if None is provided.

Parameters:

stat (StatType | None) – Statistic to check.

Return type:

StatType

Returns:

Checked statistic.

Raises:

AltiDataError – If data with the provided name were not computed. If the stat parameter is invalid.

clear_data()

Clear stored data from this Diagnostic data.

compute(data, start, end, fields, diags)

Compute the diagnostic on the provided data.

Parameters:

  • data (Dataset) – The dataset with data to read.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • diags (list[StatisticDiag]) – List of diagnostics included in this computation.

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Dictionary containing the statistic type with its associated data array.

compute_from_store(store, group, analyse_type, analyse_date, start, end, fields, delayed)

Compute diagnostic using stored data. The computation can be done in a delayed way allowing to only load requested data when needed.

Parameters:

  • store (DiagnosticStore) – Store containing diagnostics data.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • delayed (bool) – Flag indicating whether to load from store during computation or at first data request (plot/get_data).

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Computation results.

computed: bool = False
dask_full_compatible: bool = False
data_index: str
data_time: str
diag_data(**kwargs)

Diagnostic data as a dataset.

Parameters:

kwargs – Additional parameters required to retrieve diagnostic data.

Return type:

Dataset

Returns:

Dataset containing this diagnostic data.

end: np.datetime64
property ext_end: datetime64
property ext_start: datetime64
property field: Field

Field of the main variable represented by this data.

field_x: Field
field_y: Field | None = None
field_z: Field
property fields: dict[str, Field]

Dictionary of fields used to compute this data.

fill_count_with_nan()

Whether count statistics have to be filled with nan or not.

Return type:

bool

Returns:

True or False.

finalize(result)

Correctly format the diagnostic result.

Parameters:

result (TypeVar(BinnedResultType, bound= BinnedResult)) – Result to finalize.

Return type:

TypeVar(BinnedResultType, bound= BinnedResult)

Returns:

Finalized result.

static grid_2d(res_x, res_y)

Generate a Grid2D from provided parameters.

Parameters:
Return type:

Grid2D

Returns:

Grid2D with provided resolutions.

group: str = None
group_base: str = None
html_elem(style='')

HTML representation of the data container.

Parameters:

style (str) – Style to add to the element.

Return type:

str

Returns:

Html representation of elements of this object.

html_header(span)

HTML representation header of the data container.

Parameters:

span (int) – Span size.

Returns:

Object html header.

property identity: DiagnosticType
indexer: ong_indexer_t.PassIndexer | None = None
initialize()

Diagnostic initialization.

At least the following properties have to be initialized:

  • group_base

  • storage_properties

is_autonomous(dask)

Check whether this diagnostic is autonomous with or without dask enabled.

Parameters:

dask (bool) – Whether dask is enabled or not.

Returns:

Whether this is an autonomous diagnostic or not.

is_dask_compatible(freq)

Check whether this diagnostic can be computed with dask or not.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

Return type:

bool

is_post_diag: bool = False
load_diagnostic(**kwargs)

Ensure requested diagnostic data are loaded.

make_dask_compatible(freq)

Set this container, so it can be used with dask.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

merge(diagnostics)

Merge provided diagnostics.

Parameters:

diagnostics (list[TypeVar(CountResultType, bound= CountResult)]) – Diagnostics to merge.

Return type:

TypeVar(CountResultType, bound= CountResult)

Returns:

Merged diagnostic.

metadata(internal=False)

Diagnostic serializable configuration.

Parameters:

internal (bool) – Whether this diagnostic is an internal diagnostic or not.

Return type:

DiagnosticMeta

Returns:

Diagnostic metadata.

name: str
new_algorithm()

Instantiate the algorithm.

Return type:

BinnedStat2D

Returns:

Instantiated algorithm.

plot_container(template=None, stat=None, plot=None, missing=None)

Create a PlotContainer for the provided stat data.

Parameters:

  • template (PlotTemplate | None) – Parameters template to use.

  • stat (StatType | None) – Statistic included in this data.

  • plot (str | None) – Plot type to display (“BOX” or “3D”).

  • missing (bool | None) – Flag to indicate if the diagnostic is from a missing points diagnostic.

Return type:

PlotContainer

Returns:

PlotContainer containing the requested data.

projection: pyproj_t.Proj | None = None
res_lat: DataResolution
res_lon: DataResolution
results: DiagResultType
run_algorithm(data, fields)

Run the algorithm on the provided dataset.

Parameters:

  • data (Dataset) – Data on which to apply the algorithm.

  • fields (list[str]) – List of fields for which we want results.

Return type:

TypeVar(BinnedResultType, bound= BinnedResult)

Returns:

Computation result.

start: np.datetime64
stat_selection: ong_clip_t.Clip | None = None
stats: list[StatType]
storage: StoragePropertiesType = None
store(store, group, analyse_type, analyse_date, start, end, mode, fields)

Write the diagnostic results into the provided store.

Parameters:

  • store (DiagnosticStore) – DiagnosticStore used for the storage.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the computed data.

  • end (datetime64) – Ending date for the computed data.

  • mode (StorageMode) – Storage mode to use when writing data.

  • fields (list[str]) – List of all fields name used in the group of this statistic.

store_configuration(analyse_type, analyse_date, group=None)

Returns the list of storage group parameters required to store this diagnostic.

Parameters:

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date representing the set of data used in this analyse. It’s used to determine at which timestamp to store non-temporal diagnostics.

  • group (str | None) – Name of the group to use for storage.

Return type:

list[StorageGroupParams]

Returns:

List of storage group parameters.

store_params: dict[str, Any] | None = None
time_extension: bool = False
update_properties(diag)

Update diagnostic properties with the provided one.

Parameters:

diag (StatisticDiag) – Diagnostic from which to get properties.

property variable: str

Name of the main variable represented by this data.

class casys.computation.diagnostics.GeoBoxSwathDiag(name, store_params=None, *, data_index, data_time, start, end, field_x, field_y=None, indexer=None, time_extension=False, stats=<factory>, stat_selection=None, field_z, res_lon, res_lat, box_selection=None, projection=None)

Bases: GeoBoxDiag

Class representing a geographical box statistic.

Parameters:

  • name (str) – Name of the data.

  • computed – True if the data have been computed, false otherwise.

  • is_post_diag – Whether this diagnostic is a post computation diagnostic or not.

  • data_index (str) – Name of the field on which the data is indexed.

  • start (np.datetime64) – Starting date of the input dataset.

  • end (np.datetime64) – Ending date of the input dataset.

  • field_x (Field) – Field used for x axes.

  • field_y (Field | None) – Field used for y axes.

  • stats (list[StatType]) – Statistics included in this Diagnostic.

  • indexer (ong_indexer_t.PassIndexer | None) – Indexer used to read data.

  • time_extension (bool) – Whether time extension is enabled or not.

  • ext_start – Extended starting date required by the computation.

  • ext_end – Ending date for the data required by the computation.

  • group – the name of the group of this diagnostic.

  • stat_selection (ong_clip_t.Clip | None) – Selection clip used to invalidate some bins.

  • field_z (Field) – Field used for z axes.

  • res_lon (DataResolution) – Min, max and width for the longitude axis.

  • res_lat (DataResolution) – Min, max and width for the latitude axis.

  • box_selection (Field | None) – Field used as selection for computation of the count statistic. Box in which the box_selection field does not contain any data will be set to NaN instead of 0.

  • store_params (dict[str, Any] | None)

  • data_time (str)

  • projection (pyproj_t.Proj | None)

INDEX_USED: ClassVar[str] = 'INDEX_USED'
MERGEABLE_STAT: ClassVar[list[StatType]] = [MIN, MAX, COUNT, MEAN, VAR, STD, PROD, SUM]
RES_TYPE

alias of BinnedResult

SCHEMA: ClassVar[type[mm.Schema]] = None
STORABLE: ClassVar[bool] = True
apply_stats_selection(diagnostic, fields)

Filter diagnostic by applying the statistic selection clip and removing unwanted statistics.

Parameters:

  • diagnostic (TypeVar(BinnedResultType, bound= BinnedResult)) – Diagnostic to filter.

  • fields (list[str]) – List of diagnostic field’s names.

Return type:

TypeVar(BinnedResultType, bound= BinnedResult)

box_selection: Field | None = None
classmethod check_box_selection(box_selection, stats)

Check and normalize the field used as box_selection parameter.

Parameters:

  • box_selection (Field | str | None) – Value of the box_selection field.

  • stats (list[StatType]) – Available stats in the current diagnostic.

Returns:

Normalized box_selection.

check_stat(stat)

Check whether this data is computed and provided stat is valid. Determines the stat if None is provided.

Parameters:

stat (StatType | None) – Statistic to check.

Return type:

StatType

Returns:

Checked statistic.

Raises:

AltiDataError – If data with the provided name were not computed. If the stat parameter is invalid.

clear_data()

Clear stored data from this Diagnostic data.

compute(data, start, end, fields, diags)

Compute the diagnostic on the provided data.

Parameters:

  • data (Dataset) – The dataset with data to read.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • diags (list[StatisticDiag]) – List of diagnostics included in this computation.

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Dictionary containing the statistic type with its associated data array.

compute_from_store(store, group, analyse_type, analyse_date, start, end, fields, delayed)

Compute diagnostic using stored data. The computation can be done in a delayed way allowing to only load requested data when needed.

Parameters:

  • store (DiagnosticStore) – Store containing diagnostics data.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • delayed (bool) – Flag indicating whether to load from store during computation or at first data request (plot/get_data).

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Computation results.

computed: bool = False
dask_full_compatible: bool = False
data_index: str
data_time: str
diag_data(**kwargs)

Diagnostic data as a dataset.

Parameters:

kwargs – Additional parameters required to retrieve diagnostic data.

Return type:

Dataset

Returns:

Dataset containing this diagnostic data.

end: np.datetime64
property ext_end: datetime64
property ext_start: datetime64
property field: Field

Field of the main variable represented by this data.

field_x: Field
field_y: Field | None = None
field_z: Field
property fields: dict[str, Field]

Dictionary of fields used to compute this data.

fill_count_with_nan()

Whether count statistics have to be filled with nan or not.

Return type:

bool

Returns:

True or False.

finalize(result)

Correctly format the diagnostic result.

Parameters:

result (TypeVar(BinnedResultType, bound= BinnedResult)) – Result to finalize.

Return type:

TypeVar(BinnedResultType, bound= BinnedResult)

Returns:

Finalized result.

static grid_2d(res_x, res_y)

Generate a Grid2D from provided parameters.

Parameters:
Return type:

Grid2D

Returns:

Grid2D with provided resolutions.

group: str = None
group_base: str = None
html_elem(style='')

HTML representation of the data container.

Parameters:

style (str) – Style to add to the element.

Return type:

str

Returns:

Html representation of elements of this object.

html_header(span)

HTML representation header of the data container.

Parameters:

span (int) – Span size.

Returns:

Object html header.

property identity: DiagnosticType
indexer: ong_indexer_t.PassIndexer | None = None
initialize()

Diagnostic initialization.

At least the following properties have to be initialized:

  • group_base

  • storage_properties

is_autonomous(dask)

Check whether this diagnostic is autonomous with or without dask enabled.

Parameters:

dask (bool) – Whether dask is enabled or not.

Returns:

Whether this is an autonomous diagnostic or not.

is_dask_compatible(freq)

Check whether this diagnostic can be computed with dask or not.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

Return type:

bool

is_post_diag: bool = False
load_diagnostic(**kwargs)

Ensure requested diagnostic data are loaded.

make_dask_compatible(freq)

Set this container, so it can be used with dask.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

merge(diagnostics)

Merge provided diagnostics.

Parameters:

diagnostics (list[TypeVar(CountResultType, bound= CountResult)]) – Diagnostics to merge.

Return type:

TypeVar(CountResultType, bound= CountResult)

Returns:

Merged diagnostic.

metadata(internal=False)

Diagnostic serializable configuration.

Parameters:

internal (bool) – Whether this diagnostic is an internal diagnostic or not.

Return type:

DiagnosticMeta

Returns:

Diagnostic metadata.

name: str
new_algorithm()

Instantiate the algorithm.

Return type:

BinnedStat2D

Returns:

Instantiated algorithm.

plot_container(template=None, stat=None, plot=None, missing=None)

Create a PlotContainer for the provided stat data.

Parameters:

  • template (PlotTemplate | None) – Parameters template to use.

  • stat (StatType | None) – Statistic included in this data.

  • plot (str | None) – Plot type to display (“BOX” or “3D”).

  • missing (bool | None) – Flag to indicate if the diagnostic is from a missing points diagnostic.

Return type:

PlotContainer

Returns:

PlotContainer containing the requested data.

projection: pyproj_t.Proj | None = None
res_lat: DataResolution
res_lon: DataResolution
results: DiagResultType
run_algorithm(data, fields)

Run the algorithm on the provided dataset.

Parameters:

  • data (Dataset) – Data on which to apply the algorithm.

  • fields (list[str]) – List of fields for which we want results.

Return type:

TypeVar(BinnedResultType, bound= BinnedResult)

Returns:

Computation result.

start: np.datetime64
stat_selection: ong_clip_t.Clip | None = None
stats: list[StatType]
storage: StoragePropertiesType = None
store(store, group, analyse_type, analyse_date, start, end, mode, fields)

Write the diagnostic results into the provided store.

Parameters:

  • store (DiagnosticStore) – DiagnosticStore used for the storage.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the computed data.

  • end (datetime64) – Ending date for the computed data.

  • mode (StorageMode) – Storage mode to use when writing data.

  • fields (list[str]) – List of all fields name used in the group of this statistic.

store_configuration(analyse_type, analyse_date, group=None)

Returns the list of storage group parameters required to store this diagnostic.

Parameters:

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date representing the set of data used in this analyse. It’s used to determine at which timestamp to store non-temporal diagnostics.

  • group (str | None) – Name of the group to use for storage.

Return type:

list[StorageGroupParams]

Returns:

List of storage group parameters.

store_params: dict[str, Any] | None = None
time_extension: bool = False
update_properties(diag)

Update diagnostic properties with the provided one.

Parameters:

diag (StatisticDiag) – Diagnostic from which to get properties.

property variable: str

Name of the main variable represented by this data.

class casys.computation.diagnostics.HistogramDiag(name, store_params=None, *, data_index, data_time, start, end, field_x, field_y=None, indexer=None, time_extension=False, stats=<factory>, res_x)

Bases: CountDiag[CountResult, HistogramStorageProperties]

Class representing a histogram statistic.

Parameters:

  • name (str) – Name of the data.

  • computed – True if the data have been computed, false otherwise.

  • is_post_diag – Whether this diagnostic is a post computation diagnostic or not.

  • data_index (str) – Name of the field on which the data is indexed.

  • start (np.datetime64) – Starting date of the input dataset.

  • end (np.datetime64) – Ending date of the input dataset.

  • field_x (Field) – Field used for x axes.

  • field_y (Field) – Field used for y axes.

  • stats (list[StatType]) – Statistics included in this Diagnostic.

  • indexer (ong_indexer_t.PassIndexer | None) – Indexer used to read data.

  • time_extension (bool) – Whether time extension is enabled or not.

  • ext_start – Extended starting date required by the computation.

  • ext_end – Ending date for the data required by the computation.

  • group – the name of the group of this diagnostic.

  • res_x (DataResolution) – Min, max and width for the x-axis.

  • store_params (dict[str, Any] | None)

  • data_time (str)

MERGEABLE_STAT: ClassVar[list[StatType]] = [MIN, MAX, COUNT, MEAN, VAR, STD, PROD, SUM]
RES_TYPE

alias of CountResult

SCHEMA: ClassVar[type[mm.Schema]] = None
STORABLE: ClassVar[bool] = True
apply_stats_selection(diagnostic, fields)

Filter diagnostic by applying the statistic selection clip and removing unwanted statistics.

Parameters:

  • diagnostic (TypeVar(DiagResultType, bound= DiagResult)) – Diagnostic to filter.

  • fields (list[str]) – List of diagnostic field’s names.

Return type:

TypeVar(DiagResultType, bound= DiagResult)

check_stat(stat)

Check whether this data is computed and provided stat is valid. Determines the stat if None is provided.

Parameters:

stat (StatType | None) – Statistic to check.

Return type:

StatType

Returns:

Checked statistic.

Raises:

AltiDataError – If data with the provided name were not computed. If the stat parameter is invalid.

clear_data()

Clear stored data from this Diagnostic data.

compute(data, start, end, fields, diags)

Compute the diagnostic on the provided data.

Parameters:

  • data (Dataset) – The dataset with data to read.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • diags (list[StatisticDiag]) – List of diagnostics included in this computation.

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Dictionary containing the statistic type with its associated data array.

compute_from_store(store, group, analyse_type, analyse_date, start, end, fields, delayed)

Compute diagnostic using stored data. The computation can be done in a delayed way allowing to only load requested data when needed.

Parameters:

  • store (DiagnosticStore) – Store containing diagnostics data.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • delayed (bool) – Flag indicating whether to load from store during computation or at first data request (plot/get_data).

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Computation results.

computed: bool = False
dask_full_compatible: bool = False
data_index: str
data_time: str
diag_data(**kwargs)

Diagnostic data as a dataset.

Parameters:

kwargs – Additional parameters required to retrieve diagnostic data.

Return type:

Dataset

Returns:

Dataset containing this diagnostic data.

end: np.datetime64
property ext_end: datetime64
property ext_start: datetime64
property field: Field

Field of the main variable represented by this data.

field_x: Field
field_y: Field = None
property fields: dict[str, Field]

Dictionary of fields used to compute this data.

finalize(result)

Correctly format the diagnostic result.

Parameters:

result (TypeVar(CountResultType, bound= CountResult)) – Result to finalize.

Return type:

TypeVar(CountResultType, bound= CountResult)

Returns:

Finalized result.

static grid_2d(res_x, res_y)

Generate a Grid2D from provided parameters.

Parameters:
Return type:

Grid2D

Returns:

Grid2D with provided resolutions.

group: str = None
group_base: str = None
html_elem(style='')

HTML representation of the data container.

Parameters:

style (str) – Style to add to the element.

Return type:

str

Returns:

Html representation of elements of this object.

html_header(span)

HTML representation header of the data container.

Parameters:

span (int) – Span size.

Returns:

Object html header.

property identity: DiagnosticType
indexer: ong_indexer_t.PassIndexer | None = None
initialize()

Diagnostic initialization.

At least the following properties have to be initialized:

  • group_base

  • storage_properties

is_autonomous(dask)

Check whether this diagnostic is autonomous with or without dask enabled.

Parameters:

dask (bool) – Whether dask is enabled or not.

Returns:

Whether this is an autonomous diagnostic or not.

is_dask_compatible(freq)

Check whether this diagnostic can be computed with dask or not.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

Return type:

bool

is_post_diag: bool = False
load_diagnostic(**kwargs)

Ensure requested diagnostic data are loaded.

make_dask_compatible(freq)

Set this container, so it can be used with dask.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

merge(diagnostics)

Merge provided diagnostics.

Parameters:

diagnostics (list[TypeVar(CountResultType, bound= CountResult)]) – Diagnostics to merge.

Return type:

TypeVar(CountResultType, bound= CountResult)

Returns:

Merged diagnostic.

metadata(internal=False)

Diagnostic serializable configuration.

Parameters:

internal (bool) – Whether this diagnostic is an internal diagnostic or not.

Return type:

DiagnosticMeta

Returns:

Diagnostic metadata.

name: str
new_algorithm()

Instantiate the algorithm.

Return type:

Dispersion1D

Returns:

Instantiated algorithm.

plot_container(template=None, stat=None)

Create a PlotContainer for the provided stat data.

Parameters:
Return type:

PlotContainer

Returns:

PlotContainer containing the requested data.

res_x: DataResolution
results: DiagResultType
start: np.datetime64
stats: list[StatType]
storage: StoragePropertiesType = None
store(store, group, analyse_type, analyse_date, start, end, mode, fields)

Write the diagnostic results into the provided store.

Parameters:

  • store (DiagnosticStore) – DiagnosticStore used for the storage.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the computed data.

  • end (datetime64) – Ending date for the computed data.

  • mode (StorageMode) – Storage mode to use when writing data.

  • fields (list[str]) – List of all fields name used in the group of this statistic.

store_configuration(analyse_type, analyse_date, group=None)

Returns the list of storage group parameters required to store this diagnostic.

Parameters:

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date representing the set of data used in this analyse. It’s used to determine at which timestamp to store non-temporal diagnostics.

  • group (str | None) – Name of the group to use for storage.

Return type:

list[StorageGroupParams]

Returns:

List of storage group parameters.

store_params: dict[str, Any] | None = None
time_extension: bool = False
update_properties(diag)

Update diagnostic properties with the provided one.

Parameters:

diag (HistogramDiag) – Diagnostic from which to get properties.

property variable: str

Name of the main variable represented by this data.

class casys.computation.diagnostics.HistogramSwathDiag(name, store_params=None, *, data_index, data_time, start, end, field_x, field_y=None, indexer=None, time_extension=False, stats=<factory>, res_x)

Bases: HistogramDiag

Class representing a histogram statistic.

Parameters:

  • name (str) – Name of the data.

  • computed – True if the data have been computed, false otherwise.

  • is_post_diag – Whether this diagnostic is a post computation diagnostic or not.

  • data_index (str) – Name of the field on which the data is indexed.

  • start (np.datetime64) – Starting date of the input dataset.

  • end (np.datetime64) – Ending date of the input dataset.

  • field_x (Field) – Field used for x axes.

  • field_y (Field) – Field used for y axes.

  • stats (list[StatType]) – Statistics included in this Diagnostic.

  • indexer (ong_indexer_t.PassIndexer | None) – Indexer used to read data.

  • time_extension (bool) – Whether time extension is enabled or not.

  • ext_start – Extended starting date required by the computation.

  • ext_end – Ending date for the data required by the computation.

  • group – the name of the group of this diagnostic.

  • res_x (DataResolution) – Min, max and width for the x-axis.

  • store_params (dict[str, Any] | None)

  • data_time (str)

MERGEABLE_STAT: ClassVar[list[StatType]] = [MIN, MAX, COUNT, MEAN, VAR, STD, PROD, SUM]
RES_TYPE

alias of CountResult

SCHEMA: ClassVar[type[mm.Schema]] = None
STORABLE: ClassVar[bool] = True
apply_stats_selection(diagnostic, fields)

Filter diagnostic by applying the statistic selection clip and removing unwanted statistics.

Parameters:

  • diagnostic (TypeVar(DiagResultType, bound= DiagResult)) – Diagnostic to filter.

  • fields (list[str]) – List of diagnostic field’s names.

Return type:

TypeVar(DiagResultType, bound= DiagResult)

check_stat(stat)

Check whether this data is computed and provided stat is valid. Determines the stat if None is provided.

Parameters:

stat (StatType | None) – Statistic to check.

Return type:

StatType

Returns:

Checked statistic.

Raises:

AltiDataError – If data with the provided name were not computed. If the stat parameter is invalid.

clear_data()

Clear stored data from this Diagnostic data.

compute(data, start, end, fields, diags)

Compute the diagnostic on the provided data.

Parameters:

  • data (Dataset) – The dataset with data to read.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • diags (list[StatisticDiag]) – List of diagnostics included in this computation.

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Dictionary containing the statistic type with its associated data array.

compute_from_store(store, group, analyse_type, analyse_date, start, end, fields, delayed)

Compute diagnostic using stored data. The computation can be done in a delayed way allowing to only load requested data when needed.

Parameters:

  • store (DiagnosticStore) – Store containing diagnostics data.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • delayed (bool) – Flag indicating whether to load from store during computation or at first data request (plot/get_data).

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Computation results.

computed: bool = False
dask_full_compatible: bool = False
data_index: str
data_time: str
diag_data(**kwargs)

Diagnostic data as a dataset.

Parameters:

kwargs – Additional parameters required to retrieve diagnostic data.

Return type:

Dataset

Returns:

Dataset containing this diagnostic data.

end: np.datetime64
property ext_end: datetime64
property ext_start: datetime64
property field: Field

Field of the main variable represented by this data.

field_x: Field
field_y: Field = None
property fields: dict[str, Field]

Dictionary of fields used to compute this data.

finalize(result)

Correctly format the diagnostic result.

Parameters:

result (TypeVar(CountResultType, bound= CountResult)) – Result to finalize.

Return type:

TypeVar(CountResultType, bound= CountResult)

Returns:

Finalized result.

static grid_2d(res_x, res_y)

Generate a Grid2D from provided parameters.

Parameters:
Return type:

Grid2D

Returns:

Grid2D with provided resolutions.

group: str = None
group_base: str = None
html_elem(style='')

HTML representation of the data container.

Parameters:

style (str) – Style to add to the element.

Return type:

str

Returns:

Html representation of elements of this object.

html_header(span)

HTML representation header of the data container.

Parameters:

span (int) – Span size.

Returns:

Object html header.

property identity: DiagnosticType
indexer: ong_indexer_t.PassIndexer | None = None
initialize()

Diagnostic initialization.

At least the following properties have to be initialized:

  • group_base

  • storage_properties

is_autonomous(dask)

Check whether this diagnostic is autonomous with or without dask enabled.

Parameters:

dask (bool) – Whether dask is enabled or not.

Returns:

Whether this is an autonomous diagnostic or not.

is_dask_compatible(freq)

Check whether this diagnostic can be computed with dask or not.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

Return type:

bool

is_post_diag: bool = False
load_diagnostic(**kwargs)

Ensure requested diagnostic data are loaded.

make_dask_compatible(freq)

Set this container, so it can be used with dask.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

merge(diagnostics)

Merge provided diagnostics.

Parameters:

diagnostics (list[TypeVar(CountResultType, bound= CountResult)]) – Diagnostics to merge.

Return type:

TypeVar(CountResultType, bound= CountResult)

Returns:

Merged diagnostic.

metadata(internal=False)

Diagnostic serializable configuration.

Parameters:

internal (bool) – Whether this diagnostic is an internal diagnostic or not.

Return type:

DiagnosticMeta

Returns:

Diagnostic metadata.

name: str
new_algorithm()

Instantiate the algorithm.

Return type:

Dispersion1D

Returns:

Instantiated algorithm.

plot_container(template=None, stat=None)

Create a PlotContainer for the provided stat data.

Parameters:
Return type:

PlotContainer

Returns:

PlotContainer containing the requested data.

res_x: DataResolution
results: DiagResultType
start: np.datetime64
stats: list[StatType]
storage: StoragePropertiesType = None
store(store, group, analyse_type, analyse_date, start, end, mode, fields)

Write the diagnostic results into the provided store.

Parameters:

  • store (DiagnosticStore) – DiagnosticStore used for the storage.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the computed data.

  • end (datetime64) – Ending date for the computed data.

  • mode (StorageMode) – Storage mode to use when writing data.

  • fields (list[str]) – List of all fields name used in the group of this statistic.

store_configuration(analyse_type, analyse_date, group=None)

Returns the list of storage group parameters required to store this diagnostic.

Parameters:

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date representing the set of data used in this analyse. It’s used to determine at which timestamp to store non-temporal diagnostics.

  • group (str | None) – Name of the group to use for storage.

Return type:

list[StorageGroupParams]

Returns:

List of storage group parameters.

store_params: dict[str, Any] | None = None
time_extension: bool = False
update_properties(diag)

Update diagnostic properties with the provided one.

Parameters:

diag (HistogramDiag) – Diagnostic from which to get properties.

property variable: str

Name of the main variable represented by this data.

class casys.computation.diagnostics.MissingPointsDiag(name, store_params=None, *, data_index, data_time, start, end, field_x, field_y=None, indexer=None, time_extension=False, stats=<factory>, time, reference_track, method, distance_threshold, time_gap_reference, time_gap_threshold, geobox_stats, res_lon, res_lat, temporal_freq, section_min_lengths, group_names, group_grid, group_converter, reader)

Bases: StatisticDiag[MissingPointsResult, MissingPointsStorageProperties]

Class representing missing points data and statistic.

Parameters:

  • name (str) – Name of the data.

  • computed – True if the data have been computed, false otherwise.

  • is_post_diag – Whether this diagnostic is a post computation diagnostic or not.

  • data_index (str) – Name of the field on which the data is indexed.

  • start (np.datetime64) – Starting date of the input dataset.

  • end (np.datetime64) – Ending date of the input dataset.

  • field_x (Field) – Field used for x axes.

  • field_y (Field | None) – Field used for y axes.

  • stats (list[StatType]) – Statistics included in this Diagnostic.

  • indexer (ong_indexer_t.PassIndexer | None) – Indexer used to read data.

  • time_extension (bool) – Whether time extension is enabled or not.

  • ext_start – Extended starting date required by the computation.

  • ext_end – Ending date for the data required by the computation.

  • group – the name of the group of this diagnostic.

  • time (Field) – Field used for the time.

  • reference_track (ong_orbit_rt.ReferenceTrack) – Reference track.

  • method (ong_missing_t.MissingPointsMethod) – Missing points calculation mode.

  • distance_threshold (float) – Distance threshold between real and theoretical points expressed as a factor of the distance between two consecutive theoretical points. An exception is raised if the threshold is exceeded.

  • time_gap_reference (np.timedelta64 | None) – Standard time gap between two consecutive real points. Use the reference track time gap as default value.

  • time_gap_threshold (float) – [real method parameter] Time threshold between two consecutive real points expressed as a factor of time_gap_reference to determine that a point is missing. The standard value is 2.0: if two consecutive real points have a time gap of 2.1 * time_gap_reference then a point is considered to be missing.

  • geobox_stats (bool) – Whether to compute geographical box statistics or not.

  • res_lon (DataResolution) – Min, max and width for the longitude axis.

  • res_lat (DataResolution) – Min, max and width for the latitude axis.

  • temporal_freq (list[FrequencyHandler]) – List of temporal statistics frequencies to compute.

  • section_min_lengths (list[int] | None) – List of missing sections minimum length values.

  • group_names (dict[str, int]) – Dictionary containing the name of the group associated to its flag value.

  • group_grid (ong_in_grid_t.InGridParameters | None) –

    If an InGridParameters:

    Define groups using the provided parameters in association with the group_names parameter.

    If False or None:

    Does not define groups.

  • group_converter (Callable[[np.ndarray], np.ndarray]) – Callable to apply to grid values to transform them into group values. This callable has to take a numpy array as single parameter and return a new one. If set to None, no conversion is made.

  • reader (CasysReader) – Reference to the reader (used to apply shape selection on results)

  • store_params (dict[str, Any] | None)

  • data_time (str)

MERGEABLE_STAT: ClassVar[list[StatType]] = [MIN, MAX, COUNT, MEAN, VAR, STD, PROD, SUM]
RES_TYPE: ClassVar[type[DiagResult]] = None
SCHEMA: ClassVar[type[mm.Schema]] = None
STORABLE: ClassVar[bool] = True
apply_stats_selection(diagnostic, fields)

Filter diagnostic by applying the statistic selection clip and removing unwanted statistics.

Parameters:

  • diagnostic (TypeVar(DiagResultType, bound= DiagResult)) – Diagnostic to filter.

  • fields (list[str]) – List of diagnostic field’s names.

Return type:

TypeVar(DiagResultType, bound= DiagResult)

box_diag: GeoBoxDiag | None
check_stat(stat)

Check whether this data is computed and provided stat is valid. Determines the stat if None is provided.

Parameters:

stat (StatType | None) – Statistic to check.

Return type:

StatType

Returns:

Checked statistic.

Raises:

AltiDataError – If data with the provided name were not computed. If the stat parameter is invalid.

clear_data()

Clear stored data from this Diagnostic data.

compute(data, start, end, fields, diags)

Compute the diagnostic on the provided data.

Parameters:

  • data (Dataset) – The dataset with data to read.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • diags (list[StatisticDiag]) – List of diagnostics included in this computation.

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Dictionary containing the statistic type with its associated data array.

compute_from_store(store, group, analyse_type, analyse_date, start, end, fields, delayed)

Compute diagnostic using stored data. The computation can be done in a delayed way allowing to only load requested data when needed.

Parameters:

  • store (DiagnosticStore) – Store containing diagnostics data.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • delayed (bool) – Flag indicating whether to load from store during computation or at first data request (plot/get_data).

Return type:

MissingPointsResult

Returns:

Computation results.

computed: bool = False
dask_full_compatible: bool = False
data_index: str
data_time: str
diag_data(**kwargs)

Diagnostic data as a dataset.

Parameters:

kwargs – Additional parameters required to retrieve diagnostic data.

Return type:

Dataset

Returns:

Dataset containing this diagnostic data.

distance_threshold: float
end: np.datetime64
property ext_end: datetime64
property ext_start: datetime64
property field: Field

Field of the main variable represented by this data.

field_x: Field
field_y: Field | None = None
property fields: dict[str, Field]

Dictionary of fields used to compute this data.

finalize(result)

Correctly format the diagnostic result.

Parameters:

result (MissingPointsResult) – Result to finalize.

Return type:

MissingPointsResult

Returns:

Finalized result.

geobox_stats: bool
group: str = None
group_base: str = None
group_converter: Callable[[np.ndarray], np.ndarray]
group_grid: ong_in_grid_t.InGridParameters | None
group_names: dict[str, int]
html_elem(style='')

HTML representation of the data container.

Parameters:

style (str) – Style to add to the element.

Return type:

str

Returns:

Html representation of elements of this object.

html_header(span)

HTML representation header of the data container.

Parameters:

span (int) – Span size.

Returns:

Object html header.

property identity: DiagnosticType
indexer: ong_indexer_t.PassIndexer | None = None
initialize()

Diagnostic initialization.

At least the following properties have to be initialized:

  • group_base

  • storage_properties

is_autonomous(dask)

Check whether this diagnostic is autonomous with or without dask enabled.

Parameters:

dask (bool) – Whether dask is enabled or not.

Returns:

Whether this is an autonomous diagnostic or not.

is_dask: bool = False
is_dask_compatible(freq)

Check whether this diagnostic can be computed with dask or not.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

Return type:

bool

is_post_diag: bool = False
load_diagnostic(**kwargs)

Ensure requested diagnostic data are loaded.

make_dask_compatible(freq)

Set this container, so it can be used with dask.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

merge(diagnostics)

Merge provided diagnostics.

Parameters:

diagnostics (list[MissingPointsResult]) – Diagnostics to merge.

Return type:

MissingPointsResult

Returns:

Merged diagnostic.

metadata(internal=False)

Diagnostic serializable configuration.

Parameters:

internal (bool) – Whether this diagnostic is an internal diagnostic or not.

Return type:

DiagnosticMeta

Returns:

Diagnostic metadata.

method: ong_missing_t.MissingPointsMethod
name: str
static normalize_group_name(*, group)

Normalize a missing points group name.

Parameters:

group (str) – Missing points group to normalize.

Return type:

str

Returns:

Normalized missing points group.

normalize_section_min_length(min_length=None)

Normalize the section min length parameter value.

Parameters:

min_length (int | None) – Section min length value.

Returns:

Normalized section min length value.

plot_container(stat=None, template=None, group=None, plot=None, dtype=None, freq=None, section_min_length=None, sections=None)

Create a PlotContainer for the provided stat data.

Parameters:

  • template (PlotTemplate | None) – Parameters template to use.

  • stat (StatType | None) – Statistic included in this data.

  • group (str | None) – Name of the group to get the data from.

  • plot (str | DiagnosticType | None) – Type of the diagnostic data to get (“map”, “temporal”, “geobox”).

  • dtype (str | MPointsType | None) – Type of the missing points data to get (“all”, “missing”, “available”)

  • freq (str | FreqType | None) – Frequency of the requested statistic.

  • section_min_length (int | None) – Section minimal length value for section analyses.

  • sections (list[int] | None) – Sections to plot (“all”, number or list of numbers).

  • sections –

    Sections to plot:

    • ”all”: Select all sections

    • number: Select section’s requested number

    • list of numbers: Select sections’ requested numbers

Return type:

PlotContainer

Returns:

PlotContainer containing the requested data.

reader: CasysReader
reference_track: ong_orbit_rt.ReferenceTrack
res_lat: DataResolution
res_lon: DataResolution
results: DiagResultType
sect_diags: dict[str, dict[int, SectionAnalysesDiag]]
section_min_lengths: list[int] | None
start: np.datetime64
stats: list[StatType]
storage: StoragePropertiesType = None
store(store, group, analyse_type, analyse_date, start, end, mode, fields)

Write the diagnostic results into the provided store.

Parameters:

  • store (DiagnosticStore) – DiagnosticStore used for the storage.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the computed data.

  • end (datetime64) – Ending date for the computed data.

  • mode (StorageMode) – Storage mode to use when writing data.

  • fields (list[str]) – List of all fields name used in the group of this statistic.

store_configuration(analyse_type, analyse_date, group=None)

Returns the list of storage group parameters required to store this diagnostic.

Parameters:

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date representing the set of data used in this analyse. It’s used to determine at which timestamp to store non-temporal diagnostics.

  • group (str | None) – Name of the group to use for storage.

Return type:

list[StorageGroupParams]

Returns:

List of storage group parameters.

store_params: dict[str, Any] | None = None
temp_diags: dict[str, TemporalDiag]
temporal_freq: list[FrequencyHandler]
time: Field
time_extension: bool = False
time_gap_reference: np.timedelta64 | None
time_gap_threshold: float
update_properties(diag)

Update diagnostic properties with the provided one.

Parameters:

diag (StatisticDiag) – Diagnostic from which to get properties.

property use_group_grid: bool

Whether this diagnostic use a grid to define groups or not.

property variable: str

Name of the main variable represented by this data.

class casys.computation.diagnostics.MissingPointsResult(missing, geobox=None, temporal=<factory>, sections=<factory>, tmp_geobox=<factory>, tmp_temporal=<factory>)

Bases: StatDiagResult

Missing points diagnostic results.

Parameters:
property data: dict[DiagnosticType, Dataset] | dict[DiagnosticType, dict[str, Dataset]] | dict[DiagnosticType, dict[int, Analysis]]
geobox: Dataset | None = None
geobox_data(group, mp_type)

Get geobox data and field for the requested group.

Parameters:

  • group (str) – Group’s name.

  • mp_type (MPointsType) – Type of data to get.

Return type:

tuple[Dataset, Field]

Returns:

  • data – Data as a dataset.

  • field – Field to use for this data.

have_data()

Whether this result contains data or not.

Return type:

bool

Returns:

True if this result has data, false otherwise.

merge_data(result)

Merge provided result’s data into this one. These results are in a finalized state.

Parameters:

result (CountResult | None) – Result to merge.

merge_stats(results, index=None, **kwargs)

Merge provided results into this one. These results are in a non finalized state.

Parameters:
missing: Dataset | None
mp_all: ClassVar[str] = 'all'
static mp_all_gr(group)
Parameters:

group (str)

mp_available: ClassVar[str] = 'available'
static mp_available_gr(group)
Parameters:

group (str)

classmethod mp_field(mp_type)

Generate a field for the requested type.

Parameters:

mp_type (MPointsType) – Missing points’ type field.

Returns:

New missing points’ field.

Return type:

field

classmethod mp_field_gr(mp_type, group)

Generate a field for the requested type and group.

Parameters:

  • mp_type (MPointsType) – Missing points’ type field.

  • group (str) – Group’s name.

Returns:

New missing points’ field.

Return type:

field

mp_global: ClassVar[str] = 'GLOBAL'
static mp_global_gr(group)
Parameters:

group (str)

mp_groups: ClassVar[str] = 'mp_groups'
mp_missing: ClassVar[str] = 'missing'
static mp_missing_gr(group)
Parameters:

group (str)

raw_data(index, group_names, group, mp_type)

Get raw data and field for the requested group.

Parameters:

  • index (str) – Data’s index name.

  • group_names (dict[str, int]) – Group names associated to their value.

  • group (str) – Name of the group to get.

  • mp_type (MPointsType) – Type of data.

Return type:

tuple[Dataset, Field]

Returns:

  • data – Data as a dataset.

  • field – Field to use for this data.

remove_data(field=None, stats=None)

Remove results’ data from the stat of the given field.

Parameters:

  • field (str) – Name of the field

  • stats (list[StatType]) – Statistics to remove.

remove_fields(fields)

Remove provided fields from statistics.

Parameters:

fields (list[str]) – Fields to remove.

sections: dict[str, dict[int, Analysis]]
set_geobox(group, missing, available)
Parameters:
set_temporal(group, freq, missing, available)
Parameters:
statistics()

List of statistics computed for this diagnostic.

Return type:

list[StatType]

temporal: dict[str, Dataset]
temporal_data(group, freq, mp_type)

Get temporal data and field for the requested group and frequency.

Parameters:

  • group (str) – Group’s name.

  • freq (str) – Frequency.

  • mp_type (MPointsType) – Type of data to get.

Return type:

tuple[Dataset, Field]

Returns:

  • data – Data as a dataset.

  • field – Field to use for this data.

tmp_geobox: dict[str, MPointsSubDiag]
tmp_temporal: dict[str, MPointsFreq]
class casys.computation.diagnostics.PeriodogramDiag(name, store_params=None, *, indexer, base_diag, temporal_diag, diag_kwargs, stats, nbr_periods, first_period, last_period, ref_period)

Bases: PostDiagnostic[PeriodogramResult, PeriodogramStorageProperties]

Container for the periodogram diagnostic.

Parameters:

  • name (str) – Name of the data.

  • computed – True if the data have been computed, false otherwise.

  • is_post_diag – Whether this diagnostic is a post computation diagnostic or not.

  • stats (list[StatType] | None) – Statistics included in this Diagnostic.

  • indexer (PassIndexer | None) – Indexer used to read data.

  • group – The name of the group of this diagnostic.

  • base_diag (TemporalDiag | MissingPointsDiag | CrossoverDiag | CrossoverSwathDiag) – The diagnostic being/containing the temporal diagnostic.

  • temporal_diag (TemporalDiag) – The temporal diagnostic which result is used to compute the periodogram.

  • diag_kwargs (dict[str, Any]) –

    Dictionary containing the necessary information to identify a temporal sub diagnostics (similar to the get_data method ones).

    • Crossovers: “freq”, the frequency of the sub-diagnostic to process.

    • Missing points:

      • ”freq”: the frequency of the sub-diagnostic to process,

      • ”group”: one of the missing points groups, defined in “mp_groups”,

      the add_missing_points_stat method parameters (ex: “GLOBAL”)

      • ”dtype”: “missing” or “available”.

  • nbr_periods (int) – Number of periods.

  • first_period (timedelta64 | None) – First period interval (to be used with ‘last_period’ or ‘ref_period’).

  • last_period (timedelta64 | None) – Last period interval (to be used with ‘first_period’ or ‘ref_period’).

  • ref_period (timedelta64 | None) – Reference period interval (to be used with ‘first_period’ or last_period’).

  • store_params (dict[str, Any] | None)

RES_TYPE

alias of PeriodogramResult

SCHEMA: ClassVar[type[mm.Schema]] = None
STORABLE = True
base_diag: casys_diag_t.TemporalDiag | casys_diag_t.MissingPointsDiag | casys_diag_t.CrossoverDiag | casys_diag_t.CrossoverSwathDiag
computed: bool = False
dask_full_compatible: bool = False
diag_data(stat=None)

Diagnostic data as a dataset.

Parameters:

stat (StatType | None) – Statistic to get.

Return type:

Dataset

Returns:

Dataset containing this diagnostic data.

diag_kwargs: dict[str, Any]
property field: Field
first_period: np.timedelta64 | None
group: str = None
group_base: str = None
html_elem(style='')

HTML representation of the data container.

Parameters:

style (str) – Style to add to the element.

Return type:

str

Returns:

Html representation of elements of this object.

html_header(span)

HTML representation header of the data container.

Parameters:

span (int) – Span size.

Return type:

str

Returns:

Object html header.

property identity: DiagnosticType
indexer: ong_indexer_t.PassIndexer | None
initialize()

Diagnostic initialization.

At least the following properties have to be initialized:

  • group_base

  • storage_properties

is_autonomous(dask)

Check whether this diagnostic is autonomous with or without dask enabled.

Parameters:

dask (bool) – Whether dask is enabled or not.

Returns:

Whether this is an autonomous diagnostic or not.

is_post_diag: bool = False
last_period: np.timedelta64 | None
metadata(internal=False)

Diagnostic serializable configuration.

Parameters:

internal (bool) – Whether this diagnostic is an internal diagnostic or not.

Return type:

DiagnosticMeta

Returns:

Diagnostic metadata.

name: str
nbr_periods: int
plot_container(template=None, stat=None)
Parameters:
Return type:

PlotContainer

post_compute()

Compute this post-diagnostic.

Return type:

PeriodogramDiag

post_compute_from_store(store, group, analyse_type, analyse_date, start, end, fields, delayed)

Compute post-diagnostic using stored data.

Parameters:

  • store (DiagnosticStore) – Store containing diagnostics data.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • delayed (bool) – Flag indicating whether to load from store during computation or at first data request (plot/get_data).

Return type:

PeriodogramDiag

Returns:

Dictionary containing the statistic type with its associated data array.

ref_period: np.timedelta64 | None
results: DiagResultType
stats: list[StatType] | None
storage: StoragePropertiesType = None
store(store, group, analyse_type, analyse_date, start, end, mode, fields)

Write the diagnostic results into the provided store.

Parameters:

  • store (DiagnosticStore) – DiagnosticStore used for the storage.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the computed data.

  • end (datetime64) – Ending date for the computed data.

  • mode (StorageMode) – Storage mode to use when writing data.

  • fields (list[str]) – List of all fields name used in the group of this statistic.

store_configuration(analyse_type, analyse_date, group=None)

Returns the list of storage group parameters required to store this diagnostic.

Parameters:

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date representing the set of data used in this analyse. It’s used to determine at which timestamp to store non-temporal diagnostics.

  • group (str | None) – Name of the group to use for storage.

Return type:

list[StorageGroupParams]

Returns:

List of storage group parameters.

store_params: dict[str, Any] | None = None
temporal_diag: casys_diag_t.TemporalDiag
property used_diags: dict[str, PeriodogramDiag]

Statistic diagnostics used for the computation of the Post diagnostic.

class casys.computation.diagnostics.RatioDiag(name, store_params=None, *, stat_numerator, stat_denominator, total)

Bases: PostDiagnostic

Container for the ratio diagnostic.

Parameters:

  • name (str) – Name of the data.

  • computed – True if the data have been computed, false otherwise.

  • is_post_diag – Whether this diagnostic is a post computation diagnostic or not.

  • stat_numerator (StatisticDiag) – the statistic data we want to calculate the ratio

  • stat_denominator (StatisticDiag) – the statistic data that is the reference for the ratio calculation

  • total (bool) – if true, the calculation is made on the total stat_numerator+stat_denominator, if false, on stat_denominator only

  • store_params (dict[str, Any] | None)

RES_TYPE: ClassVar[type[DiagResult]] = None
SCHEMA: ClassVar[type[mm.Schema]] = None
computed: bool = False
dask_full_compatible: bool = False
html_elem(style='')

HTML representation of the data container.

Parameters:

style (str) – Style to add to the element.

Return type:

str

Returns:

Html representation of elements of this object.

html_header(span)

HTML representation header of the data container.

Parameters:

span (int) – Span size.

Returns:

Object html header.

property identity: DiagnosticType
is_autonomous(dask)

Check whether this diagnostic is autonomous with or without dask enabled.

Parameters:

dask (bool) – Whether dask is enabled or not.

Returns:

Whether this is an autonomous diagnostic or not.

is_dask_compatible(freq)
Parameters:

freq (FrequencyHandler)

Return type:

bool

is_post_diag: bool = False
metadata(internal=False)

Diagnostic serializable configuration.

Parameters:

internal (bool) – Whether this diagnostic is an internal diagnostic or not.

Return type:

DiagnosticMeta

Returns:

Diagnostic metadata.

name: str
post_compute()

Compute this post-diagnostic.

Return type:

StatisticDiag

post_compute_from_store(store, group, analyse_type, analyse_date, start, end, fields, delayed)

Compute post-diagnostic using stored data.

Parameters:

  • store (DiagnosticStore) – Store containing diagnostics data.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • delayed (bool) – Flag indicating whether to load from store during computation or at first data request (plot/get_data).

Return type:

StatisticDiag

Returns:

Dictionary containing the statistic type with its associated data array.

results: DiagResultType
stat_denominator: StatisticDiag
stat_numerator: StatisticDiag
storage: StoragePropertiesType = None
store(store, group, analyse_type, analyse_date, start, end, mode, fields)

Write the diagnostic results into the provided store.

Parameters:

  • store (DiagnosticStore) – DiagnosticStore used for the storage.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the computed data.

  • end (datetime64) – Ending date for the computed data.

  • mode (StorageMode) – Storage mode to use when writing data.

  • fields (list[str]) – List of all fields name used in the group of this statistic.

store_configuration(analyse_type, analyse_date, group=None)

Returns the list of storage group parameters required to store this diagnostic.

Parameters:

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date representing the set of data used in this analyse. It’s used to determine at which timestamp to store non-temporal diagnostics.

  • group (str | None) – Name of the group to use for storage.

Return type:

list[StorageGroupParams]

Returns:

List of storage group parameters.

store_params: dict[str, Any] | None = None
total: bool
property used_diags: dict[str, StatisticDiag]

Statistic diagnostics used for the computation of the Post diagnostic.

class casys.computation.diagnostics.RawComparison3DDiag(name, store_params=None, *, data_index, data_time, start, end, field_x, field_y=None, indexer=None, time_extension=False, field_z)

Bases: RawComparisonDiag

Class representing a raw comparison data.

Parameters:

  • name (str)

  • store_params (dict[str, Any] | None)

  • data_index (str)

  • data_time (str)

  • start (np.datetime64)

  • end (np.datetime64)

  • field_x (Field)

  • field_y (Field | None)

  • indexer (ong_indexer_t.PassIndexer | None)

  • time_extension (bool)

  • field_z (Field)

RES_TYPE

alias of RawDataResult

SCHEMA: ClassVar[type[mm.Schema]] = None
STORABLE: ClassVar[bool] = False
check_stat(stat)

Check whether this data is computed and provided stat is valid. Determines the stat if None is provided.

Parameters:

stat (StatType | None) – Statistic to check.

Return type:

StatType

Returns:

Checked statistic.

Raises:

AltiDataError – If data with the provided name were not computed. If the stat parameter is invalid.

clear_data()

Clear stored data from this Diagnostic data.

compute_from_store(**kwargs)

Compute diagnostic using stored data. The computation can be done in a delayed way allowing to only load requested data when needed.

Parameters:

  • store – Store containing diagnostics data.

  • group – Name of the group to use for storage.

  • analyse_type – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date – Date used to reference stored data.

  • start – Starting date for the data to compute.

  • end – Ending date for the data to compute.

  • fields – List of all fields to compute with this diagnostic properties.

  • delayed – Flag indicating whether to load from store during computation or at first data request (plot/get_data).

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Computation results.

computed: bool = False
dask_full_compatible: bool = False
data_index: str
data_time: str
diag_data(**kwargs)

Diagnostic data as a dataset.

Parameters:

kwargs – Additional parameters required to retrieve diagnostic data.

Return type:

Dataset

Returns:

Dataset containing this diagnostic data.

end: np.datetime64
property ext_end: datetime64
property ext_start: datetime64
property field: Field

Field of the main variable represented by this data.

field_x: Field
field_y: Field | None = None
field_z: Field
property fields: dict[str, Field]

Dictionary of fields used to compute this data.

group: str = None
group_base: str = None
html_elem(style='')

HTML representation of the data container.

Parameters:

style (str) – Style to add to the element.

Return type:

str

Returns:

Html representation of elements of this object.

html_header(span)

HTML representation header of the data container.

Parameters:

span (int) – Span size.

Returns:

Object html header.

property identity: DiagnosticType
indexer: ong_indexer_t.PassIndexer | None = None
initialize()

Diagnostic initialization.

At least the following properties have to be initialized:

  • group_base

  • storage_properties

is_autonomous(dask)

Check whether this diagnostic is autonomous with or without dask enabled.

Parameters:

dask (bool) – Whether dask is enabled or not.

Returns:

Whether this is an autonomous diagnostic or not.

is_dask_compatible(freq)

Check whether this diagnostic can be computed with dask or not.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

Return type:

bool

is_post_diag: bool = False
load_diagnostic(**kwargs)

Ensure requested diagnostic data are loaded.

make_dask_compatible(freq)

Set this container, so it can be used with dask.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

metadata(internal=False)

Diagnostic serializable configuration.

Parameters:

internal (bool) – Whether this diagnostic is an internal diagnostic or not.

Return type:

DiagnosticMeta

Returns:

Diagnostic metadata.

name: str
plot_container(template=None, stat=None, plot=None)

Create a PlotContainer for the provided stat data.

Parameters:

  • template (PlotTemplate | None) – Parameters template to use.

  • stat (StatType | None) – Statistic included in this data.

  • plot (str | None) – Plot type to display (“TIME”, “MAP” or “3D”).

Return type:

PlotContainer

Returns:

PlotContainer containing the requested data.

results: DiagResultType
start: np.datetime64
stats: list[StatType]
storage: StoragePropertiesType = None
store(store, group, analyse_type, analyse_date, start, end, mode, fields)

Write the diagnostic results into the provided store.

Parameters:

  • store (DiagnosticStore) – DiagnosticStore used for the storage.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the computed data.

  • end (datetime64) – Ending date for the computed data.

  • mode (StorageMode) – Storage mode to use when writing data.

  • fields (list[str]) – List of all fields name used in the group of this statistic.

store_configuration(analyse_type, analyse_date, group=None)

Returns the list of storage group parameters required to store this diagnostic.

Parameters:

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date representing the set of data used in this analyse. It’s used to determine at which timestamp to store non-temporal diagnostics.

  • group (str | None) – Name of the group to use for storage.

Return type:

list[StorageGroupParams]

Returns:

List of storage group parameters.

store_params: dict[str, Any] | None = None
time_extension: bool = False
property variable: str

Name of the main variable represented by this data.

class casys.computation.diagnostics.RawComparison3DSwathDiag(name, store_params=None, field_pixel=None, *, data_index, data_time, start, end, field_x, field_y=None, indexer=None, time_extension=False, field_z)

Bases: RawComparison3DDiag

Class representing a 3D (x,y,z) raw comparison data.

Parameters:

  • name (str)

  • store_params (dict[str, Any] | None)

  • field_pixel (Field)

  • data_index (str)

  • data_time (str)

  • start (np.datetime64)

  • end (np.datetime64)

  • field_x (Field)

  • field_y (Field | None)

  • indexer (ong_indexer_t.PassIndexer | None)

  • time_extension (bool)

  • field_z (Field)

RES_TYPE

alias of RawDataResult

SCHEMA: ClassVar[type[mm.Schema]] = None
STORABLE: ClassVar[bool] = False
check_stat(stat)

Check whether this data is computed and provided stat is valid. Determines the stat if None is provided.

Parameters:

stat (StatType | None) – Statistic to check.

Return type:

StatType

Returns:

Checked statistic.

Raises:

AltiDataError – If data with the provided name were not computed. If the stat parameter is invalid.

clear_data()

Clear stored data from this Diagnostic data.

compute_from_store(**kwargs)

Compute diagnostic using stored data. The computation can be done in a delayed way allowing to only load requested data when needed.

Parameters:

  • store – Store containing diagnostics data.

  • group – Name of the group to use for storage.

  • analyse_type – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date – Date used to reference stored data.

  • start – Starting date for the data to compute.

  • end – Ending date for the data to compute.

  • fields – List of all fields to compute with this diagnostic properties.

  • delayed – Flag indicating whether to load from store during computation or at first data request (plot/get_data).

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Computation results.

computed: bool = False
dask_full_compatible: bool = False
data_index: str
data_time: str
diag_data(**kwargs)

Diagnostic data as a dataset.

Parameters:

kwargs – Additional parameters required to retrieve diagnostic data.

Return type:

Dataset

Returns:

Dataset containing this diagnostic data.

end: np.datetime64
property ext_end: datetime64
property ext_start: datetime64
property field: Field

Field of the main variable represented by this data.

field_pixel: Field = None
field_x: Field
field_y: Field | None = None
field_z: Field
property fields: dict[str, Field]

Dictionary of fields used to compute this data.

group: str = None
group_base: str = None
html_elem(style='')

HTML representation of the data container.

Parameters:

style (str) – Style to add to the element.

Return type:

str

Returns:

Html representation of elements of this object.

html_header(span)

HTML representation header of the data container.

Parameters:

span (int) – Span size.

Returns:

Object html header.

property identity: DiagnosticType
indexer: ong_indexer_t.PassIndexer | None = None
initialize()

Diagnostic initialization.

At least the following properties have to be initialized:

  • group_base

  • storage_properties

is_autonomous(dask)

Check whether this diagnostic is autonomous with or without dask enabled.

Parameters:

dask (bool) – Whether dask is enabled or not.

Returns:

Whether this is an autonomous diagnostic or not.

is_dask_compatible(freq)

Check whether this diagnostic can be computed with dask or not.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

Return type:

bool

is_post_diag: bool = False
load_diagnostic(**kwargs)

Ensure requested diagnostic data are loaded.

make_dask_compatible(freq)

Set this container, so it can be used with dask.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

metadata(internal=False)

Diagnostic serializable configuration.

Parameters:

internal (bool) – Whether this diagnostic is an internal diagnostic or not.

Return type:

DiagnosticMeta

Returns:

Diagnostic metadata.

name: str
plot_container(stat=None, template=None, plot=None)

Create a PlotContainer for the provided stat data.

Parameters:

  • template (PlotTemplate | None) – Parameters template to use.

  • stat (StatType | None) – Statistic included in this data.

  • plot (str | None) – Plot type to display (“2D:n_pix” or “3D:n_pix”, with n_pix the number of the pixel to split the displayed data at).

Return type:

PlotContainer

Returns:

PlotContainer containing the requested data.

results: DiagResultType
start: np.datetime64
stats: list[StatType]
storage: StoragePropertiesType = None
store(store, group, analyse_type, analyse_date, start, end, mode, fields)

Write the diagnostic results into the provided store.

Parameters:

  • store (DiagnosticStore) – DiagnosticStore used for the storage.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the computed data.

  • end (datetime64) – Ending date for the computed data.

  • mode (StorageMode) – Storage mode to use when writing data.

  • fields (list[str]) – List of all fields name used in the group of this statistic.

store_configuration(analyse_type, analyse_date, group=None)

Returns the list of storage group parameters required to store this diagnostic.

Parameters:

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date representing the set of data used in this analyse. It’s used to determine at which timestamp to store non-temporal diagnostics.

  • group (str | None) – Name of the group to use for storage.

Return type:

list[StorageGroupParams]

Returns:

List of storage group parameters.

store_params: dict[str, Any] | None = None
time_extension: bool = False
property variable: str

Name of the main variable represented by this data.

class casys.computation.diagnostics.RawComparisonDiag(name, store_params=None, *, data_index, data_time, start, end, field_x, field_y=None, indexer=None, time_extension=False)

Bases: Diagnostic[RawDataResult, StorageProperties]

Class representing a raw comparison data.

Parameters:

  • name (str)

  • store_params (dict[str, Any] | None)

  • data_index (str)

  • data_time (str)

  • start (np.datetime64)

  • end (np.datetime64)

  • field_x (Field)

  • field_y (Field | None)

  • indexer (ong_indexer_t.PassIndexer | None)

  • time_extension (bool)

RES_TYPE

alias of RawDataResult

SCHEMA: ClassVar[type[mm.Schema]] = None
STORABLE: ClassVar[bool] = False
check_stat(stat)

Check whether this data is computed and provided stat is valid. Determines the stat if None is provided.

Parameters:

stat (StatType | None) – Statistic to check.

Return type:

StatType

Returns:

Checked statistic.

Raises:

AltiDataError – If data with the provided name were not computed. If the stat parameter is invalid.

clear_data()

Clear stored data from this Diagnostic data.

compute_from_store(**kwargs)

Compute diagnostic using stored data. The computation can be done in a delayed way allowing to only load requested data when needed.

Parameters:

  • store – Store containing diagnostics data.

  • group – Name of the group to use for storage.

  • analyse_type – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date – Date used to reference stored data.

  • start – Starting date for the data to compute.

  • end – Ending date for the data to compute.

  • fields – List of all fields to compute with this diagnostic properties.

  • delayed – Flag indicating whether to load from store during computation or at first data request (plot/get_data).

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Computation results.

computed: bool = False
dask_full_compatible: bool = False
data_index: str
data_time: str
diag_data(**kwargs)

Diagnostic data as a dataset.

Parameters:

kwargs – Additional parameters required to retrieve diagnostic data.

Return type:

Dataset

Returns:

Dataset containing this diagnostic data.

end: np.datetime64
property ext_end: datetime64
property ext_start: datetime64
property field: Field

Field of the main variable represented by this data.

field_x: Field
field_y: Field | None = None
property fields: dict[str, Field]

Dictionary of fields used to compute this data.

group: str = None
group_base: str = None
html_elem(style='')

HTML representation of the data container.

Parameters:

style (str) – Style to add to the element.

Return type:

str

Returns:

Html representation of elements of this object.

html_header(span)

HTML representation header of the data container.

Parameters:

span (int) – Span size.

Returns:

Object html header.

property identity: DiagnosticType
indexer: ong_indexer_t.PassIndexer | None = None
initialize()

Diagnostic initialization.

At least the following properties have to be initialized:

  • group_base

  • storage_properties

is_autonomous(dask)

Check whether this diagnostic is autonomous with or without dask enabled.

Parameters:

dask (bool) – Whether dask is enabled or not.

Returns:

Whether this is an autonomous diagnostic or not.

is_dask_compatible(freq)

Check whether this diagnostic can be computed with dask or not.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

Return type:

bool

is_post_diag: bool = False
load_diagnostic(**kwargs)

Ensure requested diagnostic data are loaded.

make_dask_compatible(freq)

Set this container, so it can be used with dask.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

metadata(internal=False)

Diagnostic serializable configuration.

Parameters:

internal (bool) – Whether this diagnostic is an internal diagnostic or not.

Return type:

DiagnosticMeta

Returns:

Diagnostic metadata.

name: str
plot_container(template=None, stat=None)

Create a PlotContainer for the provided stat data.

Parameters:
Return type:

PlotContainer

Returns:

PlotContainer containing the requested data.

results: DiagResultType
start: np.datetime64
stats: list[StatType]
storage: StoragePropertiesType = None
store(store, group, analyse_type, analyse_date, start, end, mode, fields)

Write the diagnostic results into the provided store.

Parameters:

  • store (DiagnosticStore) – DiagnosticStore used for the storage.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the computed data.

  • end (datetime64) – Ending date for the computed data.

  • mode (StorageMode) – Storage mode to use when writing data.

  • fields (list[str]) – List of all fields name used in the group of this statistic.

store_configuration(analyse_type, analyse_date, group=None)

Returns the list of storage group parameters required to store this diagnostic.

Parameters:

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date representing the set of data used in this analyse. It’s used to determine at which timestamp to store non-temporal diagnostics.

  • group (str | None) – Name of the group to use for storage.

Return type:

list[StorageGroupParams]

Returns:

List of storage group parameters.

store_params: dict[str, Any] | None = None
time_extension: bool = False
property variable: str

Name of the main variable represented by this data.

class casys.computation.diagnostics.RawComparisonSwathDiag(name, store_params=None, field_pixel=None, *, data_index, data_time, start, end, field_x, field_y=None, indexer=None, time_extension=False)

Bases: RawComparisonDiag

Class representing a 2D (x,y) raw comparison data.

Parameters:

  • name (str)

  • store_params (dict[str, Any] | None)

  • field_pixel (Field)

  • data_index (str)

  • data_time (str)

  • start (np.datetime64)

  • end (np.datetime64)

  • field_x (Field)

  • field_y (Field | None)

  • indexer (ong_indexer_t.PassIndexer | None)

  • time_extension (bool)

RES_TYPE

alias of RawDataResult

SCHEMA: ClassVar[type[mm.Schema]] = None
STORABLE: ClassVar[bool] = False
check_stat(stat)

Check whether this data is computed and provided stat is valid. Determines the stat if None is provided.

Parameters:

stat (StatType | None) – Statistic to check.

Return type:

StatType

Returns:

Checked statistic.

Raises:

AltiDataError – If data with the provided name were not computed. If the stat parameter is invalid.

clear_data()

Clear stored data from this Diagnostic data.

compute_from_store(**kwargs)

Compute diagnostic using stored data. The computation can be done in a delayed way allowing to only load requested data when needed.

Parameters:

  • store – Store containing diagnostics data.

  • group – Name of the group to use for storage.

  • analyse_type – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date – Date used to reference stored data.

  • start – Starting date for the data to compute.

  • end – Ending date for the data to compute.

  • fields – List of all fields to compute with this diagnostic properties.

  • delayed – Flag indicating whether to load from store during computation or at first data request (plot/get_data).

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Computation results.

computed: bool = False
dask_full_compatible: bool = False
data_index: str
data_time: str
diag_data(**kwargs)

Diagnostic data as a dataset.

Parameters:

kwargs – Additional parameters required to retrieve diagnostic data.

Return type:

Dataset

Returns:

Dataset containing this diagnostic data.

end: np.datetime64
property ext_end: datetime64
property ext_start: datetime64
property field: Field

Field of the main variable represented by this data.

field_pixel: Field = None
field_x: Field
field_y: Field | None = None
property fields: dict[str, Field]

Dictionary of fields used to compute this data.

group: str = None
group_base: str = None
html_elem(style='')

HTML representation of the data container.

Parameters:

style (str) – Style to add to the element.

Return type:

str

Returns:

Html representation of elements of this object.

html_header(span)

HTML representation header of the data container.

Parameters:

span (int) – Span size.

Returns:

Object html header.

property identity: DiagnosticType
indexer: ong_indexer_t.PassIndexer | None = None
initialize()

Diagnostic initialization.

At least the following properties have to be initialized:

  • group_base

  • storage_properties

is_autonomous(dask)

Check whether this diagnostic is autonomous with or without dask enabled.

Parameters:

dask (bool) – Whether dask is enabled or not.

Returns:

Whether this is an autonomous diagnostic or not.

is_dask_compatible(freq)

Check whether this diagnostic can be computed with dask or not.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

Return type:

bool

is_post_diag: bool = False
load_diagnostic(**kwargs)

Ensure requested diagnostic data are loaded.

make_dask_compatible(freq)

Set this container, so it can be used with dask.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

metadata(internal=False)

Diagnostic serializable configuration.

Parameters:

internal (bool) – Whether this diagnostic is an internal diagnostic or not.

Return type:

DiagnosticMeta

Returns:

Diagnostic metadata.

name: str
plot_container(template=None, stat=None)

Create a PlotContainer for the provided stat data.

Parameters:
Return type:

PlotContainer

Returns:

PlotContainer containing the requested data.

results: DiagResultType
start: np.datetime64
stats: list[StatType]
storage: StoragePropertiesType = None
store(store, group, analyse_type, analyse_date, start, end, mode, fields)

Write the diagnostic results into the provided store.

Parameters:

  • store (DiagnosticStore) – DiagnosticStore used for the storage.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the computed data.

  • end (datetime64) – Ending date for the computed data.

  • mode (StorageMode) – Storage mode to use when writing data.

  • fields (list[str]) – List of all fields name used in the group of this statistic.

store_configuration(analyse_type, analyse_date, group=None)

Returns the list of storage group parameters required to store this diagnostic.

Parameters:

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date representing the set of data used in this analyse. It’s used to determine at which timestamp to store non-temporal diagnostics.

  • group (str | None) – Name of the group to use for storage.

Return type:

list[StorageGroupParams]

Returns:

List of storage group parameters.

store_params: dict[str, Any] | None = None
time_extension: bool = False
property variable: str

Name of the main variable represented by this data.

class casys.computation.diagnostics.RawDataResult(index)

Bases: DiagResult

Raw data diagnostic results.

Parameters:

index (str)

add_data(data)

Add provided data to this object.

Parameters:

data (Dataset) – Data to add.

add_field(name, values)

Add a new field to existing data. If the field already exists, its values are updated.

Parameters:

  • name (str) – Name of the field.

  • values (ndarray) – Values of the field.

clear()
property data: Dataset
property data_rs: Dataset
drop_fields(fields)

Drop provided fields from existing data.

Parameters:

fields (set[str] | list[str]) – Name of the fields.

have_data()

Whether this result contains data or not.

Return type:

bool

Returns:

True if this result has data, false otherwise.

index: str
remove_data(field=None, stats=None)

Remove results’ data from the stat of the given field.

Parameters:

  • field (str) – Name of the field

  • stats (list[StatType]) – Statistics to remove.

set_data(data)

Set current object data.

Parameters:

data (Dataset) – Data to set.

statistics()

List of statistics computed for this diagnostic.

Return type:

list[StatType]

property variables: list[str | Hashable]

List of read fields.

class casys.computation.diagnostics.RawDiag(name, store_params=None, *, data_index, data_time, start, end, field_x, field_y=None, indexer=None, time_extension=False, field_z, time)

Bases: Diagnostic[RawDataResult, StorageProperties]

Class representing a raw data associated to its time, longitude and latitude.

Parameters:

  • name (str) – Name of the data.

  • computed – True if the data have been computed, false otherwise.

  • is_post_diag – Whether this diagnostic is a post computation diagnostic or not.

  • data_index (str) – Name of the field on which the data is indexed.

  • start (np.datetime64) – Starting date of the input dataset.

  • end (np.datetime64) – Ending date of the input dataset.

  • field_x (Field) – Field used for x axes.

  • field_y (Field | None) – Field used for y axes.

  • stats – Statistics included in this Diagnostic.

  • indexer (ong_indexer_t.PassIndexer | None) – Indexer used to read data.

  • time_extension (bool) – Whether time extension is enabled or not.

  • ext_start – Extended starting date required by the computation.

  • ext_end – Ending date for the data required by the computation.

  • field_z (Field) – Field used for z axes.

  • time (Field) – Field used for the time.

  • store_params (dict[str, Any] | None)

  • data_time (str)

RES_TYPE

alias of RawDataResult

SCHEMA: ClassVar[type[mm.Schema]] = None
STORABLE: ClassVar[bool] = False
check_stat(stat)

Check whether this data is computed and provided stat is valid. Determines the stat if None is provided.

Parameters:

stat (StatType | None) – Statistic to check.

Return type:

StatType

Returns:

Checked statistic.

Raises:

AltiDataError – If data with the provided name were not computed. If the stat parameter is invalid.

clear_data()

Clear stored data from this Diagnostic data.

compute_from_store(**kwargs)

Compute diagnostic using stored data. The computation can be done in a delayed way allowing to only load requested data when needed.

Parameters:

  • store – Store containing diagnostics data.

  • group – Name of the group to use for storage.

  • analyse_type – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date – Date used to reference stored data.

  • start – Starting date for the data to compute.

  • end – Ending date for the data to compute.

  • fields – List of all fields to compute with this diagnostic properties.

  • delayed – Flag indicating whether to load from store during computation or at first data request (plot/get_data).

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Computation results.

computed: bool = False
dask_full_compatible: bool = False
data_index: str
data_time: str
diag_data(**kwargs)

Diagnostic data as a dataset.

Parameters:

kwargs – Additional parameters required to retrieve diagnostic data.

Return type:

Dataset

Returns:

Dataset containing this diagnostic data.

end: np.datetime64
property ext_end: datetime64
property ext_start: datetime64
property field: Field

Field of the main variable represented by this data.

field_x: Field
field_y: Field | None = None
field_z: Field
property fields: dict[str, Field]

Dictionary of fields used to compute this data.

group: str = None
group_base: str = None
html_elem(style='')

HTML representation of the data container.

Parameters:

style (str) – Style to add to the element.

Return type:

str

Returns:

Html representation of elements of this object.

html_header(span)

HTML representation header of the data container.

Parameters:

span (int) – Span size.

Returns:

Object html header.

property identity: DiagnosticType
indexer: ong_indexer_t.PassIndexer | None = None
initialize()

Diagnostic initialization.

At least the following properties have to be initialized:

  • group_base

  • storage_properties

is_autonomous(dask)

Check whether this diagnostic is autonomous with or without dask enabled.

Parameters:

dask (bool) – Whether dask is enabled or not.

Returns:

Whether this is an autonomous diagnostic or not.

is_dask_compatible(freq)

Check whether this diagnostic can be computed with dask or not.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

Return type:

bool

is_post_diag: bool = False
load_diagnostic(**kwargs)

Ensure requested diagnostic data are loaded.

make_dask_compatible(freq)

Set this container, so it can be used with dask.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

metadata(internal=False)

Diagnostic serializable configuration.

Parameters:

internal (bool) – Whether this diagnostic is an internal diagnostic or not.

Return type:

DiagnosticMeta

Returns:

Diagnostic metadata.

name: str
plot_container(template=None, stat=None, plot=None)

Create a PlotContainer for the provided stat data.

Parameters:

  • template (PlotTemplate | None) – Parameters template to use.

  • stat (StatType | None) – Statistic included in this data.

  • plot (str | None) – Plot type to display (“TIME”, “MAP” or “3D”).

Return type:

PlotContainer

Returns:

PlotContainer containing the requested data.

results: DiagResultType
start: np.datetime64
stats: list[StatType]
storage: StoragePropertiesType = None
store(store, group, analyse_type, analyse_date, start, end, mode, fields)

Write the diagnostic results into the provided store.

Parameters:

  • store (DiagnosticStore) – DiagnosticStore used for the storage.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the computed data.

  • end (datetime64) – Ending date for the computed data.

  • mode (StorageMode) – Storage mode to use when writing data.

  • fields (list[str]) – List of all fields name used in the group of this statistic.

store_configuration(analyse_type, analyse_date, group=None)

Returns the list of storage group parameters required to store this diagnostic.

Parameters:

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date representing the set of data used in this analyse. It’s used to determine at which timestamp to store non-temporal diagnostics.

  • group (str | None) – Name of the group to use for storage.

Return type:

list[StorageGroupParams]

Returns:

List of storage group parameters.

store_params: dict[str, Any] | None = None
time: Field
time_extension: bool = False
property variable: str

Name of the main variable represented by this data.

class casys.computation.diagnostics.RawGenerator(name, store_params=None, *, data_index, data_time, start, end, field_x, field_y=None, indexer=None, time_extension=False, field_z, time)

Bases: RawDiag, ABC

Abstract class that represents a raw data generator.

Parameters:

  • name (str) – Name of the data.

  • computed – True if the data have been computed, false otherwise.

  • is_post_diag – Whether this diagnostic is a post computation diagnostic or not.

  • data_index (str) – Name of the field on which the data is indexed.

  • start (np.datetime64) – Starting date of the input dataset.

  • end (np.datetime64) – Ending date of the input dataset.

  • field_x (Field) – Field used for x axes.

  • field_y (Field | None) – Field used for y axes.

  • stats – Statistics included in this Diagnostic.

  • indexer (ong_indexer_t.PassIndexer | None) – Indexer used to read data.

  • time_extension (bool) – Whether time extension is enabled or not.

  • ext_start – Extended starting date required by the computation.

  • ext_end – Ending date for the data required by the computation.

  • field_z (Field) – Field used for z axes.

  • time (Field) – Field used for the time.

  • store_params (dict[str, Any] | None)

  • data_time (str)

RES_TYPE

alias of RawDataResult

SCHEMA: ClassVar[type[mm.Schema]] = None
STORABLE: ClassVar[bool] = False
check_stat(stat)

Check whether this data is computed and provided stat is valid. Determines the stat if None is provided.

Parameters:

stat (StatType | None) – Statistic to check.

Return type:

StatType

Returns:

Checked statistic.

Raises:

AltiDataError – If data with the provided name were not computed. If the stat parameter is invalid.

clear_data()

Clear stored data from this Diagnostic data.

abstract compute_data(data)

Read data and compute statistics.

Parameters:

data (Dataset) – The dataset with data to read.

Return type:

Dataset

Returns:

Dataset including the newly computed data.

compute_from_store(**kwargs)

Compute diagnostic using stored data. The computation can be done in a delayed way allowing to only load requested data when needed.

Parameters:

  • store – Store containing diagnostics data.

  • group – Name of the group to use for storage.

  • analyse_type – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date – Date used to reference stored data.

  • start – Starting date for the data to compute.

  • end – Ending date for the data to compute.

  • fields – List of all fields to compute with this diagnostic properties.

  • delayed – Flag indicating whether to load from store during computation or at first data request (plot/get_data).

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Computation results.

computed: bool = False
dask_full_compatible: bool = False
data_index: str
data_time: str
diag_data(**kwargs)

Diagnostic data as a dataset.

Parameters:

kwargs – Additional parameters required to retrieve diagnostic data.

Return type:

Dataset

Returns:

Dataset containing this diagnostic data.

end: np.datetime64
property ext_end: datetime64
property ext_start: datetime64
property field: Field

Field of the main variable represented by this data.

field_x: Field
field_y: Field | None = None
field_z: Field
property fields: dict[str, Field]

Dictionary of fields used to compute this data.

group: str = None
group_base: str = None
html_elem(style='')

HTML representation of the data container.

Parameters:

style (str) – Style to add to the element.

Return type:

str

Returns:

Html representation of elements of this object.

html_header(span)

HTML representation header of the data container.

Parameters:

span (int) – Span size.

Returns:

Object html header.

property identity: DiagnosticType
indexer: ong_indexer_t.PassIndexer | None = None
initialize()

Diagnostic initialization.

At least the following properties have to be initialized:

  • group_base

  • storage_properties

is_autonomous(dask)

Check whether this diagnostic is autonomous with or without dask enabled.

Parameters:

dask (bool) – Whether dask is enabled or not.

Returns:

Whether this is an autonomous diagnostic or not.

is_dask_compatible(freq)

Check whether this diagnostic can be computed with dask or not.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

Return type:

bool

is_post_diag: bool = False
load_diagnostic(**kwargs)

Ensure requested diagnostic data are loaded.

make_dask_compatible(freq)

Set this container, so it can be used with dask.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

metadata(internal=False)

Diagnostic serializable configuration.

Parameters:

internal (bool) – Whether this diagnostic is an internal diagnostic or not.

Return type:

DiagnosticMeta

Returns:

Diagnostic metadata.

name: str
plot_container(template=None, stat=None, plot=None)

Create a PlotContainer for the provided stat data.

Parameters:

  • template (PlotTemplate | None) – Parameters template to use.

  • stat (StatType | None) – Statistic included in this data.

  • plot (str | None) – Plot type to display (“TIME”, “MAP” or “3D”).

Return type:

PlotContainer

Returns:

PlotContainer containing the requested data.

results: DiagResultType
start: np.datetime64
stats: list[StatType]
storage: StoragePropertiesType = None
store(store, group, analyse_type, analyse_date, start, end, mode, fields)

Write the diagnostic results into the provided store.

Parameters:

  • store (DiagnosticStore) – DiagnosticStore used for the storage.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the computed data.

  • end (datetime64) – Ending date for the computed data.

  • mode (StorageMode) – Storage mode to use when writing data.

  • fields (list[str]) – List of all fields name used in the group of this statistic.

store_configuration(analyse_type, analyse_date, group=None)

Returns the list of storage group parameters required to store this diagnostic.

Parameters:

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date representing the set of data used in this analyse. It’s used to determine at which timestamp to store non-temporal diagnostics.

  • group (str | None) – Name of the group to use for storage.

Return type:

list[StorageGroupParams]

Returns:

List of storage group parameters.

store_params: dict[str, Any] | None = None
time: Field
time_extension: bool = False
property variable: str

Name of the main variable represented by this data.

class casys.computation.diagnostics.RawSwathDiag(name, store_params=None, *, data_index, data_time, start, end, field_x, field_y=None, indexer=None, time_extension=False, field_z, time, field_pixel)

Bases: RawDiag

Class representing a Swath raw data associated to its time, longitude and latitude.

Parameters:

  • name (str) – Name of the data.

  • computed – True if the data have been computed, false otherwise.

  • is_post_diag – Whether this diagnostic is a post computation diagnostic or not.

  • data_index (str) – Name of the field on which the data is indexed.

  • start (np.datetime64) – Starting date of the input dataset.

  • end (np.datetime64) – Ending date of the input dataset.

  • field_x (Field) – Field used for x axes.

  • field_y (Field | None) – Field used for y axes.

  • stats – Statistics included in this Diagnostic.

  • indexer (ong_indexer_t.PassIndexer | None) – Indexer used to read data.

  • time_extension (bool) – Whether time extension is enabled or not.

  • ext_start – Extended starting date required by the computation.

  • ext_end – Ending date for the data required by the computation.

  • field_z (Field) – Field used for z axes.

  • time (Field) – Field used for the time.

  • field_pixel (Field) – Field containing pixel information.

  • store_params (dict[str, Any] | None)

  • data_time (str)

RES_TYPE

alias of RawDataResult

SCHEMA: ClassVar[type[mm.Schema]] = None
STORABLE: ClassVar[bool] = False
check_stat(stat)

Check whether this data is computed and provided stat is valid. Determines the stat if None is provided.

Parameters:

stat (StatType | None) – Statistic to check.

Return type:

StatType

Returns:

Checked statistic.

Raises:

AltiDataError – If data with the provided name were not computed. If the stat parameter is invalid.

clear_data()

Clear stored data from this Diagnostic data.

compute_from_store(**kwargs)

Compute diagnostic using stored data. The computation can be done in a delayed way allowing to only load requested data when needed.

Parameters:

  • store – Store containing diagnostics data.

  • group – Name of the group to use for storage.

  • analyse_type – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date – Date used to reference stored data.

  • start – Starting date for the data to compute.

  • end – Ending date for the data to compute.

  • fields – List of all fields to compute with this diagnostic properties.

  • delayed – Flag indicating whether to load from store during computation or at first data request (plot/get_data).

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Computation results.

computed: bool = False
dask_full_compatible: bool = False
data_index: str
data_time: str
diag_data(**kwargs)

Diagnostic data as a dataset.

Parameters:

kwargs – Additional parameters required to retrieve diagnostic data.

Return type:

Dataset

Returns:

Dataset containing this diagnostic data.

end: np.datetime64
property ext_end: datetime64
property ext_start: datetime64
property field: Field

Field of the main variable represented by this data.

field_pixel: Field
field_x: Field
field_y: Field | None = None
field_z: Field
property fields: dict[str, Field]

Dictionary of fields used to compute this data.

group: str = None
group_base: str = None
html_elem(style='')

HTML representation of the data container.

Parameters:

style (str) – Style to add to the element.

Return type:

str

Returns:

Html representation of elements of this object.

html_header(span)

HTML representation header of the data container.

Parameters:

span (int) – Span size.

Returns:

Object html header.

property identity: DiagnosticType
indexer: ong_indexer_t.PassIndexer | None = None
initialize()

Diagnostic initialization.

At least the following properties have to be initialized:

  • group_base

  • storage_properties

is_autonomous(dask)

Check whether this diagnostic is autonomous with or without dask enabled.

Parameters:

dask (bool) – Whether dask is enabled or not.

Returns:

Whether this is an autonomous diagnostic or not.

is_dask_compatible(freq)

Check whether this diagnostic can be computed with dask or not.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

Return type:

bool

is_post_diag: bool = False
load_diagnostic(**kwargs)

Ensure requested diagnostic data are loaded.

make_dask_compatible(freq)

Set this container, so it can be used with dask.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

metadata(internal=False)

Diagnostic serializable configuration.

Parameters:

internal (bool) – Whether this diagnostic is an internal diagnostic or not.

Return type:

DiagnosticMeta

Returns:

Diagnostic metadata.

name: str
plot_container(template=None, stat=None, plot=None)

Create a PlotContainer for the provided stat data.

Parameters:

  • template (PlotTemplate | None) – Parameters template to use.

  • stat (StatType | None) – Statistic included in this data.

  • plot (str | None) – Plot type to display (“TIME:n_pix”, “MAP:n_pix” or “3D:n_pix”, with n_pix the number of the pixel to split the displayed data at).

Return type:

PlotContainer

Returns:

PlotContainer containing the requested data.

results: DiagResultType
start: np.datetime64
stats: list[StatType]
storage: StoragePropertiesType = None
store(store, group, analyse_type, analyse_date, start, end, mode, fields)

Write the diagnostic results into the provided store.

Parameters:

  • store (DiagnosticStore) – DiagnosticStore used for the storage.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the computed data.

  • end (datetime64) – Ending date for the computed data.

  • mode (StorageMode) – Storage mode to use when writing data.

  • fields (list[str]) – List of all fields name used in the group of this statistic.

store_configuration(analyse_type, analyse_date, group=None)

Returns the list of storage group parameters required to store this diagnostic.

Parameters:

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date representing the set of data used in this analyse. It’s used to determine at which timestamp to store non-temporal diagnostics.

  • group (str | None) – Name of the group to use for storage.

Return type:

list[StorageGroupParams]

Returns:

List of storage group parameters.

store_params: dict[str, Any] | None = None
time: Field
time_extension: bool = False
property variable: str

Name of the main variable represented by this data.

class casys.computation.diagnostics.ScatterDiag(name, store_params=None, *, data_index, data_time, start, end, field_x, field_y=None, indexer=None, time_extension=False, stats=<factory>, res_x, res_y)

Bases: CountDiag[CountResult, ScatterStorageProperties]

Class representing a scatter statistic.

Parameters:

  • name (str) – Name of the data.

  • computed – True if the data have been computed, false otherwise.

  • is_post_diag – Whether this diagnostic is a post computation diagnostic or not.

  • data_index (str) – Name of the field on which the data is indexed.

  • start (np.datetime64) – Starting date of the input dataset.

  • end (np.datetime64) – Ending date of the input dataset.

  • field_x (Field) – Field used for x axes.

  • field_y (Field | None) – Field used for y axes.

  • stats (list[StatType]) – Statistics included in this Diagnostic.

  • indexer (ong_indexer_t.PassIndexer | None) – Indexer used to read data.

  • time_extension (bool) – Whether time extension is enabled or not.

  • ext_start – Extended starting date required by the computation.

  • ext_end – Ending date for the data required by the computation.

  • group – the name of the group of this diagnostic.

  • field_z – Field used for z axes

  • res_x (DataResolution) – Min, max and width for the x-axis.

  • res_y (DataResolution) – Min, max and width for the y-axis.

  • store_params (dict[str, Any] | None)

  • data_time (str)

MERGEABLE_STAT: ClassVar[list[StatType]] = [MIN, MAX, COUNT, MEAN, VAR, STD, PROD, SUM]
RES_TYPE

alias of CountResult

SCHEMA: ClassVar[type[mm.Schema]] = None
STORABLE: ClassVar[bool] = True
apply_stats_selection(diagnostic, fields)

Filter diagnostic by applying the statistic selection clip and removing unwanted statistics.

Parameters:

  • diagnostic (TypeVar(DiagResultType, bound= DiagResult)) – Diagnostic to filter.

  • fields (list[str]) – List of diagnostic field’s names.

Return type:

TypeVar(DiagResultType, bound= DiagResult)

check_stat(stat)

Check whether this data is computed and provided stat is valid. Determines the stat if None is provided.

Parameters:

stat (StatType | None) – Statistic to check.

Return type:

StatType

Returns:

Checked statistic.

Raises:

AltiDataError – If data with the provided name were not computed. If the stat parameter is invalid.

clear_data()

Clear stored data from this Diagnostic data.

compute(data, start, end, fields, diags)

Compute the diagnostic on the provided data.

Parameters:

  • data (Dataset) – The dataset with data to read.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • diags (list[StatisticDiag]) – List of diagnostics included in this computation.

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Dictionary containing the statistic type with its associated data array.

compute_from_store(store, group, analyse_type, analyse_date, start, end, fields, delayed)

Compute diagnostic using stored data. The computation can be done in a delayed way allowing to only load requested data when needed.

Parameters:

  • store (DiagnosticStore) – Store containing diagnostics data.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • delayed (bool) – Flag indicating whether to load from store during computation or at first data request (plot/get_data).

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Computation results.

computed: bool = False
dask_full_compatible: bool = False
data_index: str
data_time: str
diag_data(**kwargs)

Diagnostic data as a dataset.

Parameters:

kwargs – Additional parameters required to retrieve diagnostic data.

Return type:

Dataset

Returns:

Dataset containing this diagnostic data.

end: np.datetime64
property ext_end: datetime64
property ext_start: datetime64
property field: Field

Field of the main variable represented by this data.

field_x: Field
field_y: Field | None = None
field_z: Field
property fields: dict[str, Field]

Dictionary of fields used to compute this data.

finalize(result)

Correctly format the diagnostic result.

Parameters:

result (TypeVar(CountResultType, bound= CountResult)) – Result to finalize.

Return type:

TypeVar(CountResultType, bound= CountResult)

Returns:

Finalized result.

static grid_2d(res_x, res_y)

Generate a Grid2D from provided parameters.

Parameters:
Return type:

Grid2D

Returns:

Grid2D with provided resolutions.

group: str = None
group_base: str = None
html_elem(style='')

HTML representation of the data container.

Parameters:

style (str) – Style to add to the element.

Return type:

str

Returns:

Html representation of elements of this object.

html_header(span)

HTML representation header of the data container.

Parameters:

span (int) – Span size.

Returns:

Object html header.

property identity: DiagnosticType
indexer: ong_indexer_t.PassIndexer | None = None
initialize()

Diagnostic initialization.

At least the following properties have to be initialized:

  • group_base

  • storage_properties

is_autonomous(dask)

Check whether this diagnostic is autonomous with or without dask enabled.

Parameters:

dask (bool) – Whether dask is enabled or not.

Returns:

Whether this is an autonomous diagnostic or not.

is_dask_compatible(freq)

Check whether this diagnostic can be computed with dask or not.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

Return type:

bool

is_post_diag: bool = False
load_diagnostic(**kwargs)

Ensure requested diagnostic data are loaded.

make_dask_compatible(freq)

Set this container, so it can be used with dask.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

merge(diagnostics)

Merge provided diagnostics.

Parameters:

diagnostics (list[TypeVar(CountResultType, bound= CountResult)]) – Diagnostics to merge.

Return type:

TypeVar(CountResultType, bound= CountResult)

Returns:

Merged diagnostic.

metadata(internal=False)

Diagnostic serializable configuration.

Parameters:

internal (bool) – Whether this diagnostic is an internal diagnostic or not.

Return type:

DiagnosticMeta

Returns:

Diagnostic metadata.

name: str
new_algorithm()

Instantiate the algorithm.

Return type:

Dispersion2D

Returns:

Instantiated algorithm.

plot_container(template=None, stat=None)

Create a PlotContainer for the provided stat data.

Parameters:
Return type:

PlotContainer

Returns:

PlotContainer containing the requested data.

res_x: DataResolution
res_y: DataResolution
results: DiagResultType
start: np.datetime64
stats: list[StatType]
storage: StoragePropertiesType = None
store(store, group, analyse_type, analyse_date, start, end, mode, fields)

Write the diagnostic results into the provided store.

Parameters:

  • store (DiagnosticStore) – DiagnosticStore used for the storage.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the computed data.

  • end (datetime64) – Ending date for the computed data.

  • mode (StorageMode) – Storage mode to use when writing data.

  • fields (list[str]) – List of all fields name used in the group of this statistic.

store_configuration(analyse_type, analyse_date, group=None)

Returns the list of storage group parameters required to store this diagnostic.

Parameters:

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date representing the set of data used in this analyse. It’s used to determine at which timestamp to store non-temporal diagnostics.

  • group (str | None) – Name of the group to use for storage.

Return type:

list[StorageGroupParams]

Returns:

List of storage group parameters.

store_params: dict[str, Any] | None = None
time_extension: bool = False
update_properties(diag)

Update diagnostic properties with the provided one.

Parameters:

diag (StatisticDiag) – Diagnostic from which to get properties.

property variable: str

Name of the main variable represented by this data.

class casys.computation.diagnostics.ScatterSwathDiag(name, store_params=None, *, data_index, data_time, start, end, field_x, field_y=None, indexer=None, time_extension=False, stats=<factory>, res_x, res_y)

Bases: ScatterDiag

Class representing a scatter statistic.

Parameters:

  • name (str) – Name of the data.

  • computed – True if the data have been computed, false otherwise.

  • is_post_diag – Whether this diagnostic is a post computation diagnostic or not.

  • data_index (str) – Name of the field on which the data is indexed.

  • start (np.datetime64) – Starting date of the input dataset.

  • end (np.datetime64) – Ending date of the input dataset.

  • field_x (Field) – Field used for x axes.

  • field_y (Field | None) – Field used for y axes.

  • stats (list[StatType]) – Statistics included in this Diagnostic.

  • indexer (ong_indexer_t.PassIndexer | None) – Indexer used to read data.

  • time_extension (bool) – Whether time extension is enabled or not.

  • ext_start – Extended starting date required by the computation.

  • ext_end – Ending date for the data required by the computation.

  • group – the name of the group of this diagnostic.

  • field_z – Field used for z axes

  • res_x (DataResolution) – Min, max and width for the x-axis.

  • res_y (DataResolution) – Min, max and width for the y-axis.

  • store_params (dict[str, Any] | None)

  • data_time (str)

MERGEABLE_STAT: ClassVar[list[StatType]] = [MIN, MAX, COUNT, MEAN, VAR, STD, PROD, SUM]
RES_TYPE

alias of CountResult

SCHEMA: ClassVar[type[mm.Schema]] = None
STORABLE: ClassVar[bool] = True
apply_stats_selection(diagnostic, fields)

Filter diagnostic by applying the statistic selection clip and removing unwanted statistics.

Parameters:

  • diagnostic (TypeVar(DiagResultType, bound= DiagResult)) – Diagnostic to filter.

  • fields (list[str]) – List of diagnostic field’s names.

Return type:

TypeVar(DiagResultType, bound= DiagResult)

check_stat(stat)

Check whether this data is computed and provided stat is valid. Determines the stat if None is provided.

Parameters:

stat (StatType | None) – Statistic to check.

Return type:

StatType

Returns:

Checked statistic.

Raises:

AltiDataError – If data with the provided name were not computed. If the stat parameter is invalid.

clear_data()

Clear stored data from this Diagnostic data.

compute(data, start, end, fields, diags)

Compute the diagnostic on the provided data.

Parameters:

  • data (Dataset) – The dataset with data to read.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • diags (list[StatisticDiag]) – List of diagnostics included in this computation.

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Dictionary containing the statistic type with its associated data array.

compute_from_store(store, group, analyse_type, analyse_date, start, end, fields, delayed)

Compute diagnostic using stored data. The computation can be done in a delayed way allowing to only load requested data when needed.

Parameters:

  • store (DiagnosticStore) – Store containing diagnostics data.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • delayed (bool) – Flag indicating whether to load from store during computation or at first data request (plot/get_data).

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Computation results.

computed: bool = False
dask_full_compatible: bool = False
data_index: str
data_time: str
diag_data(**kwargs)

Diagnostic data as a dataset.

Parameters:

kwargs – Additional parameters required to retrieve diagnostic data.

Return type:

Dataset

Returns:

Dataset containing this diagnostic data.

end: np.datetime64
property ext_end: datetime64
property ext_start: datetime64
property field: Field

Field of the main variable represented by this data.

field_x: Field
field_y: Field | None = None
field_z: Field
property fields: dict[str, Field]

Dictionary of fields used to compute this data.

finalize(result)

Correctly format the diagnostic result.

Parameters:

result (TypeVar(CountResultType, bound= CountResult)) – Result to finalize.

Return type:

TypeVar(CountResultType, bound= CountResult)

Returns:

Finalized result.

static grid_2d(res_x, res_y)

Generate a Grid2D from provided parameters.

Parameters:
Return type:

Grid2D

Returns:

Grid2D with provided resolutions.

group: str = None
group_base: str = None
html_elem(style='')

HTML representation of the data container.

Parameters:

style (str) – Style to add to the element.

Return type:

str

Returns:

Html representation of elements of this object.

html_header(span)

HTML representation header of the data container.

Parameters:

span (int) – Span size.

Returns:

Object html header.

property identity: DiagnosticType
indexer: ong_indexer_t.PassIndexer | None = None
initialize()

Diagnostic initialization.

At least the following properties have to be initialized:

  • group_base

  • storage_properties

is_autonomous(dask)

Check whether this diagnostic is autonomous with or without dask enabled.

Parameters:

dask (bool) – Whether dask is enabled or not.

Returns:

Whether this is an autonomous diagnostic or not.

is_dask_compatible(freq)

Check whether this diagnostic can be computed with dask or not.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

Return type:

bool

is_post_diag: bool = False
load_diagnostic(**kwargs)

Ensure requested diagnostic data are loaded.

make_dask_compatible(freq)

Set this container, so it can be used with dask.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

merge(diagnostics)

Merge provided diagnostics.

Parameters:

diagnostics (list[TypeVar(CountResultType, bound= CountResult)]) – Diagnostics to merge.

Return type:

TypeVar(CountResultType, bound= CountResult)

Returns:

Merged diagnostic.

metadata(internal=False)

Diagnostic serializable configuration.

Parameters:

internal (bool) – Whether this diagnostic is an internal diagnostic or not.

Return type:

DiagnosticMeta

Returns:

Diagnostic metadata.

name: str
new_algorithm()

Instantiate the algorithm.

Return type:

Dispersion2D

Returns:

Instantiated algorithm.

plot_container(template=None, stat=None)

Create a PlotContainer for the provided stat data.

Parameters:
Return type:

PlotContainer

Returns:

PlotContainer containing the requested data.

res_x: DataResolution
res_y: DataResolution
results: DiagResultType
start: np.datetime64
stats: list[StatType]
storage: StoragePropertiesType = None
store(store, group, analyse_type, analyse_date, start, end, mode, fields)

Write the diagnostic results into the provided store.

Parameters:

  • store (DiagnosticStore) – DiagnosticStore used for the storage.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the computed data.

  • end (datetime64) – Ending date for the computed data.

  • mode (StorageMode) – Storage mode to use when writing data.

  • fields (list[str]) – List of all fields name used in the group of this statistic.

store_configuration(analyse_type, analyse_date, group=None)

Returns the list of storage group parameters required to store this diagnostic.

Parameters:

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date representing the set of data used in this analyse. It’s used to determine at which timestamp to store non-temporal diagnostics.

  • group (str | None) – Name of the group to use for storage.

Return type:

list[StorageGroupParams]

Returns:

List of storage group parameters.

store_params: dict[str, Any] | None = None
time_extension: bool = False
update_properties(diag)

Update diagnostic properties with the provided one.

Parameters:

diag (StatisticDiag) – Diagnostic from which to get properties.

property variable: str

Name of the main variable represented by this data.

class casys.computation.diagnostics.SectionAnalysesDiag(name, store_params=None, *, data_index, data_time, start, end, field_x, field_y=None, indexer=None, time_extension=False, time, analysis)

Bases: Diagnostic

Class representing section analyses data.

Parameters:

  • name (str) – Name of the data.

  • computed – True if the data have been computed, false otherwise.

  • is_post_diag – Whether this diagnostic is a post computation diagnostic or not.

  • data_index (str) – Name of the field on which the data is indexed.

  • start (np.datetime64) – Starting date of the input dataset.

  • end (np.datetime64) – Ending date of the input dataset.

  • field_x (Field) – Field used for x axes.

  • field_y (Field | None) – Field used for y axes.

  • stats – Statistics included in this Diagnostic.

  • indexer (ong_indexer_t.PassIndexer | None) – Indexer used to read data.

  • time_extension (bool) – Whether time extension is enabled or not.

  • ext_start – Extended starting date required by the computation.

  • ext_end – Ending date for the data required by the computation.

  • time (Field) – Field used for the time.

  • analysis (ong_gts_t.Analysis) – Analysis to compute.

  • store_params (dict[str, Any] | None)

  • data_time (str)

RES_TYPE: ClassVar[type[DiagResult]] = None
SCHEMA: ClassVar[type[mm.Schema]] = None
STORABLE: ClassVar[bool] = False
analysis: ong_gts_t.Analysis
check_stat(stat)

Check whether this data is computed and provided stat is valid. Determines the stat if None is provided.

Parameters:

stat (StatType | None) – Statistic to check.

Return type:

StatType

Returns:

Checked statistic.

Raises:

AltiDataError – If data with the provided name were not computed. If the stat parameter is invalid.

clear_data()

Clear stored data from this Diagnostic data.

compute_analyses(data, analyses)
Parameters:
compute_from_store(store, group, analyse_type, analyse_date, start, end, fields, delayed)

Compute diagnostic using stored data. The computation can be done in a delayed way allowing to only load requested data when needed.

Parameters:

  • store (DiagnosticStore) – Store containing diagnostics data.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • delayed (bool) – Flag indicating whether to load from store during computation or at first data request (plot/get_data).

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Computation results.

computed: bool = False
property condition: Clip
dask_full_compatible: bool = False
data_index: str
data_time: str
diag_data(**kwargs)

Diagnostic data as a dataset.

Parameters:

kwargs – Additional parameters required to retrieve diagnostic data.

Return type:

Dataset

Returns:

Dataset containing this diagnostic data.

end: np.datetime64
property ext_end: datetime64
property ext_start: datetime64
property field: Field | None

Field of the main variable represented by this data.

field_x: Field
field_y: Field | None = None
property fields: dict[str, Field]

Dictionary of fields used to compute this data.

property fill_missing: bool
group: str = None
group_base: str = None
html_elem(style='')

HTML representation of the data container.

Parameters:

style (str) – Style to add to the element.

Return type:

str

Returns:

Html representation of elements of this object.

html_header(span)

HTML representation header of the data container.

Parameters:

span (int) – Span size.

Returns:

Object html header.

property identity: DiagnosticType
indexer: ong_indexer_t.PassIndexer | None = None
initialize()

Diagnostic initialization.

At least the following properties have to be initialized:

  • group_base

  • storage_properties

is_autonomous(dask)

Check whether this diagnostic is autonomous with or without dask enabled.

Parameters:

dask (bool) – Whether dask is enabled or not.

Returns:

Whether this is an autonomous diagnostic or not.

is_dask_compatible(freq)

Check whether this diagnostic can be computed with dask or not.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

Return type:

bool

is_post_diag: bool = False
load_diagnostic(**kwargs)

Ensure requested diagnostic data are loaded.

make_dask_compatible(freq)

Set this container, so it can be used with dask.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

property max_percent_of_false: float
metadata(internal=False)

Diagnostic serializable configuration.

Parameters:

internal (bool) – Whether this diagnostic is an internal diagnostic or not.

Return type:

DiagnosticMeta

Returns:

Diagnostic metadata.

property min_length: int
name: str
plot_container(template=None, sections=None)

Create a PlotContainer for the provided stat data.

Parameters:

  • template (PlotTemplate | None) – Parameters template to use.

  • sections (list[int] | None) –

    Sections to plot:

    • ”all”: Select all sections

    • number: Select section’s requested number

    • list of numbers: Select sections’ requested numbers

Return type:

PlotContainer

Returns:

PlotContainer containing the requested data.

results: DiagResultType
start: np.datetime64
stats: list[StatType]
storage: StoragePropertiesType = None
store(store, group, analyse_type, analyse_date, start, end, mode, fields)

Write the diagnostic results into the provided store.

Parameters:

  • store (DiagnosticStore) – DiagnosticStore used for the storage.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the computed data.

  • end (datetime64) – Ending date for the computed data.

  • mode (StorageMode) – Storage mode to use when writing data.

  • fields (list[str]) – List of all fields name used in the group of this statistic.

store_configuration(analyse_type, analyse_date, group=None)

Returns the list of storage group parameters required to store this diagnostic.

Parameters:

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date representing the set of data used in this analyse. It’s used to determine at which timestamp to store non-temporal diagnostics.

  • group (str | None) – Name of the group to use for storage.

Return type:

list[StorageGroupParams]

Returns:

List of storage group parameters.

store_params: dict[str, Any] | None = None
time: Field
time_extension: bool = False
property variable: str

Name of the main variable represented by this data.

class casys.computation.diagnostics.SpectralAnalysisDiag(name, store_params=None, *, data_index, data_time, start, end, field_x, field_y=None, indexer=None, time_extension=False, stats=<factory>, wn, time, field_z, segment_length, holes_max_length, global_resampling, delta_t, noise_amplitude, insulation_level, last_segment_overlapping, max_time_dispersion, max_resampling, segments_nb_delta_t, segments_nb_delta_x, spectral_names, spectral_types, spectral_kwargs, res_segments, res_individual_psd)

Bases: StatisticDiag[SpectralAnalysisResult, SpectralAnalysisStorageProperties]

Spectral analysis diagnostic.

Parameters:

  • name (str) – Name of the data.

  • computed – True if the data have been computed, false otherwise.

  • is_post_diag – Whether this diagnostic is a post computation diagnostic or not.

  • data_index (str) – Name of the field on which the data is indexed.

  • start (np.datetime64) – Starting date of the input dataset.

  • end (np.datetime64) – Ending date of the input dataset.

  • field_x (Field) – Field used for x axes.

  • field_y (Field | None) – Field used for y axes.

  • stats (list[StatType]) – Statistics included in this Diagnostic.

  • indexer (ong_indexer_t.PassIndexer | None) – Indexer used to read data.

  • time_extension (bool) – Whether time extension is enabled or not.

  • ext_start – Extended starting date required by the computation.

  • ext_end – Ending date for the data required by the computation.

  • group – the name of the group of this diagnostic.

  • wn (Field) – Field used for the wave number dimension.

  • time (Field) – Field used for the time.

  • field_z (Field) – Field used for z axes.

  • stats – List of statistic types used to reduce the spectral data across segments.

  • segment_length (int) – Length of a segment (section) in number of points. It should be something like a few hundred points. (Example: 500 units)

  • holes_max_length (int) – Maximum length of a hole. It should be something like a few points. (Example: 5)

  • global_resampling (bool) – Resampling Flag (Default: False). True - If one section requires to be resampled => resample all sections. False - Resample only sections requiring a resampling.

  • delta_t (np.timedelta64) – Time gap between two measurements.

  • noise_amplitude (float) – Noise amplitude in data. Default to half of the data standard deviation.

  • insulation_level (float) – Minimum valid values percentage on both sides of the hole (Default: 0.75). Left and right sides are equal to hole length.

  • last_segment_overlapping (float) – Percentage of overlap for second-to-last segment (Default: 0.5). When the section is divided in equal segments, the last segment might be too short, so it will take some part of data (amount depending on this parameter) from the previous segment.

  • max_time_dispersion (int) – Maximum allowed percentage of dispersion for delta_t (Default: 5). If delta_t dispersion exceed this threshold, a warning will be displayed.

  • max_resampling (float) – Maximum resampled data percentage (Default: 0.25). A warning will be displayed if this threshold is exceeded. The resampling of a large amount of data can have a great impact on the final result.

  • segments_nb_delta_t (int) – Number of segments used to compute the average time gap between two measurements, during the segments extraction process (Default: 1).

  • segments_nb_delta_x (int) – Number of segments used to compute the average distance between two measurements, during the segments extraction process (Default: 1).

  • spectral_names (list[str]) – List of the spectral analyses names.

  • spectral_types (list[SpectralType]) – List of spectral curve types to compute. Possibilities are “periodogram” and “welch”.

  • spectral_kwargs (list[dict[str, Any]]) – Dictionary of the spectral parameters to use for the spectral curve types. Each key representing a spectral type value is associated with a dictionary containing the parameters.

  • res_segments (bool) – Flag indicating whether to save the segments data in the spectral analysis result (Default: False). True - Saving segments data. False - Not saving segments data.

  • res_individual_psd (bool) – Flag indicating whether to save the individual power spectrum data on each segments in the spectral analysis result (Default: False). True - Saving the individual psd data. False - Not saving the individual psd data.

  • store_params (dict[str, Any] | None)

  • data_time (str)

MERGEABLE_STAT: ClassVar[list[StatType]] = [MIN, MAX, COUNT, MEAN, VAR, STD, PROD, SUM]
RES_TYPE

alias of SpectralAnalysisResult

SCHEMA: ClassVar[type[mm.Schema]] = None
STORABLE: ClassVar[bool] = True
apply_stats_selection(diagnostic, fields)

Filter diagnostic by applying the statistic selection clip and removing unwanted statistics.

Parameters:

  • diagnostic (TypeVar(DiagResultType, bound= DiagResult)) – Diagnostic to filter.

  • fields (list[str]) – List of diagnostic field’s names.

Return type:

TypeVar(DiagResultType, bound= DiagResult)

check_stat(stat)

Check whether this data is computed and provided stat is valid. Determines the stat if None is provided.

Parameters:

stat (StatType | None) – Statistic to check.

Return type:

StatType

Returns:

Checked statistic.

Raises:

AltiDataError – If data with the provided name were not computed. If the stat parameter is invalid.

clear_data()

Clear stored data from this Diagnostic data.

compute(data, start, end, fields, diags)

Compute the diagnostic on the provided data.

Parameters:

  • data (Dataset) – The dataset with data to read.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • diags (list[StatisticDiag]) – List of diagnostics included in this computation.

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Dictionary containing the statistic type with its associated data array.

compute_from_store(store, group, analyse_type, analyse_date, start, end, fields, delayed)

Compute diagnostic using stored data. The computation can be done in a delayed way allowing to only load requested data when needed.

Parameters:

  • store (DiagnosticStore) – Store containing diagnostics data.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • delayed (bool) – Flag indicating whether to load from store during computation or at first data request (plot/get_data).

Return type:

SpectralAnalysisResult

Returns:

Computation results.

computed: bool = False
dask_full_compatible: bool = False
data_index: str
data_time: str
delta_t: np.timedelta64
diag_data(pixel=None, plot=None, spectral_name=None, individual=False, segments_reduction=None)

Diagnostic data as a dataset.

Parameters:

  • pixel (int | list | tuple | None) – Single integer value or pixel tuple range (p_min, p_max). Parameter for Swath data case.

  • plot (str | None) – Type of data to extract from the results, “PSD” (default) or “SEGMENTS”.

  • spectral_name (str | None) – Name of the spectral data to get (plot=”PSD” case).

  • individual (bool) – Whether to return the unreduced or reduced spectral results (plot=”PSD” case).

  • segments_reduction (StatType | str | None) – Statistic reduction type to get (plot=”PSD”, individual=True case).

Return type:

Dataset

Returns:

Dataset containing this diagnostic data.

end: np.datetime64
property ext_end: datetime64
property ext_start: datetime64
property field: Field

Field of the main variable represented by this data.

field_x: Field
field_y: Field | None = None
field_z: Field
property fields: dict[str, Field]

Dictionary of fields used to compute this data.

finalize(result)

Correctly format the diagnostic result.

Parameters:

result (TypeVar(DiagResultType, bound= DiagResult)) – Result to finalize.

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Finalized result.

global_resampling: bool
group: str = None
group_base: str = None
holes_max_length: int
html_elem(style='')

HTML representation of the data container.

Parameters:

style (str) – Style to add to the element.

Return type:

str

Returns:

Html representation of elements of this object.

html_header(span)

HTML representation header of the data container.

Parameters:

span (int) – Span size.

Returns:

Object html header.

property identity: DiagnosticType
indexer: ong_indexer_t.PassIndexer | None = None
initialize()

Diagnostic initialization.

At least the following properties have to be initialized:

  • group_base

  • storage_properties

insulation_level: float
is_autonomous(dask)

Check whether this diagnostic is autonomous with or without dask enabled.

Parameters:

dask (bool) – Whether dask is enabled or not.

Returns:

Whether this is an autonomous diagnostic or not.

is_dask_compatible(freq)

Check whether this diagnostic can be computed with dask or not.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

Return type:

bool

is_post_diag: bool = False
last_segment_overlapping: float
load_diagnostic(plot)

Ensure requested diagnostic data are loaded.

Parameters:

plot (str | None)

make_dask_compatible(freq)

Set this container, so it can be used with dask.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

max_resampling: float
max_time_dispersion: int
merge(diagnostics)

Merge provided diagnostics.

Parameters:

diagnostics (list[SpectralAnalysisResult]) – Diagnostics to merge.

Return type:

SpectralAnalysisResult

Returns:

Merged diagnostic.

metadata(internal=False)

Diagnostic serializable configuration.

Parameters:

internal (bool) – Whether this diagnostic is an internal diagnostic or not.

Return type:

DiagnosticMeta

Returns:

Diagnostic metadata.

name: str
noise_amplitude: float
plot_container(template=None, pixel=None, plot=None, segments_reduction=None, individual=False, spectral_name=None, second_axis=False, n_bins_psd=100)

Create a PlotContainer for the provided stat data.

Parameters:

  • template (PlotTemplate | None) – Parameters template to use.

  • pixel (int | list | tuple | None) – Single integer value or pixel tuple range (p_min, p_max). Parameter for Swath data case.

  • plot (str) – Type of data to extract from the results, “PSD” (default) or “SEGMENTS”.

  • spectral_name (str | None) – Name of the spectral data to get (plot=”PSD” case).

  • individual (bool) – Whether to return the unreduced or reduced spectral results (plot=”PSD” case).

  • segments_reduction (StatType | str | None) – Statistic reduction type to get (plot=”PSD”, individual=True case).

  • second_axis (bool) – Flag allowing the display of the second x-axis, for the segment length values equivalent to the wave number.

  • n_bins_psd (int) – Integer determining the number of bins along the psd values.

Return type:

PlotContainer

Returns:

PlotContainer containing the requested data.

res_individual_psd: bool
res_segments: bool
results: DiagResultType
segment_length: int
segments_nb_delta_t: int
segments_nb_delta_x: int
property spectral_conf: dict[str, dict[str | SpectralType, Any]]

Spectral configuration in a single dictionary (used for serialization).

spectral_kwargs: list[dict[str, Any]]
spectral_names: list[str]
spectral_types: list[SpectralType]
start: np.datetime64
stats: list[StatType]
storage: StoragePropertiesType = None
store(store, group, analyse_type, analyse_date, start, end, mode, fields)

Write the diagnostic results into the provided store.

Parameters:

  • store (DiagnosticStore) – DiagnosticStore used for the storage.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the computed data.

  • end (datetime64) – Ending date for the computed data.

  • mode (StorageMode) – Storage mode to use when writing data.

  • fields (list[str]) – List of all fields name used in the group of this statistic.

store_configuration(analyse_type, analyse_date, group=None)

Returns the list of storage group parameters required to store this diagnostic.

Parameters:

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date representing the set of data used in this analyse. It’s used to determine at which timestamp to store non-temporal diagnostics.

  • group (str | None) – Name of the group to use for storage.

Return type:

list[StorageGroupParams]

Returns:

List of storage group parameters.

store_params: dict[str, Any] | None = None
sub_diag_store_configuration(data, analyse_type, group)
Parameters:
Return type:

list[StorageGroupParams]

time: Field
time_extension: bool = False
update_properties(diag)

Update diagnostic properties with the provided one.

Parameters:

diag (StatisticDiag) – Diagnostic from which to get properties.

property variable: str

Name of the main variable represented by this data.

wn: Field
class casys.computation.diagnostics.SpectralAnalysisResult(data, segments, time, dimension, analyses)

Bases: StatDiagResult

Spectral analysis diagnostic results.

Parameters:

  • data (dict[str, dict[str, SpectralConfResult]] | None) – Spectral analysis results for each field.

  • segments (dict[str, Dataset] | None) – Segments analysis results for each field.

  • time (str) – Time’s field name.

  • dimension (str) – Main dimension name.

  • analyses (list[str]) – Spectral analyses included in the result container.

analyses: list[str]
data: dict[str, dict[str, SpectralConfResult]] | None
dimension: str
have_data()

Whether this result contains data or not.

Return type:

bool

Returns:

True if this result has data, false otherwise.

have_segments()
Return type:

bool

merge_data(result)

Merge provided result’s data into this one. These results are in a finalized state.

Parameters:

result (SpectralAnalysisResult | None) – Result to merge.

merge_stats(results, **kwargs)

Merge provided results into this one. These results are in a non finalized state.

Parameters:

results (list[SpectralAnalysisResult]) – Results to merge.

remove_data(field=None, stats=None)

Remove results’ data from the stat of the given field.

Parameters:

  • field (str) – Name of the field

  • stats (list[StatType]) – Statistics to remove.

segments: dict[str, Dataset] | None
set_data(data)
Parameters:

data (dict[str, dict[str, SpectralConfResult]] | None)

set_segments(data)
Parameters:

data (dict[str, Dataset] | None)

statistics()

List of statistics computed for this diagnostic.

Return type:

list[StatType]

time: str
class casys.computation.diagnostics.SpectralAnalysisSwathDiag(name, store_params=None, special_fields=<factory>, pixels=None, pixels_selection=ALL, pixels_reduction=MEAN, *, data_index, data_time, start, end, field_x, field_y=None, indexer=None, time_extension=False, stats=<factory>, wn, time, field_z, segment_length, holes_max_length, global_resampling, delta_t, noise_amplitude, insulation_level, last_segment_overlapping, max_time_dispersion, max_resampling, segments_nb_delta_t, segments_nb_delta_x, spectral_names, spectral_types, spectral_kwargs, res_segments, res_individual_psd)

Bases: SpectralAnalysisDiag

Swath spectral analysis diagnostic.

Parameters:

  • name (str) – Name of the data.

  • computed – True if the data have been computed, false otherwise.

  • is_post_diag – Whether this diagnostic is a post computation diagnostic or not.

  • data_index (str) – Name of the field on which the data is indexed.

  • start (np.datetime64) – Starting date of the input dataset.

  • end (np.datetime64) – Ending date of the input dataset.

  • field_x (Field) – Field used for x axes.

  • field_y (Field | None) – Field used for y axes.

  • stats (list[StatType]) – Statistics included in this Diagnostic.

  • indexer (ong_indexer_t.PassIndexer | None) – Indexer used to read data.

  • time_extension (bool) – Whether time extension is enabled or not.

  • ext_start – Extended starting date required by the computation.

  • ext_end – Ending date for the data required by the computation.

  • group – the name of the group of this diagnostic.

  • wn (Field) – Field used for the wave number dimension.

  • time (Field) – Field used for the time.

  • field_z (Field) – Field used for z axes.

  • stats – List of statistic types used to reduce the spectral data across segments.

  • segment_length (int) – Length of a segment (section) in number of points. It should be something like a few hundred points. (Example: 500 units)

  • holes_max_length (int) – Maximum length of a hole. It should be something like a few points. (Example: 5)

  • global_resampling (bool) – Resampling Flag (Default: False). True - If one section requires to be resampled => resample all sections. False - Resample only sections requiring a resampling.

  • delta_t (np.timedelta64) – Time gap between two measurements.

  • noise_amplitude (float) – Noise amplitude in data. Default to half of the data standard deviation.

  • insulation_level (float) – Minimum valid values percentage on both sides of the hole (Default: 0.75). Left and right sides are equal to hole length.

  • last_segment_overlapping (float) – Percentage of overlap for second-to-last segment (Default: 0.5). When the section is divided in equal segments, the last segment might be too short, so it will take some part of data (amount depending on this parameter) from the previous segment.

  • max_time_dispersion (int) – Maximum allowed percentage of dispersion for delta_t (Default: 5). If delta_t dispersion exceed this threshold, a warning will be displayed.

  • max_resampling (float) – Maximum resampled data percentage (Default: 0.25). A warning will be displayed if this threshold is exceeded. The resampling of a large amount of data can have a great impact on the final result.

  • segments_nb_delta_t (int) – Number of segments used to compute the average time gap between two measurements, during the segments extraction process (Default: 1).

  • segments_nb_delta_x (int) – Number of segments used to compute the average distance between two measurements, during the segments extraction process (Default: 1).

  • spectral_names (list[str]) – List of the spectral analyses names.

  • spectral_types (list[SpectralType]) – List of spectral curve types to compute. Possibilities are “periodogram” and “welch”.

  • spectral_kwargs (list[dict[str, Any]]) – Dictionary of the spectral parameters to use for the spectral curve types. Each key representing a spectral type value is associated with a dictionary containing the parameters.

  • res_segments (bool) – Flag indicating whether to save the segments data in the spectral analysis result (Default: False). True - Saving segments data. False - Not saving segments data.

  • res_individual_psd (bool) – Flag indicating whether to save the individual power spectrum data on each segments in the spectral analysis result (Default: False). True - Saving the individual psd data. False - Not saving the individual psd data.

  • special_fields (dict[FieldType, Field]) – Dictionary of fields specific to the read data (dimensions and characteristics).

  • pixels (list[int | tuple[int, int]] | None) – Pixels indexes along cross track distance dimension for which to compute the diagnostic. Either a single integer (for a single pixel) or a list/tuple of: - integers: pix - list/tuple of two integers, describing a range (pix_min, pix_max)

  • pixels_selection (SelectionType) – Mode of spectral data reduction for the different spectral curves computed for the provided pixel values (Default: “all”). “all”: reduce all the spectral curves for the reduction. “range”: reduce the spectral curves within a cross track distance range. “none”: no reduction performed.

  • pixels_reduction (StatType) – Statistic type used to reduce the spectral data across pixels.

  • store_params (dict[str, Any] | None)

  • data_time (str)

MERGEABLE_STAT: ClassVar[list[StatType]] = [MIN, MAX, COUNT, MEAN, VAR, STD, PROD, SUM]
RES_TYPE

alias of SpectralAnalysisResult

SCHEMA: ClassVar[type[mm.Schema]] = None
STORABLE: ClassVar[bool] = True
apply_stats_selection(diagnostic, fields)

Filter diagnostic by applying the statistic selection clip and removing unwanted statistics.

Parameters:

  • diagnostic (TypeVar(DiagResultType, bound= DiagResult)) – Diagnostic to filter.

  • fields (list[str]) – List of diagnostic field’s names.

Return type:

TypeVar(DiagResultType, bound= DiagResult)

check_stat(stat)

Check whether this data is computed and provided stat is valid. Determines the stat if None is provided.

Parameters:

stat (StatType | None) – Statistic to check.

Return type:

StatType

Returns:

Checked statistic.

Raises:

AltiDataError – If data with the provided name were not computed. If the stat parameter is invalid.

clear_data()

Clear stored data from this Diagnostic data.

compute(data, start, end, fields, diags)

Compute the diagnostic on the provided data.

Parameters:

  • data (Dataset) – The dataset with data to read.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • diags (list[StatisticDiag]) – List of diagnostics included in this computation.

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Dictionary containing the statistic type with its associated data array.

compute_from_store(store, group, analyse_type, analyse_date, start, end, fields, delayed)

Compute diagnostic using stored data. The computation can be done in a delayed way allowing to only load requested data when needed.

Parameters:

  • store (DiagnosticStore) – Store containing diagnostics data.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • delayed (bool) – Flag indicating whether to load from store during computation or at first data request (plot/get_data).

Return type:

SpectralAnalysisSwathResult

Returns:

Computation results.

compute_spectral_reduction(*, fields, analyses_names, results_pix)
Parameters:

  • fields – List of all fields name used in the group of this statistic.

  • analyses_names – List of the analyses names.

  • results_pix – Dictionaries of the spectral results for each pixel value.

Returns:

Reduced spectral results

computed: bool = False
dask_full_compatible: bool = False
data_index: str
data_time: str
delta_t: np.timedelta64
diag_data(pixel=None, plot=None, spectral_name=None, individual=False, segments_reduction=None)

Diagnostic data as a dataset.

Parameters:

  • pixel (int | list | tuple | None) – Single integer value or pixel tuple range (p_min, p_max). Parameter for Swath data case.

  • plot (str | None) – Type of data to extract from the results, “PSD” (default) or “SEGMENTS”.

  • spectral_name (str | None) – Name of the spectral data to get (plot=”PSD” case).

  • individual (bool) – Whether to return the unreduced or reduced spectral results (plot=”PSD” case).

  • segments_reduction (StatType | str | None) – Statistic reduction type to get (plot=”PSD”, individual=True case).

Return type:

Dataset

Returns:

Dataset containing this diagnostic data.

end: np.datetime64
property ext_end: datetime64
property ext_start: datetime64
property field: Field

Field of the main variable represented by this data.

field_x: Field
field_y: Field | None = None
field_z: Field
property fields: dict[str, Field]

Dictionary of fields used to compute this data.

finalize(result)

Correctly format the diagnostic result.

Parameters:

result (TypeVar(DiagResultType, bound= DiagResult)) – Result to finalize.

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Finalized result.

global_resampling: bool
group: str = None
group_base: str = None
holes_max_length: int
html_elem(style='')

HTML representation of the data container.

Parameters:

style (str) – Style to add to the element.

Return type:

str

Returns:

Html representation of elements of this object.

html_header(span)

HTML representation header of the data container.

Parameters:

span (int) – Span size.

Returns:

Object html header.

property identity: DiagnosticType
indexer: ong_indexer_t.PassIndexer | None = None
initialize()

Diagnostic initialization.

At least the following properties have to be initialized:

  • group_base

  • storage_properties

insulation_level: float
is_autonomous(dask)

Check whether this diagnostic is autonomous with or without dask enabled.

Parameters:

dask (bool) – Whether dask is enabled or not.

Returns:

Whether this is an autonomous diagnostic or not.

is_dask_compatible(freq)

Check whether this diagnostic can be computed with dask or not.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

Return type:

bool

is_post_diag: bool = False
last_segment_overlapping: float
load_diagnostic(plot)

Ensure requested diagnostic data are loaded.

Parameters:

plot (str | None)

make_dask_compatible(freq)

Set this container, so it can be used with dask.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

max_resampling: float
max_time_dispersion: int
merge(diagnostics)

Merge provided diagnostics.

Parameters:

diagnostics (list[SpectralAnalysisSwathResult]) – Diagnostics to merge.

Return type:

SpectralAnalysisSwathResult

Returns:

Merged diagnostic.

metadata(internal=False)

Diagnostic serializable configuration.

Parameters:

internal (bool) – Whether this diagnostic is an internal diagnostic or not.

Return type:

DiagnosticMeta

Returns:

Diagnostic metadata.

name: str
noise_amplitude: float
pixels: list[int | tuple[int, int]] | None = None
pixels_diag: dict[int | tuple[int, int], SpectralAnalysisDiag]
pixels_reduction: StatType = 5
pixels_selection: SelectionType = 2
plot_container(template=None, pixel=None, plot=None, segments_reduction=None, individual=False, spectral_name=None, second_axis=False, n_bins_psd=100)

Create a PlotContainer for the provided stat data.

Parameters:

  • template (PlotTemplate | None) – Parameters template to use.

  • pixel (int | list | tuple | None) – Single integer value or pixel tuple range (p_min, p_max). Parameter for Swath data case.

  • plot (str) – Type of data to extract from the results, “PSD” (default) or “SEGMENTS”.

  • spectral_name (str | None) – Name of the spectral data to get (plot=”PSD” case).

  • individual (bool) – Whether to return the unreduced or reduced spectral results (plot=”PSD” case).

  • segments_reduction (StatType | str | None) – Statistic reduction type to get (plot=”PSD”, individual=True case).

  • second_axis (bool) – Flag allowing the display of the second x-axis, for the segment length values equivalent to the wave number.

  • n_bins_psd (int) – Integer determining the number of bins along the psd values.

Return type:

PlotContainer

Returns:

PlotContainer containing the requested data.

property real_indexes: list[int]
res_individual_psd: bool
res_segments: bool
results: DiagResultType
segment_length: int
segments_nb_delta_t: int
segments_nb_delta_x: int
special_fields: dict[FieldType, Field]
property spectral_conf: dict[str, dict[str | SpectralType, Any]]

Spectral configuration in a single dictionary (used for serialization).

spectral_kwargs: list[dict[str, Any]]
spectral_names: list[str]
spectral_types: list[SpectralType]
start: np.datetime64
stats: list[StatType]
storage: StoragePropertiesType = None
store(store, group, analyse_type, analyse_date, start, end, mode, fields)

Write the diagnostic results into the provided store.

Parameters:

  • store (DiagnosticStore) – DiagnosticStore used for the storage.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the computed data.

  • end (datetime64) – Ending date for the computed data.

  • mode (StorageMode) – Storage mode to use when writing data.

  • fields (list[str]) – List of all fields name used in the group of this statistic.

store_configuration(analyse_type, analyse_date, group=None)

Returns the list of storage group parameters required to store this diagnostic.

Parameters:

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date representing the set of data used in this analyse. It’s used to determine at which timestamp to store non-temporal diagnostics.

  • group (str | None) – Name of the group to use for storage.

Return type:

list[StorageGroupParams]

Returns:

List of storage group parameters.

store_params: dict[str, Any] | None = None
sub_diag_store_configuration(data, analyse_type, group)
Parameters:
Return type:

list[StorageGroupParams]

time: Field
time_extension: bool = False
update_properties(diag)

Update diagnostic properties with the provided one.

Parameters:

diag (StatisticDiag) – Diagnostic from which to get properties.

property variable: str

Name of the main variable represented by this data.

wn: Field
class casys.computation.diagnostics.SpectralAnalysisSwathResult(data)

Bases: StatDiagResult

Spectral diagnostic Swath results.

Parameters:

data (dict[tuple | int | None, SpectralAnalysisResult] | None) – Spectral results for each pixel.

data: dict[tuple | int | None, SpectralAnalysisResult] | None
have_data()

Whether this result contains data or not.

Return type:

bool

Returns:

True if this result has data, false otherwise.

merge_data(result)

Merge provided result’s data into this one. These results are in a finalized state.

Parameters:

result (SpectralAnalysisSwathResult | None) – Result to merge.

merge_stats(results, **kwargs)

Merge provided results into this one. These results are in a non finalized state.

Parameters:

results (list[SpectralAnalysisSwathResult]) – Results to merge.

remove_data(field=None, stats=None)

Remove results’ data from the stat of the given field.

Parameters:

  • field (str) – Name of the field

  • stats (list[StatType]) – Statistics to remove.

statistics()

List of statistics computed for this diagnostic.

Return type:

list[StatType]

class casys.computation.diagnostics.SpectralConfResult(data, wn, individual, nbr_segments, delta_t)

Bases: object

Spectral analysis diagnostic sub results for spectral curves.

Parameters:

  • data (dict[StatType, dict[str, ndarray]]) – Spectral analysis curves results for a specific spectral conf.

  • wn (ndarray) – Wave number values.

  • individual (ndarray) – Whether to save individual spectral curves results.

  • nbr_segments (int) – Number of segments.

  • delta_t (float) – Time delta value.

data: dict[StatType, dict[str, ndarray]]
delta_t: float
field_data(field, individual, reduction=None)

Extract field’s data result.

Parameters:

  • field (str) – Field’s name.

  • individual (bool) – Whether to get individual results or not.

  • reduction (StatType) – Segment’s reduction type for which to get results.

Returns:

Field’s data result.

individual: ndarray
nbr_segments: int
wn: ndarray
class casys.computation.diagnostics.StatDiagResult

Bases: DiagResult, ABC

Common result base for diagnostics computing statistics.

abstract have_data()

Whether this result contains data or not.

Return type:

bool

Returns:

True if this result has data, false otherwise.

abstract merge_data(result)

Merge provided result’s data into this one. These results are in a finalized state.

Parameters:

result (DiagResult | None) – Result to merge.

abstract merge_stats(results, **kwargs)

Merge provided results into this one. These results are in a non finalized state.

Parameters:

results (list[DiagResult]) – Results to merge.

abstract remove_data(field=None, stats=None)

Remove results’ data from the stat of the given field.

Parameters:

  • field (str) – Name of the field

  • stats (list[StatType]) – Statistics to remove.

abstract statistics()

List of statistics computed for this diagnostic.

Return type:

list[StatType]

class casys.computation.diagnostics.StatisticDiag(name, store_params=None, *, data_index, data_time, start, end, field_x, field_y=None, indexer=None, time_extension=False, stats=<factory>)

Bases: Diagnostic[CountResultType, StoragePropertiesType], ABC

Abstract class defining the common structure of a statistical diagnostic: computation, results, storage, schema and visualization, and related methods and attributes.

Parameters:

  • name (str) – Name of the data.

  • computed – True if the data have been computed, false otherwise.

  • is_post_diag – Whether this diagnostic is a post computation diagnostic or not.

  • data_index (str) – Name of the field on which the data is indexed.

  • start (datetime64) – Starting date of the input dataset.

  • end (datetime64) – Ending date of the input dataset.

  • field_x (Field) – Field used for x axes.

  • field_y (Field | None) – Field used for y axes.

  • stats (list[StatType]) – Statistics included in this Diagnostic.

  • indexer (PassIndexer | None) – Indexer used to read data.

  • time_extension (bool) – Whether time extension is enabled or not.

  • ext_start – Extended starting date required by the computation.

  • ext_end – Ending date for the data required by the computation.

  • group – the name of the group of this diagnostic.

  • store_params (dict[str, Any] | None)

  • data_time (str)

MERGEABLE_STAT: ClassVar[list[StatType]] = [MIN, MAX, COUNT, MEAN, VAR, STD, PROD, SUM]
RES_TYPE: ClassVar[type[DiagResult]] = None
SCHEMA: ClassVar[type[Schema]] = None
STORABLE: ClassVar[bool]
apply_stats_selection(diagnostic, fields)

Filter diagnostic by applying the statistic selection clip and removing unwanted statistics.

Parameters:

  • diagnostic (TypeVar(DiagResultType, bound= DiagResult)) – Diagnostic to filter.

  • fields (list[str]) – List of diagnostic field’s names.

Return type:

TypeVar(DiagResultType, bound= DiagResult)

check_stat(stat)

Check whether this data is computed and provided stat is valid. Determines the stat if None is provided.

Parameters:

stat (StatType | None) – Statistic to check.

Return type:

StatType

Returns:

Checked statistic.

Raises:

AltiDataError – If data with the provided name were not computed. If the stat parameter is invalid.

clear_data()

Clear stored data from this Diagnostic data.

compute(data, start, end, fields, diags)

Compute the diagnostic on the provided data.

Parameters:

  • data (Dataset) – The dataset with data to read.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • diags (list[StatisticDiag]) – List of diagnostics included in this computation.

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Dictionary containing the statistic type with its associated data array.

compute_from_store(store, group, analyse_type, analyse_date, start, end, fields, delayed)

Compute diagnostic using stored data. The computation can be done in a delayed way allowing to only load requested data when needed.

Parameters:

  • store (DiagnosticStore) – Store containing diagnostics data.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • delayed (bool) – Flag indicating whether to load from store during computation or at first data request (plot/get_data).

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Computation results.

computed: bool = False
dask_full_compatible: bool = False
data_index: str
data_time: str
diag_data(**kwargs)

Diagnostic data as a dataset.

Parameters:

kwargs – Additional parameters required to retrieve diagnostic data.

Return type:

Dataset

Returns:

Dataset containing this diagnostic data.

end: datetime64
property ext_end: datetime64
property ext_start: datetime64
abstract property field: Field

Field of the main variable represented by this data.

field_x: Field
field_y: Field | None = None
abstract property fields: dict[str, Field]

Dictionary of fields used to compute this data.

finalize(result)

Correctly format the diagnostic result.

Parameters:

result (TypeVar(DiagResultType, bound= DiagResult)) – Result to finalize.

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Finalized result.

group: str = None
group_base: str = None
abstract html_elem(style='')

HTML representation of the data container.

Parameters:

style (str) – Style to add to the element.

Return type:

str

Returns:

Html representation of elements of this object.

abstract html_header(span)

HTML representation header of the data container.

Parameters:

span (int) – Span size.

Return type:

str

Returns:

Object html header.

property identity: DiagnosticType
indexer: PassIndexer | None = None
abstract initialize()

Diagnostic initialization.

At least the following properties have to be initialized:

  • group_base

  • storage_properties

is_autonomous(dask)

Check whether this diagnostic is autonomous with or without dask enabled.

Parameters:

dask (bool) – Whether dask is enabled or not.

Returns:

Whether this is an autonomous diagnostic or not.

abstract is_dask_compatible(freq)

Check whether this diagnostic can be computed with dask or not.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

Return type:

bool

is_post_diag: bool = False
load_diagnostic(**kwargs)

Ensure requested diagnostic data are loaded.

make_dask_compatible(freq)

Set this container, so it can be used with dask.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

abstract merge(diagnostics)

Merge provided diagnostics.

Parameters:

diagnostics (list[TypeVar(DiagResultType, bound= DiagResult)]) – Diagnostics to merge.

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Merged diagnostic.

metadata(internal=False)

Diagnostic serializable configuration.

Parameters:

internal (bool) – Whether this diagnostic is an internal diagnostic or not.

Return type:

DiagnosticMeta

Returns:

Diagnostic metadata.

name: str
abstract plot_container(template=None, **kwargs)

Create a PlotContainer for the provided stat data.

Parameters:

  • template (PlotTemplate | None) – Parameters template to use.

  • kwargs – Additional parameters required to generate the container.

Return type:

PlotContainer

Returns:

PlotContainer containing the requested data.

results: TypeVar(DiagResultType, bound= DiagResult)
start: datetime64
stats: list[StatType]
storage: TypeVar(StoragePropertiesType, bound= StorageProperties) = None
store(store, group, analyse_type, analyse_date, start, end, mode, fields)

Write the diagnostic results into the provided store.

Parameters:

  • store (DiagnosticStore) – DiagnosticStore used for the storage.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the computed data.

  • end (datetime64) – Ending date for the computed data.

  • mode (StorageMode) – Storage mode to use when writing data.

  • fields (list[str]) – List of all fields name used in the group of this statistic.

store_configuration(analyse_type, analyse_date, group=None)

Returns the list of storage group parameters required to store this diagnostic.

Parameters:

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date representing the set of data used in this analyse. It’s used to determine at which timestamp to store non-temporal diagnostics.

  • group (str | None) – Name of the group to use for storage.

Return type:

list[StorageGroupParams]

Returns:

List of storage group parameters.

store_params: dict[str, Any] | None = None
time_extension: bool = False
update_properties(diag)

Update diagnostic properties with the provided one.

Parameters:

diag (StatisticDiag) – Diagnostic from which to get properties.

property variable: str

Name of the main variable represented by this data.

class casys.computation.diagnostics.TemporalDiag(name, store_params=None, *, data_index, data_time, start, end, field_x, field_y=None, indexer=None, time_extension=False, stats=<factory>, stat_selection=None, frequency)

Bases: BinnedDiagBase[BinnedResult, TemporalStorageProperties]

Class representing a time statistic.

Parameters:

  • name (str) – Name of the data.

  • computed – True if the data have been computed, false otherwise.

  • is_post_diag – Whether this diagnostic is a post computation diagnostic or not.

  • data_index (str) – Name of the field on which the data is indexed.

  • start (np.datetime64) – Starting date of the input dataset.

  • end (np.datetime64) – Ending date of the input dataset.

  • field_x (Field) – Field used for x axes.

  • field_y (Field | None) – Field used for y axes.

  • stats (list[StatType]) – Statistics included in this Diagnostic.

  • indexer (ong_indexer_t.PassIndexer | None) – Indexer used to read data.

  • time_extension (bool) – Whether time extension is enabled or not.

  • ext_start – Extended starting date required by the computation.

  • ext_end – Ending date for the data required by the computation.

  • group – the name of the group of this diagnostic.

  • stat_selection (ong_clip_t.Clip | None) – Selection clip used to invalidate some bins.

  • frequency (FrequencyHandler) – Frequency (day, pass, cycle).

  • store_params (dict[str, Any] | None)

  • data_time (str)

INDEX_USED: ClassVar[str] = 'INDEX_USED'
MERGEABLE_STAT: ClassVar[list[StatType]] = [MIN, MAX, COUNT, MEAN, VAR, STD, PROD, SUM]
RES_TYPE

alias of BinnedResult

SCHEMA: ClassVar[type[mm.Schema]] = None
STORABLE: ClassVar[bool] = True
apply_stats_selection(diagnostic, fields)

Filter diagnostic by applying the statistic selection clip and removing unwanted statistics.

Parameters:

  • diagnostic (TypeVar(BinnedResultType, bound= BinnedResult)) – Diagnostic to filter.

  • fields (list[str]) – List of diagnostic field’s names.

Return type:

TypeVar(BinnedResultType, bound= BinnedResult)

check_stat(stat)

Check whether this data is computed and provided stat is valid. Determines the stat if None is provided.

Parameters:

stat (StatType | None) – Statistic to check.

Return type:

StatType

Returns:

Checked statistic.

Raises:

AltiDataError – If data with the provided name were not computed. If the stat parameter is invalid.

clear_data()

Clear stored data from this Diagnostic data.

compute(data, start, end, fields, diags)

Compute the diagnostic on the provided data.

Parameters:

  • data (Dataset) – The dataset with data to read.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • diags (list[StatisticDiag]) – List of diagnostics included in this computation.

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Dictionary containing the statistic type with its associated data array.

compute_from_store(store, group, analyse_type, analyse_date, start, end, fields, delayed)

Compute diagnostic using stored data. The computation can be done in a delayed way allowing to only load requested data when needed.

Parameters:

  • store (DiagnosticStore) – Store containing diagnostics data.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • delayed (bool) – Flag indicating whether to load from store during computation or at first data request (plot/get_data).

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Computation results.

computed: bool = False
dask_full_compatible: bool = False
data_index: str
data_time: str
diag_data(**kwargs)

Diagnostic data as a dataset.

Parameters:

kwargs – Additional parameters required to retrieve diagnostic data.

Return type:

Dataset

Returns:

Dataset containing this diagnostic data.

end: np.datetime64
property ext_end: datetime64
property ext_start: datetime64
property field: Field

Field of the main variable represented by this data.

field_x: Field
field_y: Field | None = None
property fields: dict[str, Field]

Dictionary of fields used to compute this data.

fill_count_with_nan()

Whether count statistics have to be filled with nan or not.

Return type:

bool

Returns:

True or False.

finalize(result)

Correctly format the diagnostic result.

Parameters:

result (TypeVar(BinnedResultType, bound= BinnedResult)) – Result to finalize.

Return type:

TypeVar(BinnedResultType, bound= BinnedResult)

Returns:

Finalized result.

frequency: FrequencyHandler
static grid_2d(res_x, res_y)

Generate a Grid2D from provided parameters.

Parameters:
Return type:

Grid2D

Returns:

Grid2D with provided resolutions.

property grid_x: Grid1D
group: str = None
group_base: str = None
html_elem(style='')

HTML representation of the data container.

Parameters:

style (str) – Style to add to the element.

Return type:

str

Returns:

Html representation of elements of this object.

html_header(span)

HTML representation header of the data container.

Parameters:

span (int) – Span size.

Returns:

Object html header.

property identity: DiagnosticType
indexer: ong_indexer_t.PassIndexer | None = None
initialize()

Diagnostic initialization.

At least the following properties have to be initialized:

  • group_base

  • storage_properties

is_autonomous(dask)

Check whether this diagnostic is autonomous with or without dask enabled.

Parameters:

dask (bool) – Whether dask is enabled or not.

Returns:

Whether this is an autonomous diagnostic or not.

is_dask_compatible(freq)

Check whether this diagnostic can be computed with dask or not.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

Return type:

bool

is_post_diag: bool = False
load_diagnostic(**kwargs)

Ensure requested diagnostic data are loaded.

make_dask_compatible(freq)

Set this container, so it can be used with dask.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

merge(diagnostics)

Merge provided diagnostics.

Parameters:

diagnostics (list[BinnedResult]) – Diagnostics to merge.

Return type:

BinnedResult

Returns:

Merged diagnostic.

metadata(internal=False)

Diagnostic serializable configuration.

Parameters:

internal (bool) – Whether this diagnostic is an internal diagnostic or not.

Return type:

DiagnosticMeta

Returns:

Diagnostic metadata.

name: str
new_algorithm()

Instantiate the algorithm.

Return type:

BinnedStat

Returns:

Instantiated algorithm.

plot_container(template=None, stat=None, missing=False)

Create a PlotContainer for the provided stat data.

Parameters:

  • template (PlotTemplate | None) – Parameters template to use.

  • stat (StatType | None) – Statistic included in this data.

  • missing (bool) – Flag to indicate if the diagnostic is from a missing points diagnostic.

Return type:

PlotContainer

Returns:

PlotContainer containing the requested data.

results: DiagResultType
run_algorithm(data, fields)

Run the algorithm on the provided dataset.

Parameters:

  • data (Dataset) – Data on which to apply the algorithm.

  • fields (list[str]) – List of fields for which we want results.

Return type:

TypeVar(BinnedResultType, bound= BinnedResult)

Returns:

Computation result.

start: np.datetime64
stat_selection: ong_clip_t.Clip | None = None
stats: list[StatType]
storage: StoragePropertiesType = None
store(store, group, analyse_type, analyse_date, start, end, mode, fields)

Write the diagnostic results into the provided store.

Parameters:

  • store (DiagnosticStore) – DiagnosticStore used for the storage.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the computed data.

  • end (datetime64) – Ending date for the computed data.

  • mode (StorageMode) – Storage mode to use when writing data.

  • fields (list[str]) – List of all fields name used in the group of this statistic.

store_configuration(analyse_type, analyse_date, group=None)

Returns the list of storage group parameters required to store this diagnostic.

Parameters:

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date representing the set of data used in this analyse. It’s used to determine at which timestamp to store non-temporal diagnostics.

  • group (str | None) – Name of the group to use for storage.

Return type:

list[StorageGroupParams]

Returns:

List of storage group parameters.

store_params: dict[str, Any] | None = None
time_extension: bool = False
update_properties(diag)

Update diagnostic properties with the provided one.

Parameters:

diag (StatisticDiag) – Diagnostic from which to get properties.

property variable: str

Name of the main variable represented by this data.

class casys.computation.diagnostics.TemporalSwathDiag(name, store_params=None, *, data_index, data_time, start, end, field_x, field_y=None, indexer=None, time_extension=False, stats=<factory>, stat_selection=None, frequency)

Bases: TemporalDiag

Class representing a time statistic.

Parameters:

  • name (str) – Name of the data.

  • computed – True if the data have been computed, false otherwise.

  • is_post_diag – Whether this diagnostic is a post computation diagnostic or not.

  • data_index (str) – Name of the field on which the data is indexed.

  • start (np.datetime64) – Starting date of the input dataset.

  • end (np.datetime64) – Ending date of the input dataset.

  • field_x (Field) – Field used for x axes.

  • field_y (Field | None) – Field used for y axes.

  • stats (list[StatType]) – Statistics included in this Diagnostic.

  • indexer (ong_indexer_t.PassIndexer | None) – Indexer used to read data.

  • time_extension (bool) – Whether time extension is enabled or not.

  • ext_start – Extended starting date required by the computation.

  • ext_end – Ending date for the data required by the computation.

  • group – the name of the group of this diagnostic.

  • stat_selection (ong_clip_t.Clip | None) – Selection clip used to invalidate some bins.

  • frequency (FrequencyHandler) – Frequency (day, pass, cycle).

  • store_params (dict[str, Any] | None)

  • data_time (str)

INDEX_USED: ClassVar[str] = 'INDEX_USED'
MERGEABLE_STAT: ClassVar[list[StatType]] = [MIN, MAX, COUNT, MEAN, VAR, STD, PROD, SUM]
RES_TYPE

alias of BinnedResult

SCHEMA: ClassVar[type[mm.Schema]] = None
STORABLE: ClassVar[bool] = True
apply_stats_selection(diagnostic, fields)

Filter diagnostic by applying the statistic selection clip and removing unwanted statistics.

Parameters:

  • diagnostic (TypeVar(BinnedResultType, bound= BinnedResult)) – Diagnostic to filter.

  • fields (list[str]) – List of diagnostic field’s names.

Return type:

TypeVar(BinnedResultType, bound= BinnedResult)

check_stat(stat)

Check whether this data is computed and provided stat is valid. Determines the stat if None is provided.

Parameters:

stat (StatType | None) – Statistic to check.

Return type:

StatType

Returns:

Checked statistic.

Raises:

AltiDataError – If data with the provided name were not computed. If the stat parameter is invalid.

clear_data()

Clear stored data from this Diagnostic data.

compute(data, start, end, fields, diags)

Compute the diagnostic on the provided data.

Parameters:

  • data (Dataset) – The dataset with data to read.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • diags (list[StatisticDiag]) – List of diagnostics included in this computation.

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Dictionary containing the statistic type with its associated data array.

compute_from_store(store, group, analyse_type, analyse_date, start, end, fields, delayed)

Compute diagnostic using stored data. The computation can be done in a delayed way allowing to only load requested data when needed.

Parameters:

  • store (DiagnosticStore) – Store containing diagnostics data.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the data to compute.

  • end (datetime64) – Ending date for the data to compute.

  • fields (list[str]) – List of all fields to compute with this diagnostic properties.

  • delayed (bool) – Flag indicating whether to load from store during computation or at first data request (plot/get_data).

Return type:

TypeVar(DiagResultType, bound= DiagResult)

Returns:

Computation results.

computed: bool = False
dask_full_compatible: bool = False
data_index: str
data_time: str
diag_data(**kwargs)

Diagnostic data as a dataset.

Parameters:

kwargs – Additional parameters required to retrieve diagnostic data.

Return type:

Dataset

Returns:

Dataset containing this diagnostic data.

end: np.datetime64
property ext_end: datetime64
property ext_start: datetime64
property field: Field

Field of the main variable represented by this data.

field_x: Field
field_y: Field | None = None
property fields: dict[str, Field]

Dictionary of fields used to compute this data.

fill_count_with_nan()

Whether count statistics have to be filled with nan or not.

Return type:

bool

Returns:

True or False.

finalize(result)

Correctly format the diagnostic result.

Parameters:

result (TypeVar(BinnedResultType, bound= BinnedResult)) – Result to finalize.

Return type:

TypeVar(BinnedResultType, bound= BinnedResult)

Returns:

Finalized result.

frequency: FrequencyHandler
static grid_2d(res_x, res_y)

Generate a Grid2D from provided parameters.

Parameters:
Return type:

Grid2D

Returns:

Grid2D with provided resolutions.

property grid_x: Grid1D
group: str = None
group_base: str = None
html_elem(style='')

HTML representation of the data container.

Parameters:

style (str) – Style to add to the element.

Return type:

str

Returns:

Html representation of elements of this object.

html_header(span)

HTML representation header of the data container.

Parameters:

span (int) – Span size.

Returns:

Object html header.

property identity: DiagnosticType
indexer: ong_indexer_t.PassIndexer | None = None
initialize()

Diagnostic initialization.

At least the following properties have to be initialized:

  • group_base

  • storage_properties

is_autonomous(dask)

Check whether this diagnostic is autonomous with or without dask enabled.

Parameters:

dask (bool) – Whether dask is enabled or not.

Returns:

Whether this is an autonomous diagnostic or not.

is_dask_compatible(freq)

Check whether this diagnostic can be computed with dask or not.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

Return type:

bool

is_post_diag: bool = False
load_diagnostic(**kwargs)

Ensure requested diagnostic data are loaded.

make_dask_compatible(freq)

Set this container, so it can be used with dask.

Parameters:

freq (FrequencyHandler) – Frequency criteria used by dask to split data.

merge(diagnostics)

Merge provided diagnostics.

Parameters:

diagnostics (list[BinnedResult]) – Diagnostics to merge.

Return type:

BinnedResult

Returns:

Merged diagnostic.

metadata(internal=False)

Diagnostic serializable configuration.

Parameters:

internal (bool) – Whether this diagnostic is an internal diagnostic or not.

Return type:

DiagnosticMeta

Returns:

Diagnostic metadata.

name: str
new_algorithm()

Instantiate the algorithm.

Return type:

BinnedStat

Returns:

Instantiated algorithm.

plot_container(template=None, stat=None, missing=False)

Create a PlotContainer for the provided stat data.

Parameters:

  • template (PlotTemplate | None) – Parameters template to use.

  • stat (StatType | None) – Statistic included in this data.

  • missing (bool) – Flag to indicate if the diagnostic is from a missing points diagnostic.

Return type:

PlotContainer

Returns:

PlotContainer containing the requested data.

results: DiagResultType
run_algorithm(data, fields)

Run the algorithm on the provided dataset.

Parameters:

  • data (Dataset) – Data on which to apply the algorithm.

  • fields (list[str]) – List of fields for which we want results.

Return type:

TypeVar(BinnedResultType, bound= BinnedResult)

Returns:

Computation result.

start: np.datetime64
stat_selection: ong_clip_t.Clip | None = None
stats: list[StatType]
storage: StoragePropertiesType = None
store(store, group, analyse_type, analyse_date, start, end, mode, fields)

Write the diagnostic results into the provided store.

Parameters:

  • store (DiagnosticStore) – DiagnosticStore used for the storage.

  • group (str | None) – Name of the group to use for storage.

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date used to reference stored data.

  • start (datetime64) – Starting date for the computed data.

  • end (datetime64) – Ending date for the computed data.

  • mode (StorageMode) – Storage mode to use when writing data.

  • fields (list[str]) – List of all fields name used in the group of this statistic.

store_configuration(analyse_type, analyse_date, group=None)

Returns the list of storage group parameters required to store this diagnostic.

Parameters:

  • analyse_type (FreqType) – Type of period covered by this analyse (cycle, pass or custom). It’s used to determine the type of storage group to create.

  • analyse_date (datetime64) – Date representing the set of data used in this analyse. It’s used to determine at which timestamp to store non-temporal diagnostics.

  • group (str | None) – Name of the group to use for storage.

Return type:

list[StorageGroupParams]

Returns:

List of storage group parameters.

store_params: dict[str, Any] | None = None
time_extension: bool = False
update_properties(diag)

Update diagnostic properties with the provided one.

Parameters:

diag (StatisticDiag) – Diagnostic from which to get properties.

property variable: str

Name of the main variable represented by this data.