qblox_scheduler.helpers.dataset_adapters

Utilities for dataset (python object) handling.

Classes

DatasetAdapterBase	A generic interface for a dataset adapter.
DatasetAdapterIdentity	A dataset adapter that does not modify the datasets in any way.
AdapterH5NetCDF	Qblox Scheduler dataset adapter for the h5netcdf engine.

Module Contents

class DatasetAdapterBase

A generic interface for a dataset adapter.

Note

It might be difficult to grasp the generic purpose of this class. See AdapterH5NetCDF for a specialized use case.

A dataset adapter is intended to “adapt”/”convert” a dataset to a format compatible with some other piece of software such as a function, interface, read/write back end, etc.. The main use case is to define the interface of the AdapterH5NetCDF that converts the Qblox Scheduler dataset for loading and writing to/from disk.

Subclasses implementing this interface are intended to be a two-way bridge to some other object/interface/backend to which we refer to as the “Target” of the adapter.

The function .adapt() should return a dataset to be consumed by the Target.

The function .recover() should receive a dataset generated by the Target.

classmethod adapt(dataset: xarray.Dataset) → xarray.Dataset

Abstractmethod:

Converts the dataset to a format consumed by the Target.

classmethod recover(dataset: xarray.Dataset) → xarray.Dataset

Abstractmethod:

Inverts the action of the .adapt() method.

class DatasetAdapterIdentity

A dataset adapter that does not modify the datasets in any way.

Intended to be used just as an object that respects the adapter interface defined by DatasetAdapterBase.

A particular use case is the backwards compatibility for loading and writing older versions of the Qblox Scheduler dataset.

classmethod adapt(dataset: xarray.Dataset) → xarray.Dataset

Returns:

: Same dataset with no modifications.

classmethod recover(dataset: xarray.Dataset) → xarray.Dataset

Returns:

: Same dataset with no modifications.

class AdapterH5NetCDF

Bases: DatasetAdapterBase

Qblox Scheduler dataset adapter for the h5netcdf engine.

It has the functionality of adapting the Qblox Scheduler dataset to a format compatible with the h5netcdf xarray backend engine that is used to write and load the dataset to/from disk.

Warning

The h5netcdf engine has minor issues when performing a two-way trip of the dataset. The type of some attributes are not preserved. E.g., list- and tuple-like objects are loaded as numpy arrays of dtype=object.

classmethod adapt(dataset: xarray.Dataset) → xarray.Dataset

Serializes to JSON the dataset and variables attributes.

To prevent the JSON serialization for specific items, their names should be listed under the attribute named json_serialize_exclude (for each attrs dictionary).

Parameters:

dataset – Dataset that needs to be adapted.

Returns:

: Dataset in which the attributes have been replaced with their JSON strings version.

classmethod recover(dataset: xarray.Dataset) → xarray.Dataset

Reverts the action of .adapt().

To prevent the JSON de-serialization for specific items, their names should be listed under the attribute named json_serialize_exclude (for each attrs dictionary).

Parameters:

dataset – Dataset from which to recover the original format.

Returns:

: Dataset in which the attributes have been replaced with their python objects version.

static attrs_convert(attrs: dict, inplace: bool = False, vals_converter: collections.abc.Callable[..., Any] = json.dumps) → dict

Converts to/from JSON string the values of the keys which are not listed in the json_serialize_exclude list.

Parameters:

• attrs – The input dictionary.

• inplace – If True the values are replaced in place, otherwise a deepcopy of attrs is performed first.

• vals_converter – A serializer. By default json.dumps.

classmethod _transform(dataset: xarray.Dataset, vals_converter: collections.abc.Callable[..., Any] = json.dumps) → xarray.Dataset