dsbuilder: creating datasets, to specification

dsbuilder is a python module to help you build and write data files (primarily NetCDF) with a specified format of dimensions, variables and metadata etc. It is built on top of xarray.

Although xarray itself provides a simple interface for creating datasets, building complex datasets to particular standards in a documented way can become a bit of a pain. Particularly when defining standard variable formats, e.g. flags or uncertainties. dsbuilder looks to solve this problem by building template datasets with easily defined dictionaries, that may be populated with data and written to file using xarray’s built in functionality.

A simple dataset containing a radiance image may be built as follows:

from dsbuilder import create_template_dataset

ds_variables = {
    "radiance": {
        "dim": ["lambda", "x", "y"],
        "dtype": np.float32,
        "attributes": {
            "standard_name": "toa_outgoing_radiance_per_unit_wavelength",
            "long_name": "'toa' means top of atmosphere. The TOA outgoing radiance"
                         " is the upwelling radiance, i.e., toward outer space. "
                         "Radiance is the radiative flux in a particular direction, "
                         "per unit of solid angle. In accordance with common usage "
                         "in geophysical disciplines, 'flux' implies per unit area, "
                         "called 'flux density' in physics",
            "units": "W m-2 sr-1 m-1",
            "preferred_symbol": "L"
        }
    }
}

ds = create_template_dataset(
    variables_dict = ds_variables,
    dim_sizes_dict = {"lambda": 10, "x": 1000, "y": 1000},
)

ds["radiance"] = ... # populate variable with radiance image array

ds.to_netcdf("path/to/file.nc")

dsbuilder is developed within Earth Observation metrology community, so it is designed with adhering to the cf conventions particularly in mind. Eventually, dsbuilder will do this more automatically.

For more information see the Users and Developer documentation.

For Users

For Developers

Contributions

dsbuilder is written and maintained by Sam Hunt.