disdrodb.l0.readers package

Submodules

disdrodb.l0.readers.reader_template module

disdrodb.l0.readers.reader_template.reader(raw_dir, processed_dir, station_name, force=False, verbose=False, parallel=False, debugging_mode=False)[source]

Script to convert the raw data to L0A format.

Parameters:

  • raw_dir (str) –

    The directory path where all the raw content of a specific campaign is stored. The path must have the following structure:

    <…>/DISDRODB/Raw/<data_source>/<campaign_name>’.

    Inside the raw_dir directory, it is required to adopt the following structure: - /data/<station_name>/<raw_files> - /metadata/<station_name>.yaml Important points: - For each <station_name> there must be a corresponding YAML file in the metadata subfolder. - The <campaign_name> must semantically match between:

    • the raw_dir and processed_dir directory paths;

    • with the key ‘campaign_name’ within the metadata YAML files.

    • The campaign_name are expected to be UPPER CASE.

  • processed_dir (str) –

    The desired directory path for the processed DISDRODB L0A and L0B products. The path should have the following structure:

    <…>/DISDRODB/Processed/<data_source>/<campaign_name>’

    For testing purpose, this function exceptionally accept also a directory path simply ending with <campaign_name> (i.e. /tmp/<campaign_name>).

  • station_name (str) – Station name

  • force (bool) – If True, overwrite existing data into destination directories. If False, raise an error if there are already data into destination directories. The default is False.

  • verbose (bool) – Whether to print detailed processing information into terminal. The default is True.

  • parallel (bool) – If True, the files are processed simultanously in multiple processes. The number of simultaneous processes can be customized using the dask.distributed LocalCluster. If False, the files are processed sequentially in a single process. If False, multi-threading is automatically exploited to speed up I/0 tasks.

  • debugging_mode (bool) – If True, it reduces the amount of data to process. It processes just the first 3 raw data files. The default is False.

Module contents