Click here to download this notebook.

How to use annex tables

When working with CLS tables you might want to save your own editings or processings done with these data.

This can be done using annex tables, a mechanism allowing to use a common read-only reference table while being able to write in a personal annex table.

Warning: Using the AnnexTableManager will change the environment variable GES_TABLE_DIR used to access CLS tables. If a table was already open (using NadirData or any TableMeasure object) this operation will fail. You need to restart your kernel !

The AnnexTableManager is in charge of managing your “annex” environment. It creates and update all the link to the reference tables and allows you to create your personal AnnexTable.

Parameters:

• location: Location of your annex tables.

• [optional]reference: Location of the reference tables.

Methods:

• create_annex: Create a new annex table.

  • name: Name of the annex table.

  • reference: Name of the reference table.

  • fields: List of fields to include in the annex table.

• remove_annex: Remove the provided annex table data and configuration files.

  • name: Name of the annex table.

• store_data: Store the provided data into the annex table (Only fields declared at the annex ta creation are stored).

  • annex: Name of the annex table.

  • data: xarray dataset containing fields to store.

Create the annex table

In the following example, we create a new table named TABLE_C_J3_CUSTOM based on TABLE_C_J3_B_GDRD with 5 new fields.

Info: Might want to restart the kernel here

os.makedirs(location, exist_ok=True)

anx_manager = AnnexTableManager(location=location, reference=reference)

# Define your fields
field_edt1 = Field(name="EDITING_1", description="Editing 1 based on ... (xx)")
field_edt2 = Field(name="EDITING_2", description="Editing 2 based on ... (xx)")
field_edt3 = Field(name="EDITING_3", description="Editing 3 based on ... (xx)")
field_custom1 = Field(
    name="FIELD_CUSTOM_1", description="Distance in cm ... (xx)", unit="cm"
)
field_custom2 = Field(
    name="FIELD_CUSTOM_2", description="Area in m^2 ... (xx)", unit="m**2"
)

new_fields = [field_edt1, field_edt2, field_edt3, field_custom1, field_custom2]

Warning: Only do this step once

anx_manager.create_annex(
    name="TABLE_C_J3_CUSTOM", reference="TABLE_C_J3_B_GDRD", fields=new_fields
)

Use the annex table to computed new fields

# Use your table

start = DateHandler.from_orf(orf="C_J3_GDRD", cycle_nb=122, pass_nb=1, pos="first")
end = DateHandler.from_orf(orf="C_J3_GDRD", cycle_nb=122, pass_nb=2, pos="last")

reader = CLSTableReader(
    name="TABLE_C_J3_CUSTOM",
    date_start=start,
    date_end=end,
    orf="C_J3_GDRD",
    time="time",
    longitude="LONGITUDE",
    latitude="LATITUDE",
)

ad_anx = NadirData(source=reader)

# New fields are included
ad_anx.show_fields(containing="(xx)")

Name	Description	Unit
EDITING_1	Editing 1 based on ... (xx)	unknown
EDITING_2	Editing 2 based on ... (xx)	unknown
EDITING_3	Editing 3 based on ... (xx)	unknown
FIELD_CUSTOM_1	Distance in cm ... (xx)	cm
FIELD_CUSTOM_2	Area in m^2 ... (xx)	m**2

# Already existing ones too
ad_anx.show_fields(containing="wind")

Name	Description	Unit
DYNAMICAL_ATMOSPHERIC_CORRECTION.MODEL.ERA_INTERIM	MOG2D high resolution forced with ERA INTERIM pressure and wind fields plus inverse barometer. This correction is computed by a	m
DYNAMICAL_ATMOSPHERIC_CORRECTION.MODEL.MOG2D_HR	MOG2D high resolution forced with ECMWF pressure and wind fields plus inverse barometer. This correction is computed by adding	m
WIND_SPEED.ALTI	Wind Speed modulus computed from the radar echoes. Should not be used over land.	m/s
WIND_SPEED.ALTI.RTK_MLE3	Wind Speed modulus computed from the radar echoes. Should not be used over land.	m/s
WIND_SPEED.ALTI.TRAN2015_2D_MLE4_BIASED	Wind speed Tran2015	m/s
WIND_SPEED.MODEL	Wind speed from model, should not be used over land. This atmospheric correction is provided courtesy of ECMWF.	m/s
WIND_SPEED.MODEL.ERA_INTERIM	Wind speed from ERA model. Should not be used over land. The main interest of using this model is to have a continuity in the c	m/s
WIND_SPEED.MODEL.U_COMP	U component of the model wind vector at the altimeter time-tag.	m/s
WIND_SPEED.MODEL.V_COMP	V component of the model wind vector at the altimeter time tag	m/s
WIND_SPEED.RAD	Radiometer wind speed	m/s

New fields might be created using NadirData diagnostics such as the Editing or manually:

ad_anx.add_raw_data(name="Wind speed", field=Field(name="WIND_SPEED.ALTI"))

ad_anx.compute()

2025-05-14 11:00:27 INFO    Reading ['LONGITUDE', 'LATITUDE', 'WIND_SPEED.ALTI', 'time']
2025-05-14 11:00:28 INFO    Computing done.

ds = ad_anx.data

ds[field_edt1.name] = xr.where(ds["WIND_SPEED.ALTI"] > 5, 1, 0)
ds[field_edt2.name] = xr.where(ds["WIND_SPEED.ALTI"] > 8, 1, 0)
ds[field_custom1.name] = ds["WIND_SPEED.ALTI"] * 36 + 4

ds

<xarray.Dataset> Size: 372kB
Dimensions:          (time: 6637)
Coordinates:
  * time             (time) datetime64[ns] 53kB 2019-06-01T05:30:29.807497 .....
Data variables:
    LONGITUDE        (time) float64 53kB 17.01 17.14 17.27 ... -11.53 -11.4
    LATITUDE         (time) float64 53kB -66.15 -66.15 -66.15 ... -66.15 -66.15
    WIND_SPEED.ALTI  (time) float64 53kB nan nan nan nan ... 9.85 10.19 7.84
    EDITING_1        (time) int64 53kB 0 0 0 0 0 0 0 0 0 0 ... 1 1 1 1 1 1 1 1 1
    EDITING_2        (time) int64 53kB 0 0 0 0 0 0 0 0 0 0 ... 1 1 1 1 1 1 1 1 0
    FIELD_CUSTOM_1   (time) float64 53kB nan nan nan nan ... 358.6 370.8 286.2

xarray.Dataset

• Dimensions:
  • time: 6637
• Coordinates: (1)
  • time
    (time)
    datetime64[ns]
    2019-06-01T05:30:29.807497 ... 2...

    standard_name :

    time

    array(['2019-06-01T05:30:29.807497000', '2019-06-01T05:30:30.826208000',
           '2019-06-01T05:30:31.844918000', ..., '2019-06-01T07:22:53.047737000',
           '2019-06-01T07:22:54.066446000', '2019-06-01T07:22:55.085157000'],
          dtype='datetime64[ns]')

• Data variables: (6)
  • LONGITUDE
    (time)
    float64
    17.01 17.14 17.27 ... -11.53 -11.4

    array([ 17.006113,  17.135617,  17.265119, ..., -11.663281, -11.533787,
           -11.404292])

  • LATITUDE
    (time)
    float64
    -66.15 -66.15 ... -66.15 -66.15

    array([-66.149034, -66.148977, -66.148805, ..., -66.147316, -66.147546,
           -66.14766 ])

  • WIND_SPEED.ALTI
    (time)
    float64
    nan nan nan nan ... 9.85 10.19 7.84

    array([  nan,   nan,   nan, ...,  9.85, 10.19,  7.84])

  • EDITING_1
    (time)
    int64
    0 0 0 0 0 0 0 0 ... 1 1 1 1 1 1 1 1

    array([0, 0, 0, ..., 1, 1, 1])

  • EDITING_2
    (time)
    int64
    0 0 0 0 0 0 0 0 ... 1 1 1 1 1 1 1 0

    array([0, 0, 0, ..., 1, 1, 0])

  • FIELD_CUSTOM_1
    (time)
    float64
    nan nan nan ... 358.6 370.8 286.2

    array([   nan,    nan,    nan, ..., 358.6 , 370.84, 286.24])

• Indexes: (1)
  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['2019-06-01 05:30:29.807497', '2019-06-01 05:30:30.826208',
                   '2019-06-01 05:30:31.844918', '2019-06-01 05:30:32.863628',
                   '2019-06-01 05:30:33.882337', '2019-06-01 05:30:34.901048',
                   '2019-06-01 05:30:35.919757', '2019-06-01 05:30:36.938466',
                   '2019-06-01 05:30:37.957177', '2019-06-01 05:30:38.975886',
                   ...
                   '2019-06-01 07:22:45.916766', '2019-06-01 07:22:46.935478',
                   '2019-06-01 07:22:47.954186', '2019-06-01 07:22:48.972898',
                   '2019-06-01 07:22:49.991606', '2019-06-01 07:22:51.010317',
                   '2019-06-01 07:22:52.029027', '2019-06-01 07:22:53.047737',
                   '2019-06-01 07:22:54.066446', '2019-06-01 07:22:55.085157'],
                  dtype='datetime64[ns]', name='time', length=6637, freq=None))

• Attributes: (0)

Store newly created data

anx_manager.store_data(annex="TABLE_C_J3_CUSTOM", data=ds)

Make diagnostics using these new data

# Creating a field based on newly added ones

field_custom1_edited = Field(
    name="Custom_Edited_Field",
    source=f"IIF({field_edt1.name} :== 1, {field_custom1.name}, DV)",
)

field_custom1_edited

Name	Description	Unit
Custom_Edited_Field	IIF(EDITING_1 :== 1, FIELD_CUSTOM_1, DV)	unknown

ad_anx.add_raw_data(name="My editing 1", field=field_edt1)
ad_anx.add_raw_data(name="My custom field 1", field=field_custom1)

ad_anx.add_time_stat(
    name="Stat made based on my custom field",
    field=field_custom1_edited,
    freq="20min",
    stats=["mean", "median", "std"],
)

ad_anx.compute()

2025-05-14 11:00:28 INFO    Reading ['FIELD_CUSTOM_1', 'EDITING_1', 'Custom_Edited_Field']
2025-05-14 11:00:28 INFO    Computing diagnostics ['Stat made based on my custom field']
2025-05-14 11:00:28 INFO    Computing done.

cf_plot = CasysPlot(
    data=ad_anx, data_name="Stat made based on my custom field", stat="median"
)

cf_plot.show()

# Cleaning everything
shutil.rmtree(path=location)