inputs
- class anemoi.inference.input.Input(context: Context, metadata: Metadata, *, variables: list[str] | None = None, pre_processors: list[str | dict[str, Any]] | None = None, purpose: str | None = None)
Bases:
ABCAbstract base class for input handling.
Initialise the Input object.
- Parameters:
context (Context) – The context for the input.
metadata (Metadata) – Metadata corresponding to the dataset this input is handling.
variables (list of str or None) – List of variable names to be handled by the input, or None for all available variables.
pre_processors (list of ProcessorConfig or None, optional) – List of pre-processors to apply to the input. If None, no pre-processing is performed.
purpose (str or None, optional) – The purpose of the input (e.g., ‘forcings’, ‘constants’). Used for debugging and logging.
- pre_process(x: Any) Any
Run pre-processors.
- Parameters:
x (Any) – Input to pre-process.
- Returns:
Pre-processed input
- Return type:
Any
- abstractmethod create_input_state(*, dates: list[str | datetime | int], **kwargs) dict[str, Any]
Create the input state dictionary.
- abstractmethod load_forcings_state(*, dates: list[str | datetime | int], current_state: dict[str, Any]) dict[str, Any]
Load forcings (constant and dynamic).
- input_variables() list[str]
Return the list of input variables.
- Returns:
The list of input variables.
- Return type:
List[str]
- patch_data_request(request: Any) Any
Patch the data request.
Uses both the context and input preprocessors.
- Parameters:
request (Any) – The data request.
- Returns:
The patched data request.
- Return type:
Any
- set_private_attributes(state: dict[str, Any], value: Any) None
Provide a way to a subclass to set private attributes in the state dictionary, that may be needed by the output object.
- Parameters:
state (State) – The state dictionary.
value (Any) – The value to set.
- default_initial_date() datetime
Return the initial date to use when none is given in the config (assuming a forecasting context).
The runner calls this when date isn’t set in the config or on the CLI. Each input decides how to pick the date: some read it from the data (e.g. the latest valid time in a file), others use a fixed rule (e.g. yesterday at 00:00), and some don’t implement this method at all.
inputs
cds
- anemoi.inference.inputs.cds.retrieve(requests: list[dict[str, Any]], grid: str | list[float] | None, area: list[float] | str | None, dataset: str | dict[str, Any], **kwargs: Any) FieldList
Retrieve data from CDS.
- Parameters:
- Returns:
Retrieved data.
- Return type:
Any
- class anemoi.inference.inputs.cds.CDSInput(context: Context, metadata: Metadata, *, variables: list[str] | None = None, pre_processors: list[str | dict[str, Any]] | None = None, dataset: str | dict[str, Any], namer: Any | None = None, purpose: str | None = None, **kwargs: Any)
Bases:
GribInputGet input fields from CDS.
Initialize the CDSInput.
- Parameters:
context (Context) – The context in which the input is used.
metadata (Metadata) – Metadata corresponding to the dataset this input is handling.
variables (list[str] | None) – List of variables to be handled by the input, or None for a sensible default variables.
pre_processors (Optional[List[ProcessorConfig]], default None) – Pre-processors to apply to the input
namer (Optional[Any]) – Optional namer for the input.
**kwargs (Any) – Additional keyword arguments.
- trace_name = 'cds'
- create_input_state(*, dates: list[str | datetime | int], **kwargs) dict[str, Any]
Create the input state for the given date.
- retrieve(variables: list[str], dates: list[str | datetime | int]) Any
Retrieve data for the given variables and dates.
- load_forcings_state(*, dates: list[str | datetime | int], current_state: dict[str, Any]) dict[str, Any]
Load the forcings state for the given variables and dates.
- default_initial_date() datetime
Return the initial date to use when none is given in the config (assuming a forecasting context).
The runner calls this when date isn’t set in the config or on the CLI. Each input decides how to pick the date: some read it from the data (e.g. the latest valid time in a file), others use a fixed rule (e.g. yesterday at 00:00), and some don’t implement this method at all.
cutout
- class anemoi.inference.inputs.cutout.Cutout(context: Context, metadata: Metadata, *args: dict[str, dict], sources: list[dict[str, dict]] | None = None, **kwargs)
Bases:
InputCombines one or more LAMs into a global source using cutouts.
Create a cutout input from a list of sources.
- Parameters:
- create_input_state(*, dates: list[str | datetime | int], **kwargs) dict[str, Any]
Create the input state for the given date.
dataset
- class anemoi.inference.inputs.dataset.DatasetInput(context: Context, metadata: Metadata, *, open_dataset_args: tuple[Any, ...], open_dataset_kwargs: dict[str, Any], grid_indices: Any = None, use_trajectories: bool = False, **kwargs: Any)
Bases:
InputHandles anemoi-datasets dataset as input.
Initialize the DatasetInput.
- Parameters:
context (Any) – The context in which the input is used.
metadata (Metadata) – Metadata corresponding to the dataset this input is handling.
open_dataset_args (Tuple[Any, ...]) – Arguments for the dataset.
open_dataset_kwargs (Dict[str, Any]) – Keyword arguments for the dataset.
grid_indices (Optional[Any]) – Indices to reduce the input grid. If None, the full grid is used.
use_trajectories (bool) – Whether to expect dataset as a trajectory (i.e. with a step / forecast dimension), and to get multiple dates from within a single trajectory.
**kwargs (Any) – Additional keyword arguments.
- property ds_dates: ndarray
Return the dates of the dataset.
- create_input_state(*, dates: list[str | datetime | int], ref_date_index: int = -1, **kwargs) dict[str, Any]
Create the input state for the given date.
- class anemoi.inference.inputs.dataset.DatasetInputArgsKwargs(context: Context, metadata: Metadata, *args: Any, use_original_paths: bool = False, variables: list[str] | None, pre_processors: list[str | dict[str, Any]] | None = None, grid_indices=None, purpose: str | None = None, use_trajectories: bool = False, **kwargs: Any)
Bases:
DatasetInputHandles anemoi-datasets dataset as input.
Initialize the DatasetInputArgsKwargs.
- Parameters:
context (Context) – The context in which the input is used.
metadata (Metadata) – Metadata corresponding to the dataset this input is handling.
use_original_paths (bool) – Whether to use original paths.
use_trajectories (bool) – Whether to expect dataset as a trajectory (i.e. with a step / forecast dimension), and to get multiple dates from within a single trajectory.
- trace_name = 'dataset/provided'
- class anemoi.inference.inputs.dataset.DataloaderInput(context: Context, metadata: Metadata, *, use_original_paths: bool = False, **kwargs: Any)
Bases:
DatasetInputHandles anemoi-datasets dataset as input.
Initialize the DataloaderInput.
- class anemoi.inference.inputs.dataset.TestInput(context: Context, metadata: Metadata, *, use_original_paths: bool = False, **kwargs: Any)
Bases:
DataloaderInputHandles anemoi-datasets dataset as input.
Initialize the DataloaderInput.
- Parameters:
- trace_name = 'dataset/test'
- name = 'test'
- class anemoi.inference.inputs.dataset.ValidationInput(context: Context, metadata: Metadata, *, use_original_paths: bool = False, **kwargs: Any)
Bases:
DataloaderInputHandles anemoi-datasets dataset as input.
Initialize the DataloaderInput.
- Parameters:
- trace_name = 'dataset/validation'
- name = 'validation'
- class anemoi.inference.inputs.dataset.TrainingInput(context: Context, metadata: Metadata, *, use_original_paths: bool = False, **kwargs: Any)
Bases:
DataloaderInputHandles anemoi-datasets dataset as input.
Initialize the DataloaderInput.
- Parameters:
- trace_name = 'dataset/training'
- name = 'training'
dummy
Dummy input used for testing.
It will generate fields with constant values for each variable and date. These values are then tested in the mock model.
- class anemoi.inference.inputs.dummy.DummyInput(context: Context, metadata: Metadata, **kwargs)
Bases:
EkdInputDummy input used for testing.
Initialize the EkdInput.
- Parameters:
- trace_name = 'dummy'
- create_input_state(*, dates: list[str | datetime | int], **kwargs) dict[str, Any]
Create the input state for the given date.
- load_forcings_state(*, dates: list[str | datetime | int], current_state: dict[str, Any]) dict[str, Any]
Load the forcings state for the given variables and dates.
- default_initial_date() datetime
Return the initial date to use when none is given in the config (assuming a forecasting context).
The runner calls this when date isn’t set in the config or on the CLI. Each input decides how to pick the date: some read it from the data (e.g. the latest valid time in a file), others use a fixed rule (e.g. yesterday at 00:00), and some don’t implement this method at all.
ekd
- anemoi.inference.inputs.ekd.find_variable(data: FieldList, name: str, namer: callable, **kwargs: Any) FieldList
Find a variable in an earthkit FieldList.
- Parameters:
data (ekd.FieldList) – The data to search.
name (str) – The name of the variable to find.
namer (callable) – The namer function to use for naming fields.
**kwargs (Any) – Additional arguments for selecting the variable.
- Returns:
The selected variable (FieldList subset).
- Return type:
ekd.FieldList
- class anemoi.inference.inputs.ekd.RulesNamer(rules: Any, default_namer: Callable[[Any, dict[str, Any]], str])
Bases:
objectA namer that uses rules to generate names.
Initialize the RulesNamer.
- Parameters:
empty
Dummy input used for testing.
It will generate fields with constant values for each variable and date. These values are then tested in the mock model.
- class anemoi.inference.inputs.empty.EmptyInput(context: Context, metadata: Metadata, **kwargs: Any)
Bases:
InputAn Input that is always empty.
Initialise the EmptyInput.
- Parameters:
- trace_name = 'empty'
- create_input_state(*, dates: list[str | datetime | int], **kwargs) dict[str, Any]
Create an empty input state.
fdb
- class anemoi.inference.inputs.fdb.FDBInput(context: Context, metadata: Metadata, *, fdb_config: dict | None = None, fdb_userconfig: dict | None = None, variables: list[str] | None = None, pre_processors: list[str | dict[str, Any]] | None = None, namer: Any | None = None, purpose: str | None = None, **kwargs: Any)
Bases:
GribInputGet input fields from FDB.
Initialise the FDB input.
- Parameters:
context (Context) – The context for the input.
metadata (Metadata) – Metadata corresponding to the dataset this input is handling.
fdb_config (dict, optional) – The FDB config to use.
fdb_userconfig (dict, optional) – The FDB userconfig to use.
variables (list[str] | None) – List of variables to be handled by the input, or None for a sensible default variables.
pre_processors (list[ProcessorConfig], optional) – Pre-processors to apply to the retrieved data.
namer (Optional[Any]) – Optional namer for the input.
purpose (str, optional) – The purpose of the input.
kwargs (dict, optional) – Additional keyword arguments for the request to FDB.
- trace_name = 'fdb'
- create_input_state(*, dates: list[str | datetime | int], **kwargs) dict[str, Any]
Create the input state dictionary.
grib
gribfile
mars
- anemoi.inference.inputs.mars.rounded_area(area: list[float] | None) list[float] | None
Round the area to a global extent if the surface is greater than 0.98.
- anemoi.inference.inputs.mars.grid_is_valid(grid: str | list[float] | None) bool
Check if the grid is valid.
- anemoi.inference.inputs.mars.area_is_valid(area: list[float] | None) bool
Check if the area is valid.
- anemoi.inference.inputs.mars.postproc(grid: str | list[float] | None, area: list[float] | str | None) dict[str, str | list[float]]
Post-process the grid and area.
- anemoi.inference.inputs.mars.retrieve(requests: list[dict[str, Any]], grid: str | list[float] | None, area: list[float] | None, patch: Any | None = None, log: bool = True, **kwargs: Any) Any
Retrieve data from MARS.
- Parameters:
requests (List[Dict[str, Any]]) – The list of requests to be retrieved.
grid (Optional[Union[str, List[float]]]) – The grid for the retrieval.
area (Optional[List[float]]) – The area for the retrieval.
patch (Optional[Any], optional) – Optional patch for the request, by default None.
log (bool, optional) – Whether to log the requests, by default True.
**kwargs (Any) – Additional keyword arguments.
- Returns:
The retrieved data.
- Return type:
Any
- class anemoi.inference.inputs.mars.MarsInput(context: Context, metadata: Metadata, *, variables: list[str] | None = None, patches: list[tuple[dict[str, Any], dict[str, Any]]] | None = None, log: bool = True, pre_processors: list[str | dict[str, Any]] | None = None, namer: Any | None = None, purpose: str | None = None, **kwargs: Any)
Bases:
GribInputGet input fields from MARS.
Initialize the MarsInput.
- Parameters:
context (Any) – The context in which the input is used.
metadata (Metadata) – Metadata corresponding to the dataset this input is handling.
variables (list[str] | None) – List of variables to be handled by the input, or None for a sensible default variables.
namer (Optional[Any]) – Optional namer for the input.
patches (Optional[List[Tuple[Dict[str, Any], Dict[str, Any]]]]) – Optional list of patches for the input.
log (bool) – Whether to log the requests to MARS, by default True.
**kwargs (Any) – Additional keyword to pass to the request to MARS.
- trace_name = 'mars'
- create_input_state(*, dates: list[str | datetime | int], ref_date_index=-1, **kwargs) dict[str, Any]
Create the input state for the given date.
- retrieve(variables: list[str], dates: list[str | datetime | int]) Any
Retrieve data for the given variables and dates.
- Parameters:
variables (List[str]) – The list of variables to retrieve.
dates (List[Any]) – The list of dates for which to retrieve the data.
- Returns:
The retrieved data.
- Return type:
Any
- load_forcings_state(*, dates: list[str | datetime | int], current_state: dict[str, Any]) dict[str, Any]
Load the forcings state for the given variables and dates.
- default_initial_date() datetime
Return the initial date to use when none is given in the config (assuming a forecasting context).
The runner calls this when date isn’t set in the config or on the CLI. Each input decides how to pick the date: some read it from the data (e.g. the latest valid time in a file), others use a fixed rule (e.g. yesterday at 00:00), and some don’t implement this method at all.
netcdf
opendap
repeated_dates
- class anemoi.inference.inputs.repeated_dates.RepeatedDatesInput(context: Context, metadata: Metadata, *, source: str, mode: str = 'constant', **kwargs: Any)
Bases:
InputThis class is identical to the one used to in anemoi-datasets/create It uses a source of constants (e.g. a source containing the bathymetry) available only for a given date and returns its content whever date is requested by the runner
Initialise the Input object.
- Parameters:
context (Context) – The context for the input.
metadata (Metadata) – Metadata corresponding to the dataset this input is handling.
variables (list of str or None) – List of variable names to be handled by the input, or None for all available variables.
pre_processors (list of ProcessorConfig or None, optional) – List of pre-processors to apply to the input. If None, no pre-processing is performed.
purpose (str or None, optional) – The purpose of the input (e.g., ‘forcings’, ‘constants’). Used for debugging and logging.
- trace_name = 'repeated dates'
- create_input_state(*, dates: list[str | datetime | int], ref_date_index=-1, **kwargs) dict[str, Any]
Create the input state for the repeated-dates input.
split
- class anemoi.inference.inputs.split.SplitInput(context: Context, metadata: Metadata, *splits, **kwargs: Any)
Bases:
InputInitialise the Input object.
- Parameters:
context (Context) – The context for the input.
metadata (Metadata) – Metadata corresponding to the dataset this input is handling.
variables (list of str or None) – List of variable names to be handled by the input, or None for all available variables.
pre_processors (list of ProcessorConfig or None, optional) – List of pre-processors to apply to the input. If None, no pre-processing is performed.
purpose (str or None, optional) – The purpose of the input (e.g., ‘forcings’, ‘constants’). Used for debugging and logging.
- trace_name = 'split input'
- create_input_state(*, dates: list[str | datetime | int], ref_date_index=-1, **kwargs) dict[str, Any]
Create the input state for the repeated-dates input.