Skip to content

Commit fd0877c

Browse files
b8raoultgmertesHCookiemchantrybdvllrs
authored
feat: Better support for dynamic forcings and coupling (#262)
## Description Some refactoring of inference to allow the clean split between inputs: initial conditions, forcings and constants, so each category can be loaded from a different source. ### Variables categories For that we first categorise the variables into: - computed - forcing - diagnostic - prognostic - constant - accumulation A variable can belong to several categories, i.e. `computed` and `constant`. The runner now selects variables by categories using the new `Checkpoint.select_variables()`, which leads to a cleaner and much shorted code. ### Separate inputs The runner has now three inputs: - `input`: used to fetch the prognostics for the initial conditions (e.g. `2t` in an atmospheric model). - `constant_forcings`: used to fetch the constants for the initial conditions (e.g. `lsm` or orography) - `dynamic_forcings`: used to fetch the forcings needed be some models throughout the length of the forecast (e.g. atmospheric fields used as forcing to an ocean model) To ensure backward compatibility, unless given explicitly in the config, `constant_forcings` and `dynamic_forcings` both fallback to the `input` entry. A new config option lets the user select which category of variables are written to the output if `write_initial_condistions` is `true`. For backward compatibility, it defaults to `prognotic` and `constant_forcings` ### New commands `anemoi-inference inspect --variables checkpoint.ckpt` Will list the training variables and their categories. `anemoi-inference inspect --datasets checkpoint.ckpt` Will print the arguments passed to anemoi-dataset during training. `anemoi-inference inspect --requirements checkpoint.ckpt` Will print a Python's `requirements.txt` based on the versions of the packages used during training. ## What problem does this change solve? <!-- Describe if it's a bugfix, new feature, doc update, or breaking change --> ## What issue or task does this change relate to? Closes #245 ## Additional notes ## <!-- Include any additional information, caveats, or considerations that the reviewer should be aware of. --> ***As a contributor to the Anemoi framework, please ensure that your changes include unit tests, updates to any affected dependencies and documentation, and have been tested in a parallel setting (i.e., with multiple GPUs). As a reviewer, you are also responsible for verifying these aspects and requesting changes if they are not adequately addressed. For guidelines about those please refer to https://anemoi.readthedocs.io/en/latest/*** By opening this pull request, I affirm that all authors agree to the [Contributor License Agreement.](https://github.com/ecmwf/codex/blob/main/Legal/contributor_license_agreement.md) BEGIN_COMMIT_OVERRIDE feat: Better support for dynamic forcings and coupling in prepml (#262) feat: Add requirements subcommand (#245) feat: Lazy torch loading (#277) END_COMMIT_OVERRIDE <!-- readthedocs-preview anemoi-inference start --> ---- 📚 Documentation preview 📚: https://anemoi-inference--262.org.readthedocs.build/en/262/ <!-- readthedocs-preview anemoi-inference end --> --------- Co-authored-by: Gert Mertes <[email protected]> Co-authored-by: Harrison Cook <[email protected]> Co-authored-by: Matthew Chantry <[email protected]> Co-authored-by: bdvllrs <[email protected]>
1 parent 4b7c495 commit fd0877c

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

66 files changed

+2456
-779
lines changed

.gitignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -141,3 +141,5 @@ _version.py
141141
*.to_upload
142142
tempCodeRunnerFile.python
143143
Untitled-*.py
144+
data
145+
tmp/

docs/cli/inspect.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
.. _inspect_command:
1+
.. _inspect-command:
22

33
Inspect Command
44
===============

docs/cli/introduction.rst

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -13,9 +13,9 @@ The tools can provide help with the ``--help`` options:
1313
1414
The commands are:
1515

16-
- :ref:`Run Command <run_command>`
17-
- :ref:`Metadata Command <metadata_command>`
18-
- :ref:`Inspect Command <inspect_command>`
19-
- :ref:`Validate Command <validate_command>`
20-
- :ref:`Patch Command <patch_command>`
21-
- :ref:`Requests Command <requests_command>`
16+
- :ref:`Run Command <run-command>`
17+
- :ref:`Metadata Command <metadata-command>`
18+
- :ref:`Inspect Command <inspect-command>`
19+
- :ref:`Validate Command <validate-command>`
20+
- :ref:`Patch Command <patch-command>`
21+
- :ref:`Requests Command <requests-command>`

docs/cli/metadata.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
.. _metadata_command:
1+
.. _metadata-command:
22

33
Metadata Command
44
================

docs/cli/patch.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
.. _patch_command:
1+
.. _patch-command:
22

33
Patch Command
44
=============

docs/cli/requests.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
.. _requests_command:
1+
.. _requests-command:
22

33
Requests Command
44
================

docs/cli/run.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
.. _run_command:
1+
.. _run-command:
22

33
Run Command
44
===========

docs/cli/validate.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
.. _validate_command:
1+
.. _validate-command:
22

33
Validate Command
44
================

docs/index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -128,6 +128,7 @@ You may also have to install pandoc on MacOS:
128128
:hidden:
129129
:caption: User Guide
130130

131+
inference/input-types
131132
inference/parallel
132133
inference/apis/introduction
133134
inference/configs/introduction

docs/inference/apis/level3.rst

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
##################
66

77
You can run the inference from the command line using the
8-
:ref:`anemoi-inference run <run_command>` command.
8+
:ref:`anemoi-inference run <run-command>` command.
99

1010
You must first create a configuration file in YAML format. The simplest
1111
configuration must contain the path to the checkpoint:
@@ -66,7 +66,7 @@ checkpoint that was trained with one the ICON grid:
6666
.. literalinclude:: code/level3_4.yaml
6767
:language: yaml
6868

69-
See :ref:`run_command` for more details on the configuration file.
69+
See :ref:`run-command` for more details on the configuration file.
7070

7171
.. warning::
7272

0 commit comments

Comments
 (0)