Skip to content

Conversation

enssow
Copy link
Contributor

@enssow enssow commented Oct 3, 2025

Description

  • Adds default config so uv run --offline evaluate is all required
  • Had to change some values in the plot_config_template.yml
  1. original run_id couldnt be found so changed to grwnhykd
  2. changed results_base_dir to "/p/scratch/weatherai/shared_work/results/" instead of "./results"

Issue Number

Closes #693

Is this PR a draft? Mark it as draft.

Checklist before asking for review

  • I have performed a self-review of my code
  • My changes comply with basic sanity checks:
    • I have fixed formatting issues with ./scripts/actions.sh lint
    • I have run unit tests with ./scripts/actions.sh unit-test
    • I have documented my code and I have updated the docstrings.
    • I have added unit tests, if relevant
  • I have tried my changes with data and code:
    • I have run the integration tests with ./scripts/actions.sh integration-test: FAILS
    • (bigger changes) I have run a full training and I have written in the comment the run_id(s): launch-slurm.py --time 60
    • (bigger changes and experiments) I have shared a hegdedoc in the github issue with all the configurations and runs for this experiments
  • I have informed and aligned with people impacted by my change:
    • for config changes: the MatterMost channels and/or a design doc
    • for changes of dependencies: the MatterMost software development channel

@enssow enssow marked this pull request as draft October 3, 2025 18:47
@enssow
Copy link
Contributor Author

enssow commented Oct 3, 2025

TO DISCUSS: what is best run_id to have as default

Copy link
Collaborator

@iluise iluise left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!
About your point on the Run_Id: I'd just copy paste the config that we have now in the WG and keep the run_id we have there. it is just to have an example then people will change it as they like.

@enssow enssow marked this pull request as ready for review October 6, 2025 10:44
Copy link
Contributor

@MatKbauer MatKbauer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking this over, Sorcha. I have added a couple of comments to improve applicability accross HPCs and prevent additional workload when the evaluation pipeline changes.

I verified the code with the changes I have made. It runs without passing --config config/eval_config.yml. Kindly have a look; curious to hear your opinion.

@github-project-automation github-project-automation bot moved this to In Progress in WeatherGen-dev Oct 6, 2025
@MatKbauer
Copy link
Contributor

Another thought: How about adding the config/eval_config.yml as default argument to the eval parser? Something like

parser.add_argument(
        "--config",
        type=str,
        default=Path("config/eval_config.yml"),
        help="Path to the configuration yaml file for plotting. e.g. config/eval_config.yaml",
    )

Then we would not have to do the if/else check further below.

@enssow
Copy link
Contributor Author

enssow commented Oct 6, 2025

My only concern with the default option is that the user won't know that the default is being parsed (potentially not a good idea considering it's to be used as a template). Will discuss in the evaluation meeting tomorrow and resolve the final comments then! Thank you for the review :)

Copy link
Contributor

@MatKbauer MatKbauer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Code tested successfully, default ./config/evaluate/eval_config.yml is selected when no argument is provided. Passing a particular config file still works with --config path/to/eval_config.yml

@MatKbauer MatKbauer merged commit 2f669cf into ecmwf:develop Oct 8, 2025
5 checks passed
@github-project-automation github-project-automation bot moved this from In Progress to Done in WeatherGen-dev Oct 8, 2025
@MatKbauer MatKbauer mentioned this pull request Oct 13, 2025
6 tasks
sophie-xhonneux pushed a commit that referenced this pull request Oct 17, 2025
* adding default config + changing yml path locations

* linter checks

* linter checks

* revert .yml file

* updates

---------

Co-authored-by: iluise <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Default path for evaluation config file

3 participants