Skip to content

Conversation

simone99n
Copy link

Description

Refactor of normalize_* and denormalize_* methods (normalize_source_channels, normalize_target_channels, denormalize_source_channels, denormalize_target_channels) in data_reader_base.py to avoid code duplication.
Introduced helper functions (_normalize and _denormalize) that encapsulate the shared logic. Each method would then call the helper with the appropriate indices, mean, and standard deviation arrays.
Replaced the assert statement with an "if + ValueError"

Issue Number

Closes #956

Checklist before asking for review

  • I have performed a self-review of my code
  • My changes comply with basic sanity checks:
    • I have fixed formatting issues with ./scripts/actions.sh lint
    • I have run unit tests with ./scripts/actions.sh unit-test
    • I have documented my code and I have updated the docstrings.
    • I have added unit tests, if relevant
  • I have tried my changes with data and code:
    • I have run the integration tests with ./scripts/actions.sh integration-test
    • (bigger changes) I have run a full training and I have written in the comment the run_id(s): launch-slurm.py --time 60
    • (bigger changes and experiments) I have shared a hegdedoc in the github issue with all the configurations and runs for this experiments
  • I have informed and aligned with people impacted by my change:
    • for config changes: the MatterMost channels and/or a design doc
    • for changes of dependencies: the MatterMost software development channel

@simone99n
Copy link
Author

simone99n commented Sep 30, 2025

I did not run the integration test because of the following error:

09:36:34 norberti1@jwlogin23 $ jutil env activate -p weatherai
09:36:48 norberti1@jwlogin23 $ ./scripts/actions.sh integration-test
srun: error: job_submit_filter: please specify the job's account with -A or --account option
srun: error: Hint: to find available budget accounts from each project please run: jutil user projects
srun: error: Unable to allocate resources: Invalid account or account/partition combination specified

I'm probably missing something.

However, I did run a training using launch-slurm.py --time 60.
run_id: qmef4b7w

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

Refactor normalization/denormalization methods in data_reader_base.py to reduce code duplication
1 participant