Skip to content

Creating a standalone

Jump to bottom
vectorflux edited this page Jun 1, 2018 · 13 revisions

This short tutorial describes the main steps required to create (or update) a standalone from ICON using the scripts provided in eniac-scripts/standalones.

For this tutorial, a standalone for the nsurf_diag routine in atm_phy_echam/mo_surface_diag.f90 will be created.

Preparing the repositories

  1. Clone the icon-eniac.git repository
#$> git clone --recursive [email protected]:C2SM-RCM/icon-eniac.git
  1. Clone the eniac-scripts.git repository inside icon-eniac
#$> cd icon-eniac/
#$> git clone [email protected]:C2SM-RCM/eniac-scripts.git

Creating the scripts

  1. Change into eniac-scripts/standalones 
#$> cd eniac-scripts/standalones/
  1. Create a new directory under standalones and initialize it with the scripts from the template. The name of the subroutine used to create the standalone has to be used. A simple script is provided to generate the main scripts based on a template.
#$> ./common/init_new_standalone.sh nsurf_diag mo_surface_diag atm_phy_echam nsurf_diag
  1. If the standalone does not require the dynamical code, disable existing OpenACC statements.
#$> ln -rs common/noacc_list.txt nsurf_diag/
  1. Create a list of external dependencies, one path per line, in a file named external_dependencies.txt 
#$> echo "externals/mtime" >> nsurf_diag/external_dependencies.txt

Extracting the standalone

  1. Move back into the root of icon-eniac.git 
#$> cd ../..
  1. If you're creating a new standalone or you need to update intermediate patches, use the interactive mode. The following steps assume this is the case.
#$> export ENIAC_INTERACTIVE=1
  1. Start the extraction script providing the machine you're running on (Kesch or Daint).
#$> ./eniac-scripts/standalones/nsurf_diag/extract_standalone.sh kesch

  1. Step 1: clones the FTG tools and configures them to use a modified icon_standalone template.

  2. Step 2: applies all the patches found in nsurf_diag/patches/extract/init

  3. Step 3: configure and build icon and generate the capture code with FTG.

  4. If Step 2 or 3 was successful continue with next step. Otherwise, fix the compilation errors, generate the new patches and update the eniac-scripts.git repository.

#$> (fix errors)
#$> ./eniac-scripts/standalones/nsurf_diag/generate_patches.sh kesch extract init 
#$> (replace patches in eniac-scripts/standalones/nsurf_diag/patches/extract/init with new version from eniac_patches/extract/init)
#$> /eniac-scripts/standalones/nsurf_diag/extract_standalone.sh kesch # skip steps until Step 3

  1. Step 4: applies all the patches found in nsurf_diag/patches/extract/intermezzo

  2. Step 5: configure and build ICON and generate the replay code with FTG.

  3. If Step 4 or 5 was successful continue with next step. Otherwise, fix the compilation errors, generate the new patches and update the eniac-scripts.git repository.

#$> (fix errors)
#$> ./eniac-scripts/standalones/nsurf_diag/generate_patches.sh kesch extract intermezzo 
#$> (replace patches in eniac-scripts/standalones/nsurf_diag/patches/extract/intermezzo with new version from eniac_patches/extract/intermezzo)
#$> /eniac-scripts/standalones/nsurf_diag/extract_standalone.sh kesch # skip steps until Step 5

  1. Step 6: applies all the patches found in nsurf_diag/patches/extract/finish

  2. If Step 6 was successful continue with next step. Otherwise, fix the compilation errors, generate the new patches and update the eniac-scripts.git repository.

#$> (fix errors)
#$> ./eniac-scripts/standalones/nsurf_diag/generate_patches.sh kesch extract intermezzo 
#$> (replace patches in eniac-scripts/standalones/nsurf_diag/patches/extract/intermezzo with new version from eniac_patches/extract/intermezzo)
#$> /eniac-scripts/standalones/nsurf_diag/extract_standalone.sh kesch # skip steps until Step 5

  1. Step 7: extracts a minimal standalone and copies required files into standalone/nsurf_diag.

  2. Step 8: reset icon-eniac.git repository to starting state. Newly created standalone/nsurf_diag and if produced eniac_patches directories are left in repository.

  3. Step 9: generate patches required to create reference data and run the standalone validation. The patches are stored in standalone/nsurf_diag/eniac_patches.

  4. Replace the patches in eniac-scripts/standalones/nsurf_diag/patches/run/ with the new versions in standalone/nsurf_diag/eniac_patches

  5. Copy the FTG standalone executable in standalone/nsurf_diag/src/tests/ftg_nsurf_diag_test.f90 to eniac-scripts/standalones/nsurf_diag/src/tests.

  6. Generally the patches required for running the validation test and generating the serialization data are the same. All the patches in eniac-scripts/standalones/nsurf_diag/patches/reference will be applied.

#$> cd eniac-scripts/standalones/nsurf_diag
#$> ln -sr patches/run patches/reference
#$> cd ../../../
  1. Commit the changes to the eniac-scripts.git repository and push to GitHub.

Creating reference serialization data

  1. SLURM submission credentials and partition can be overwritten by exporting SBATCH_ACCOUNT and SBATCH_PARTITION`
#$> export SBATCH_ACCOUNT=g110
#$> export SBATCH_PARTITION=debug
  1. Run the create_reference.sh script to generate reference data for different values of nproma. The values used to generate the data is read from the nproma_list.txt file (default setup runs nproma = 16, 2729, 20480). The following command used GCC as a compiler and runs the CPU version on Kesch.
#$> ./eniac-scripts/standalones/nsurf_diag/create_reference.sh kesch gcc cpu

  1. The new serialization data is stored in experiments/atm_amip_test/ftg. Current version of the serialization data is stored under /project/c14/data-eniac/standalones/nsurf_diag.

  2. The directory /project/c14/data-eniac/standalones/nsurf_diag/experiments/atm_amip_test contains other files required to run the standalone (generally an empty directory).

Running standalone validation test

  1. SLURM submission credentials and partition can be overwritten by exporting SBATCH_ACCOUNT and SBATCH_PARTITION`
#$> export SBATCH_ACCOUNT=g110
#$> export SBATCH_PARTITION=debug
  1. Run the run_standalone_test.sh script in order to perform a validation run of the standalone against known reference data.
#$> ./eniac-scripts/standalones/nsurf_diag/run_standalone_test.sh kesch gcc cpu /project/c14/data-eniac/standalones/nsurf_diag
  1. The results from Serialbox2's compare.py are summarized on standard output. The full validation output is stored in the experiments/atm_amip_test/validation_nproma_###.result files. The output data from the standalone is stored in experiments/atm_amip_test/ftg and the input data and output reference data are copied (rsync'd) to ${SCRATCH}/data/icon-eniac/sa_nsurf_diag/ftg/gcc/kesch/nproma_###.

Clone this wiki locally