You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+81-50Lines changed: 81 additions & 50 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,60 +1,84 @@
1
-
# JUHPC: a community software development project for everyone - including end users
2
-
JUHPC is an attempt to convert the numerous efforts at different HPC sites for defining a suitable Julia HPC setup into a community software development project... *which suits everyone*. The objective is to gather all the experience of the Julia HPC community and transform it in an portable automatic Julia HPC setup, enhanced and maintained jointly by the Julia HPC community. JUHPC can be used stand-alone (by end users) or as part of a recipe for automated software stack generation (by HPC sites) as, e.g., the generation of modules or [uenvs](https://eth-cscs.github.io/uenv/) (used on the ALPS supercomputer at the Swiss National Supercomputing Centre, see [here](https://confluence.cscs.ch/display/KB/UENV+user+environments)).
1
+
# JUHPC <!-- omit from toc -->
3
2
4
-
An important lesson learned by the Julia HPC community for providing Julia at HPC sites is not to preinstall any packages site wide. JUHPC pushes this insight even one step further and does not preinstall Julia either. Instead, juliaup is leveraged and the installation of juliaup, Julia and packages is preconfigured for being automatically executed by the end user.
3
+
## Introduction: a community project for everyone - including end users
5
4
6
-
Concretely, JUHPC creates an HPC setup for juliaup, Julia and some HPC key packages (MPI.jl, CUDA.jl, HDF5.jl, ADIOS2.jl, ...), including
5
+
JUHPC is an attempt to convert the numerous efforts at different HPC sites for defining a suitable Julia HPC setup into a community scripting project... *which suits everyone*. The objective is to gather all the experience of the Julia HPC community and transform it in an portable automatic Julia HPC setup, enhanced and maintained jointly by the Julia HPC community. JUHPC can be used stand-alone (by end users) or as part of a recipe for automated software stack generation (by HPC sites) as, e.g., the generation of modules or [uenvs](https://eth-cscs.github.io/uenv/) (used on the ALPS supercomputer at the Swiss National Supercomputing Centre, see [here](https://confluence.cscs.ch/display/KB/UENV+user+environments)).
6
+
7
+
An important lesson learned by the Julia HPC community for providing Julia at HPC sites is not to preinstall any packages site wide. JUHPC pushes this insight even one step further and does not preinstall Julia either. Instead, Juliaup is leveraged and the installation of Juliaup, Julia and packages is preconfigured for being automatically executed by the end user. Furthermore, for maximal robustness, the preferences are created using the available API calls of the corresponding packages.
8
+
9
+
Concretely, JUHPC creates an HPC setup for Juliaup, Julia and some HPC key packages (MPI.jl, CUDA.jl, HDF5.jl, ADIOS2.jl, ...), including
7
10
- preferences for HPC key packages that require system libraries;
8
-
- a wrapper for juliaup that will install juliaup (and latest Julia) automatically in a predefined location (e.g., scratch) when the end user calls `juliaup` the first time;
9
-
- an activation script that sets environment variables for juliaup, Julia and HPC key packages;
11
+
- a wrapper for Juliaup that will install Juliaup (and latest Julia) automatically in a predefined location (e.g., scratch) when the end user calls `juliaup` the first time;
12
+
- an activation script that sets environment variables for Juliaup, Julia and HPC key packages;
10
13
- optional execution of a site-specific post installation Julia script, using the project where preferences were set (e.g, to modify preferences or to create an uenv view equivalent to the activation script).
11
14
12
15
HPC sites can install the HPC setup into a folder in a location accessible to all users (which can also be part, e.g., of a uenv). HPC end users can install the HPC setup into any folder to their liking, accessible from the compute nodes; it is then enough to source the activate script in this folder in order to activate the HPC setup.
13
16
17
+
## Table of contents <!-- omit from toc -->
18
+
-[Introduction: a community project for everyone - including end users](#introduction-a-community-project-for-everyone---including-end-users)
19
+
-[Usage](#usage)
20
+
-[1. Export environment variables for the installation of some HPC key packages](#1-export-environment-variables-for-the-installation-of-some-hpc-key-packages)
21
+
-[2. Call JUHPC](#2-call-juhpc)
22
+
-[Examples: HPC setup installations on the ALPS supercomputer (CSCS)](#examples-hpc-setup-installations-on-the-alps-supercomputer-cscs)
23
+
-[Example 1: using Cray Programming Environment](#example-1-using-cray-programming-environment)
24
+
-[Example 2: using UENV](#example-2-using-uenv)
25
+
-[Test of example 1](#test-of-example-1)
26
+
-[Test of example 2](#test-of-example-2)
27
+
-[Your contributions](#your-contributions)
28
+
-[Contributors](#contributors)
29
+
14
30
15
-
# It's as simple as that
31
+
## Usage
32
+
Installing a HPC setup for Juliaup, Julia and some HPC key packages, requires only two steps:
33
+
1. Export environment variables for the installation of some HPC key packages.
34
+
2. Call JUHPC.
16
35
17
-
# 1. Export environment variables for the installation of some HPC key packages
36
+
Details are given in the following two subsections.
18
37
19
-
## CUDA
20
-
-`JUHPC_CUDA_HOME`: Activates HPC setup for CUDA and is used for CUDA.jl runtime discovery (set as CUDA_HOME in the activate script).
21
-
-`JUHPC_CUDA_RUNTIME_VERSION`: Used to set CUDA.jl preferences (fixes runtime version enabling pre-compilation on login nodes).
38
+
### 1. Export environment variables for the installation of some HPC key packages
22
39
23
-
## AMDGPU
24
-
-`JUHPC_ROCM_HOME`: Activates HPC setup for AMDGPU and is used for AMDGPU.jl runtime discovery (set as ROCM_PATH in the activate script).
40
+
- CUDA
41
+
-`JUHPC_CUDA_HOME`: Activates HPC setup for CUDA and is used for CUDA.jl runtime discovery (set as CUDA_HOME in the activate script).
42
+
-`JUHPC_CUDA_RUNTIME_VERSION`: Used to set CUDA.jl preferences (fixes runtime version enabling pre-compilation on login nodes).
25
43
26
-
## MPI
27
-
-`JUHPC_MPI_HOME`: Activates HPC setup for MPI and is used to set MPI.jl preferences. Incompatible with `JUHPC_MPI_VENDOR`
28
-
-`JUHPC_MPI_VENDOR`: Activates HPC setup for MPI and is used to set MPI.jl preferences (currently only "cray" is valid, see [here](https://juliaparallel.org/MPI.jl/stable/configuration/#Notes-about-vendor-provided-MPI-backends)). Incompatible with `JUHPC_MPI_HOME`.
29
-
-`JUHPC_MPI_EXEC`: Used to set MPI.jl preferences (exec command definition). Arguments are space separated, e.g., "srun -C gpu".
44
+
- AMDGPU
45
+
-`JUHPC_ROCM_HOME`: Activates HPC setup for AMDGPU and is used for AMDGPU.jl runtime discovery (set as ROCM_PATH in the activate script).
30
46
31
-
## HDF5
32
-
-`JUHPC_HDF5_HOME`: Activates HPC setup for HDF5 and is used to set HDF5.jl preferences.
47
+
- MPI
48
+
-`JUHPC_MPI_HOME`: Activates HPC setup for MPI and is used to set MPI.jl preferences. Incompatible with `JUHPC_MPI_VENDOR`
49
+
-`JUHPC_MPI_VENDOR`: Activates HPC setup for MPI and is used to set MPI.jl preferences (currently only "cray" is valid, see [here](https://juliaparallel.org/MPI.jl/stable/configuration/#Notes-about-vendor-provided-MPI-backends)). Incompatible with `JUHPC_MPI_HOME`.
50
+
-`JUHPC_MPI_EXEC`: Used to set MPI.jl preferences (exec command definition). Arguments are space separated, e.g., "srun -C gpu".
33
51
34
-
## ADIOS2
35
-
-`JUHPC_ADIOS2_HOME`: Activates HPC setup for ADIOS2 and is used to set ADIOS2.jl preferences.
52
+
- HDF5
53
+
-`JUHPC_HDF5_HOME`: Activates HPC setup for HDF5 and is used to set HDF5.jl preferences.
36
54
55
+
- ADIOS2
56
+
-`JUHPC_ADIOS2_HOME`: Activates HPC setup for ADIOS2 and is used to set ADIOS2.jl preferences.
-`JUHPC_SETUP_INSTALLDIR`: the folder to install the HPC setup into, e.g., `"$SCRATCH/../julia/${HOSTNAME%%-*}/juhpc_setup"` (assuming `$SCRATCH/../julia` is a wipe out protected folder on scratch).
46
-
-`JULIAUP_INSTALLDIR`: the folder juliaup and Julia will automatically be installed the first time the end user calls `juliaup`. *User environment variables should be escaped* in order not to have them expanded during HPC setup installation, but during its usage by the end user, e.g., `"\$SCRATCH/../julia/\$USER/\${HOSTNAME%%-*}/juliaup"` (assuming `$SCRATCH/../julia` is a wipe out protected folder on scratch).
47
-
-`JUHPC_POST_INSTALL` (optional): site-specific post installation Julia script, using the project where preferences were set (e.g, to modify preferences or to create an uenv view equivalent to the activation script).
66
+
-`JUHPC_SETUP_INSTALLDIR`: the folder into which the HPC setup is installed, e.g., `"$SCRATCH/../julia/${HOSTNAME%%-*}/juhpc_setup"` (assuming `$SCRATCH/../julia` is a wipe out protected folder on scratch).
67
+
-`JULIAUP_INSTALLDIR`: the folder into which Juliaup and Julia will automatically be installed the first time the end user calls `juliaup`. *User environment variables should be escaped* in order not to have them expanded during HPC setup installation, but during its usage by the end user, e.g., `"\$SCRATCH/../julia/\$USER/\${HOSTNAME%%-*}/juliaup"` (assuming `$SCRATCH/../julia` is a wipe out protected folder on scratch).
68
+
-`JUHPC_POST_INSTALL_JL` (optional): site-specific post installation Julia script, using the project where preferences were set (e.g, to modify preferences or to create an uenv view equivalent to the activation script).
69
+
70
+
71
+
> ℹ️ The above examples assume that `$SCRATCH/../julia` is a wipe out protected folder on scratch.
48
72
49
-
> :note: The above examples assume that `$SCRATCH/../julia` is a wipe out protected folder on scratch.
50
-
> :note: Separate installation by HOSTNAME is required if different hosts with different architectures share file system used for installation (e.g., daint and eiger on ALPS).
73
+
> ⚠️ Separate installation by HOSTNAME is required if different hosts with different architectures share file system used for installation (e.g., daint and eiger on ALPS).
51
74
52
75
53
-
# Examples of HPC setup installations on the ALPS supercomputer (CSCS)
76
+
##Examples: HPC setup installations on the ALPS supercomputer (CSCS)
54
77
55
78
Examples of HPC setup installations are found in the folder `configs` of which two are featured in the following.
56
79
57
-
## Example 1: using Cray Programming Environment
80
+
### Example 1: using Cray Programming Environment
81
+
58
82
```bash
59
83
# Load required modules (including correct CPU and GPU target modules)
Any contribution is valuable to the Julia HPC community:
208
+
- contribute the HPC setup config file for your cluster or supercomputer in `configs` as an example for others;
209
+
- open a PR with some enhancement or fix;
210
+
- open an issue for a bug or an idea for enhancement.
211
+
182
212
183
-
The initial version of JUHPC was contributed by Samuel Omlin, Swiss National Supercomputing Centre, ETH Zurich (@omlins). The following people have provided valuable contributions over the years in the effort of defining a suitable Julia HPC setup (this is based on the list found [here](https://github.com/hlrs-tasc/julia-on-hpc-systems); please add missing people or let us know):
213
+
## Contributors
184
214
215
+
The initial version of JUHPC was contributed by Samuel Omlin, Swiss National Supercomputing Centre, ETH Zurich (@omlins). The following people have provided valuable contributions over the years in the effort of defining a suitable Julia HPC setup (this is based on the list found [here](https://github.com/hlrs-tasc/julia-on-hpc-systems); please add missing people or let us know; the list is ordered alphabetically):
0 commit comments