Skip to content

Commit e625bc8

Browse files
authored
Update Spack slides, demo notes and exercise (#245)
* Update Spack slides and demo notes * Add timetable entry for Spack lecture and exercise * Update duration in README and slim down Spack demo notes * Formatting * Minor changes to the exercise
1 parent 4002d77 commit e625bc8

File tree

5 files changed

+181
-213
lines changed

5 files changed

+181
-213
lines changed

03_building_and_packaging/README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,5 +22,5 @@ Learning goals:
2222
| 30 minutes | [pip_demo.md](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/03_building_and_packaging/pip_demo.md) |
2323
| 90 minutes | [pypi_exercise.md](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/03_building_and_packaging/pypi_exercise.md) |
2424
| 45 minutes | [pypi_slides.md](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/03_building_and_packaging/pypi_slides.md) |
25-
| 45 minutes | [spack_slides.md](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/03_building_and_packaging/spack_slides.md), [spack_demo.md](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/03_building_and_packaging/spack_demo.md) |
26-
| 45 minutes | [spack_exercise.md](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/03_building_and_packaging/spack_exercise.md) |
25+
| 90 minutes | [spack_slides.md](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/03_building_and_packaging/spack_slides.md), [spack_demo.md](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/03_building_and_packaging/spack_demo.md) |
26+
| 90 minutes | [spack_exercise.md](https://github.com/Simulation-Software-Engineering/Lecture-Material/blob/main/03_building_and_packaging/spack_exercise.md) |

03_building_and_packaging/spack_demo.md

Lines changed: 37 additions & 53 deletions
Original file line numberDiff line numberDiff line change
@@ -1,28 +1,24 @@
1-
# Packaging for High-Performance Computing (Notes)
1+
# Packaging for High-Performance Computing Demo Notes
22

3-
**Note**: It is recommended to try out Spack in a fresh Docker container. To understand how Spack itself is installed, follow Step 1 in a fresh Ubuntu container. To make things simpler from Step 2 onwards, create a container from the [spack/ubuntu-jammy](https://hub.docker.com/r/spack/ubuntu-jammy) image, so that Spack is preinstalled.
3+
**Note**: It is recommended to try out Spack in a fresh Docker container. To understand how Spack itself is installed, follow Step 1 in a fresh Ubuntu Noble container (`docker run --rm -it ubuntu:noble`). To make things simpler from Step 2 onwards, create a container from the [spack/ubuntu-noble](https://hub.docker.com/r/spack/ubuntu-noble) image, so that Spack is preinstalled.
44

55
## 1. Spack Setup/Installation
66

7-
- Git repository of Python scripts
7+
- This demo (and Spack) needs Python, Git, a C/C++ compiler, patch, make, tar, and a few more tools. Basically `build-essential`, `git`, and `python` on Ubuntu Noble.
8+
9+
- Get the Spack repository
810

911
```bash
10-
git clone -b v0.23.0 -c feature.manyFiles=true --depth=2 https://github.com/spack/spack.git
12+
git clone -b v1.1.0 --depth=2 https://github.com/spack/spack.git
1113
```
1214

13-
- `v0.23.0` is currently the latest major release
14-
15-
- **Note:** Install `curl`, `libcurl4-openssl-dev`, and `vim` to ensure that Spack v0.23.0 works in a fresh Ubuntu Jammy container.
16-
1715
- Initializing Spack with
1816

1917
```bash
2018
. <spack_prefix>/share/spack/setup-env.sh
2119
```
2220

23-
will set `SPACK_ROOT` and also add `spack` to `PATH`.
24-
25-
Note that the `.` operator will run the commands in the supplied script as if we would type the commands supplied by the script in the shell ourselves. In bash the `.` operator is equivalent to `source`. However, `source` is not specified in POSIX and thus using `.` is likely to work on more platforms.
21+
will set `SPACK_ROOT` and also add `spack` to `PATH`. Note that the `.` operator will run the commands in the supplied script as if we would type the commands supplied by the script in the shell ourselves. In bash the `.` operator is equivalent to `source`. However, `source` is not specified in POSIX and thus using `.` is likely to work on more platforms.
2622

2723
- Finish Spack setup
2824

@@ -40,25 +36,23 @@
4036

4137
prints the list of compilers that Spack has added.
4238

43-
- Find external packages (optional)
39+
- Find external packages
4440

4541
```bash
4642
spack external find
4743
```
4844

4945
This command tries to find preinstalled software packages which are also in Spack's package database. This way we do not have to recompile everything from scratch.
5046

51-
**Note** `spack external find` is an experimental feature and might fail. System packages [can be defined manually](https://spack.readthedocs.io/en/latest/getting_started.html#system-packages).
52-
53-
- Found packages (including version, configuration etc.) are stored in `~/.spack/packages.yaml`. There one can add further packages manually. **Note**: Maybe show content of this file.
47+
Found packages (including version, configuration etc.) are stored in `~/.spack/packages.yaml`. There one can add further packages manually. Look at contents of this file.
5448

55-
- Concretize a spec to trigger bootstrap process (optional)
49+
- Concretize a spec to trigger bootstrap process
5650

5751
```bash
5852
spack spec zlib
5953
```
6054

61-
This will try to concretize the package and its dependencies. `clingo`, Spack's concretizer, is needed for this. If one calls this command the very first time, a bootstrapping process is triggered that installs `clingo`.
55+
This will try to concretize the package and its dependencies. `clingo`, Spack's concretizer, is needed for this. Running `spack spec` for the first time triggers a bootstrapping process that installs `clingo`.
6256

6357
## 2. Package Installation/Management with Spack
6458

@@ -68,15 +62,7 @@
6862
spack info zlib
6963
```
7064

71-
- Install a simple example package
72-
73-
```bash
74-
spack spec zlib
75-
```
76-
77-
This will trigger the bootstrap process of Spack. It installs `clingo`, the dependency resolver, if not done before.
78-
79-
- Install `zlib`
65+
- Install the package
8066

8167
```bash
8268
spack install --reuse zlib
@@ -102,7 +88,7 @@
10288

10389
Spack will fetch the archive and try to deduce some basic settings. It will also check for other releases and asks whether to add checksums. We want all checksums.
10490

105-
- Afterwards Spack drops us in the boilerplate package.
91+
- Spack immediately drops you in a boilerplate package file.
10692

10793
```Python
10894
class Helloworld(CMakePackage):
@@ -133,16 +119,16 @@
133119

134120
At least the `FIXME` statements should be fixed before releasing the package. The Spack package recipe is written in Python so we can also do common Python magic here.
135121

136-
- Spack has deduced a good number of information already.
122+
- Spack has deduced a good amount of information already.
137123
- We work with CMake
138124
- There are 3 releases of the software
139-
- The URL to download packages
140-
- The name of the Package we work with `class Helloworld`. This is also the name of the software now within Spack.
125+
- The URL to download the package
126+
- The name of the package we work with `class Helloworld`. This is also the name of the software now within Spack.
141127
- We want to fix/extend the package with some standard information
142128
- Package description
143129
- Set URL to SSE homepage
144-
- Add our GitHub username as maintainer
145-
- Remove the `cmake_args` part as we only have a standard CMake arguments. Here we could give extra/special arguments specific to the software package.
130+
- Add GitHub username as maintainer
131+
- Remove the `cmake_args` part as there are only standard CMake arguments.
146132

147133
- Concretize the package
148134

@@ -158,7 +144,7 @@
158144
^[email protected]%[email protected]~doc+ncurses+openssl+ownlibs~qt build_type=Release patches=1c540040c7e203dd8e27aa20345ecb07fe06570d56410a24a266ae570b1c4c39,bf695e3febb222da2ed94b3beea600650e4318975da90e4a71d6f31a6d5d8c3d arch=linux-ubuntu20.04-skylake
159145
```
160146

161-
We see that `cmake` is an implicit dependency as we need it for building our package.
147+
`cmake` is an implicit dependency as we need it for building our package.
162148

163149
- If the same Docker container as in step 2 is used, make sure to uninstall `zlib` before installing `helloworld`.
164150

@@ -180,15 +166,15 @@
180166
spack find
181167
```
182168

183-
- Load installed package, run code and unload
169+
- Load installed package, run, and unload
184170

185171
```bash
186172
spack load helloworld
187173
helloworld
188174
spack unload helloworld
189175
```
190176

191-
- Install different version of code
177+
- Install different version
192178

193179
```bash
194180
spack install [email protected]
@@ -204,15 +190,15 @@
204190

205191
or open `package.py` file in `${HOME}/var/spack/repos/builtin/packages/helloworld/`
206192

207-
- **Optional**: one could add `main` branch and thus GitHub repository
193+
- Add the `main` branch as a version
208194

209195
```diff
210-
+ git = "https://github.com/Simulation-Software-Engineering/HelloWorld.git"
196+
+ git = "https://github.com/Simulation-Software-Engineering/HelloWorld.git"
211197
+
212198
+ version("main", branch="main")
213199
```
214200

215-
This can also be used for `develop` branches etc. It is useful if one needs really the newest version of a package or if one develops software using Spack.
201+
This can also be used for `develop` branches etc. It is useful if the development state is required, or if one develops software using Spack.
216202

217203
- Add artificial dependencies
218204

@@ -221,17 +207,17 @@
221207
+ depends_on("zlib@:1.2")
222208
```
223209

224-
This means that the package depends on Python `3.0.0` or newer and newer if we use `helloworld` of version `0.3.0` or newer. The software also requires at most `zlib` in version `1.2.10`
210+
This states that the package depends on Python version `3.0.0` or newer if we use `helloworld` of version `0.3.0` or newer. The software also requires at most `zlib` in version `1.2.10`
225211

226-
- Show new dependencies
212+
- Show new dependencies
227213

228-
```bash
229-
spack spec helloworld
230-
spack spec [email protected]
231-
spack info helloworld
232-
```
214+
```bash
215+
spack spec helloworld
216+
spack spec [email protected]
217+
spack info helloworld
218+
```
233219

234-
The Python dependency will only show up for the newest version of our software package.
220+
The Python dependency will only show up for version `0.3.0`.
235221

236222
- Add an artificial variant
237223

@@ -247,28 +233,26 @@
247233
spack info helloworld
248234
```
249235

250-
where Python now shows up as variant with its description. We can deactivate Python by specifying
236+
where Python now shows up as variant with its description. Python dependency is removed by doing
251237

252238
```bash
253239
spack info helloworld -python
254240
```
255241

256-
`~` can be (often) used instead of `-`. There are [examples in the documentation](https://spack.readthedocs.io/en/latest/basic_usage.html#variants).
257-
258242
## Further material
259243

260244
### References
261245

262-
- [EasyBuild](https://github.com/easybuilders/easybuild)
263-
- [EasyConfigs](https://github.com/easybuilders/easybuild-easyconfigs)
264246
- [Spack](https://spack.io/)
265247
- [Spack docs](https://spack.readthedocs.io/en/latest/)
266248
- [Spack 101 tutorials](https://spack-tutorial.readthedocs.io/en/latest/)
249+
- [EasyBuild](https://github.com/easybuilders/easybuild)
250+
- [EasyConfigs](https://github.com/easybuilders/easybuild-easyconfigs)
267251
- [archspec project](https://github.com/archspec/)
268252

269-
### Talks
253+
### Interesting Talks On This Topic
270254

271-
Talks at FOSDEM
255+
FOSDEM talks:
272256

273257
- 2020: [Spack's new Concretizer](https://archive.fosdem.org/2020/schedule/event/dependency_solving_not_just_sat/)
274258
- 2020: [Build for your microarchitecture: experiences with Spack and archspec](https://archive.fosdem.org/2020/schedule/event/archspec/)

03_building_and_packaging/spack_exercise.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
# Packages with Spack
22

3-
This exercise is about packaging code with Spack. We work with a simplified version of the code from the previous week's exercise and create packages for it.
3+
This exercise is about packaging code with Spack. We work with a simplified version of the code from the previous week's exercise and create a Spack package of it.
44

5-
At the end of the exercise you find a section with hints and remarks. Make sure to check this section.
5+
**Note**: At the end of the exercise you find a section with hints and remarks. Make sure to check this section.
66

7-
Deadline: **Wednesday, December 4th, 2024, 09:00**
7+
Deadline: **Wednesday, December 3rd, 2025, 09:00**
88

99
## Creation of a Spack Package
1010

@@ -20,7 +20,7 @@ The [code repository](https://github.com/Simulation-Software-Engineering/spack-e
2020

2121
**Note**: We require `yaml-cpp` in version `0.7.0`. This package will be automatically installed via Spack if you specify the dependency correctly in you package. You should **not** install `yaml-cpp` manually in this exercise.
2222

23-
Create a Spack package for all releases of the given code and make sure that the dependencies are specified appropriately. Make sure you also add yourself as maintainer to your package. Your final package should not contain any `FIXME` parts.
23+
Create a Spack package for all releases of the given code and make sure that the dependencies are specified appropriately. Make sure you also add yourself as maintainer to your package. Your packaging file should not contain any `FIXME` parts.
2424

2525
### Development/Packaging Environment
2626

0 commit comments

Comments
 (0)