You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It is common to save the state of a simulation at regular intervals, especially for long-running simulations. This allows you to restart the simulation from the last saved state in case of interruptions or to continue the simulation at a later time without losing progress. JustRelax provides a simple way to save and load checkpoint files. Two checkpointing functions are available for the most common file extensions (HDF5 and JLD2). By loading the `DataIO` module, you gain access to these checkpointing functions as well as VTK saving functions for later visualization with [ParaView](https://www.paraview.org/).
4
+
For more details on the vtk, see the [here](./subduction2D/subduction2D.md).
5
+
6
+
!!! tip "JustPIC checkpointing" A similar checkpointing function is defined by [JustPIC.jl](https://juliageodynamics.github.io/JustPIC.jl/dev/IO/) to save the state of the particles.
7
+
8
+
9
+
:::code-group
10
+
11
+
```julia [2D module]
12
+
using JustRelax, JustRelax.JustRelax2D
13
+
using JustRelax.DataIO
14
+
```
15
+
16
+
```julia [3D module]
17
+
using JustRelax, JustRelax.JustRelax3D
18
+
using JustRelax.DataIO
19
+
```
20
+
:::
21
+
22
+
Unless you have a specific reason to use hdf5 files, we recommend using JLD2 files for checkpointing. JLD2 files are generally faster to read and write, and retain the original data types of the variables. However, we made sure to provide both options for maximum flexibility.
23
+
24
+
### Saving and loading checkpoint with HDF5
25
+
The HDF5 checkpointing function saves the most important model variables (pressure, temperature, velocity components, viscosity, time, and timestep) to a `checkpoint.h5` file in your destination folder.
26
+
27
+
```julia
28
+
dst ="Your_checkpointing_directory"
29
+
checkpointing_hdf5(dst, stokes, thermal.T, time, timestep)
30
+
```
31
+
32
+
To load the checkpoint, use `load_checkpoint_hdf5`. This function returns the variables in the same order as saved:
JLD2 checkpointing is recommended for most users due to its speed and ability to preserve Julia data types. In contrast to the HDF5 function, the JLD2 checkpointing function saves all stokes and thermal arrays (optional) while being MPI agnostic. This means that if you run your model with multiple processors, each processor will save its own checkpoint file in the specified directory with MPI rank attached to the name (e.g. `checkpoint0000.jld2`, `checkpoint0001.jld2`). The function automatically handles the naming of these files to avoid overwriting. Additionally, you can save any custom fields by passing them as keyword arguments.
41
+
42
+
!!! warning "Checkpointing" All checkpointing functions will save the arrays as CPU arrays no matter your backend. This means that if you are using a GPU backend, the arrays will be transferred to the CPU before saving, which may take some time depending on the size of your model.
43
+
44
+
:::code-group
45
+
46
+
```julia [Normal use]
47
+
dst ="Your_checkpointing_directory"
48
+
checkpointing_jld2(dst, stokes, thermal, time, dt)
49
+
```
50
+
51
+
```julia [MPI]
52
+
dst ="Your_checkpointing_directory"
53
+
checkpointing_jld2(dst, stokes, thermal, time, dt, igg)
To load the checkpoint, you can use the preexisting `load_checkpoint_jld2` function or use the `JLD2` loading function directly. The `load_checkpoint_jld2` function is MPI agnostic and will automatically load the correct file for each processor based on its rank:
If you save additional fields, it is the easiest to load the checkpointing file directly using the `JLD2` package. This way, you can access all saved variables by their names:
78
+
79
+
```julia
80
+
using JLD2
81
+
fname =joinpath(dst, "checkpoint000$(igg.me).jld2") # Adjust filename for MPI if needed
82
+
data = JLD2.load(fname)
83
+
```
84
+
which then returns a dictionary with all your saved variables. You can access them like this:
To restart a simulation from a previously saved checkpoint file, you can make use of the checkpointing functions described in the [Checkpointing documentation](./checkpointing.md). Depending on the file format you used to save your checkpoint (HDF5 or JLD2), you can load the saved state of your simulation using the corresponding loading function.
4
+
5
+
In this example, we will demonstrate how to set up a script to restart a simulation from a JLD2 checkpoint file as we can save the entire structures of the `StokesArrays` and `ThermalArrays` which makes it easier to restart the simulation. We will assume that you have already saved a checkpoint file using the `checkpointing_jld2` function for the example of a 2D subduction model.
6
+
Ideally, one does not need to change much in the initial script used to start the simulation from scratch. The main difference is that instead of initializing the `StokesArrays` and `ThermalArrays` from scratch, we will load them from the checkpoint file.
7
+
For a detailed description of the 2D subduction model setup, please refer to the [2D subduction documentation](./subduction2D/subduction2D.md). The following example can be found [here](https://github.com/PTsolvers/JustRelax.jl/blob/d63ca8f08860859700913b575c9befc33d5c4f2a/miniapps/subduction/2D/Subduction2D_restart).
8
+
9
+
Load JustRelax necessary modules and define backend.
10
+
```julia
11
+
using CUDA # comment this out if you are not using CUDA; or load AMDGPU.jl if you are using an AMD GPU
12
+
using JustRelax, JustRelax.JustRelax2D, JustRelax.DataIO
For this benchmark we will use particles to track the advection of the material phases and their information. For this, we will use [JustPIC.jl](https://github.com/JuliaGeodynamics/JustPIC.jl)
!!! tip "Script" Leave most of your original script unchanged and only change the parts we highlight in this example, unless you want to explicitly change some model parameters (e.g., rheology, boundary conditions, etc.). Make sure you dont accidentally overwrite your loaded arrays/particles with new initializations.
23
+
24
+
## Load and initialize particles fields
25
+
The `JustPIC` specific function `TA()` will convert the loaded particles to the correct backend.
26
+
```julia
27
+
data =load(joinpath("Your_checkpointing_directory", "particles.jld2"))
The loaded arrays are CPU arrays, so we need to convert them to the correct backend.
55
+
```julia
56
+
stokes =PTArray(backend_JR, stokes_cpu)
57
+
thermal =PTArray(backend_JR, thermal_cpu)
58
+
```
59
+
From here on you should be able to continue the simulation as usual. Make sure to adjust the time loop to start from the loaded time `t` and iteration `it` if you loaded them.
0 commit comments