C2SM · AnnikaLau · Jul 16, 2025 · Jul 17, 2025 · Jul 17, 2025 · Jul 17, 2025
diff --git a/docs/best_practices/data_handling.md b/docs/best_practices/data_handling.md
@@ -9,7 +9,7 @@
    * [Forschungsdatenmanagement und Datenerhalt :material-open-in-new:](https://documentation.library.ethz.ch/display/DD/Forschungsdatenmanagement+und+Datenerhalt){:target="_blank"}
    * [ETH Guidelines for Research Integrity :material-open-in-new:](https://doi.org/10.3929/ethz-b-000179298){:target="_blank"}
    * [ETH Research Collection :material-open-in-new:](https://www.research-collection.ethz.ch){:target="_blank"}
-   * [Add your ORCID ID to your ETH account :material-open-in-new:](https://documentation.library.ethz.ch/display/RC/Assign+ORCID+iD){:target="_blank"}
+   * [Author profil and assign ORCID iD :material-open-in-new:](https://unlimited.ethz.ch/spaces/RC/pages/194119877/Author+profil+and+assign+ORCID+iD){:target="_blank"}
    * [ETH Guidelines for data management plans :material-open-in-new:](https://unlimited.ethz.ch/pages/viewpage.action?pageId=194127962){:target="_blank"}
 
 ## ETH Contacts
@@ -49,6 +49,6 @@
    * [IAC internal documentation on NetCDF in general :material-open-in-new:](https://wiki.iac.ethz.ch/IT/LinuxNetCDF){:target="_blank"} (!with IAC login only)
    * [Climate and Forecast (CF) conventions :material-open-in-new:](http://cfconventions.org){:target="_blank"}
    * [Python CF checker :material-open-in-new:](https://github.com/cedadev/cf-checker){:target="_blank"}
-   * [CMIP6 controlled vocabularies :material-open-in-new:](https://cmor.llnl.gov/mydoc_cmor3_CV/){:target="_blank"}
+   * [Controlled Vocabularies (CVs) for use in CMIP6 :material-open-in-new:](https://wcrp-cmip.github.io/CMIP6_CVs/){:target="_blank"}
 
 
diff --git a/docs/glossary/index.md b/docs/glossary/index.md
@@ -21,7 +21,7 @@
 | **ECMWF** | European Centre for Medium-range Weather Forecasts | [ECMWF website :material-open-in-new:](https://www.ecmwf.int/en/about){:target="_blank"} |
 | **ED** | Executive Director ||
 | **ERA5** | ECMWF ReAnalysis v5 | [ERA4 website :material-open-in-new:](https://www.ecmwf.int/en/forecasts/dataset/ecmwf-reanalysis-v5){:target="_blank"} |
-| **ESGF** | Earth System Grid Federation | [ESGF website :material-open-in-new:](https://esgf.llnl.gov){:target="_blank"} |
+| **ESGF** | Earth System Grid Federation | [ESGF website :material-open-in-new:](https://esgf.github.io/index.html){:target="_blank"} |
 | **EXCLAIM** | EXtreme scale Computing and data platform for cLoud-resolving weAther and clImate Modeling | [EXCLAIM website :material-open-in-new:](https://exclaim.ethz.ch){:target="_blank"} |
 | **EXTPAR** | EXTernal PARameters for numerical weather prediction and climate application | [EXTPAR documentation :material-open-in-new:](https://c2sm.github.io/extpar/){:target="_blank"} |
 | **FPS** | Flagship Pilot Studies ||

diff --git a/docs/models/icon/SUMMARY.md b/docs/models/icon/SUMMARY.md
@@ -1,4 +1,5 @@
 * [ICON](index.md)
 * [Compile and Run](compile_and_run.md)
-* [Large Use Cases](large_use_cases.md)
+* [Validate Custom Namelist](validate_custom_namelist.md)
+* [Run Probtest on Säntis](probtest.md)
 * [ICON-CLM](icon-clm.md)
diff --git a/docs/models/icon/compile_and_run.md b/docs/models/icon/compile_and_run.md
@@ -36,25 +36,57 @@ Clone the ICON repository:
 
 ### Säntis
 
-!!! info "Last update: 2025-05-22"
+!!! info "Last update: 2025-08-18"
 
     Säntis is regularly maintained by CSCS. In addition, the [uenvs](../../alps/uenvs.md) are updated irregularly. Therefore, some of the information provided here may be out of date. Please use the [C2SM support forum :material-open-in-new:](https://github.com/C2SM/Tasks-Support/discussions){:target="_blank"} in case of questions regarding building ICON on Säntis.
 
-Run the following after navigating into ICON root folder (replace `cpu` by `gpu` if applicable):
+Run the following after navigating into the ICON root folder (replace `gpu` by `cpu` if applicable):
 
 ```console
 UENV_VERSION=$(cat config/cscs/SANTIS_ENV_TAG)
-uenv run ${UENV_VERSION} -- ./config/cscs/santis.cpu.nvhpc
+uenv run ${UENV_VERSION} -- ./config/cscs/santis.gpu.nvhpc
 ```
 
 !!! Note
 
-    If you have never used a uenv on Säntis, you need to create a uenv repo first: `uenv repo create`.
+    If you have never used a uenv on Säntis, you need to create a uenv repo first:
+    ```
+    uenv repo create
+    ```
+
+    In case you are using the uenv version for the first time, you need to pull the image first:
+    ```
+    uenv image pull $UENV_VERSION
+    ```
+
 
-    In case you are using the uenv version for the first time, you need to pull the image first: `uenv image pull $UENV_VERSION`.
+#### Building out-of-source
 
-    For out-of-source builds navigate into the build folder and adapt the path to the configure wrapper above.
+Out-of-source builds are useful if you want to have two or more compiled versions of ICON in the same repository.
+To achieve that, you simply need to create separate folders in the ICON root folder 
+and run the configure wrapper from there.
 
+For example, if you want to compile ICON both for `cpu` and `gpu`, create those directories:
+
+```bash
+mkdir nvhpc_cpu
+mkdir nvhpc_gpu
+```
+
+Then, navigate into the corresponding folder and compile with:
+
+=== "`cpu`"
+    ```bash
+    UENV_VERSION=$(cat config/cscs/SANTIS_ENV_TAG)
+    cd nvhpc_cpu
+    uenv run ${UENV_VERSION} -- ./../config/cscs/santis.cpu.nvhpc 
+    ```
+=== "`gpu`"
+    ```bash
+    UENV_VERSION=$(cat config/cscs/SANTIS_ENV_TAG)
+    cd nvhpc_gpu
+    uenv run ${UENV_VERSION} -- ./../config/cscs/santis.gpu.nvhpc 
+    ```
 
 ### Euler
 

diff --git a/docs/models/icon/large_use_cases.md b/docs/models/icon/large_use_cases.md
diff --git a/docs/models/icon/probtest.md b/docs/models/icon/probtest.md
@@ -0,0 +1,119 @@
+# Run Probtest on Säntis
+
+Use Probtest to verify whether your test case produces consistent results on GPU. It compares a GPU test run to a CPU ensemble with perturbed input conditions.
+
+## 1. Compile ICON
+Compile ICON on CPU and on GPU as [out-of-source builds](compile_and_run.md#building-out-of-source). Note that the build directories need to be sub-directories of the ICON root folder. Otherwise the probtest container does not have access to the data.
+
+## 2. Set Up the Probtest Container and Environment on Säntis
+To run Probtest for ICON on Säntis, use the prebuilt container available on Docker Hub ([Probtest Container :material-open-in-new:](https://github.com/MeteoSwiss/probtest?tab=readme-ov-file#probtest-container){:target="_blank"}). ICON provides the wrapper script [`probtest_container_wrapper.py` :material-open-in-new:](https://gitlab.dkrz.de/icon/icon-nwp/-/blob/add_test_cases_santis/scripts/cscs_ci/probtest_container_wrapper.py?ref_type=heads){:target="_blank"}.
+
+!!! note
+    If your ICON version doesn’t include this script, add it to `scripts/cscs_ci/probtest_container_wrapper.py`, along with the appropriate [PROBTEST_TAG :material-open-in-new:](https://gitlab.dkrz.de/icon/icon/-/blob/main/run/tolerance/PROBTEST_TAG?ref_type=heads){:target="_blank"} under `run/tolerance/PROBTEST_TAG` and [yaml_experiment_test_processor.py :material-open-in-new:](https://gitlab.dkrz.de/icon/icon/-/blob/main/scripts/experiments/yaml_experiment_test_processor.py?ref_type=heads){:target="_blank"} under `scripts/experiments/yaml_experiment_test_processor.py` (replace if already available).
+
+
+### When Setting Up ICON from Scratch
+In your ICON root directory, import the container:
+
+```console
+PROBTEST_TAG=$(cat run/tolerance/PROBTEST_TAG)
+enroot import docker://c2sm/probtest:${PROBTEST_TAG}
+```
+
+Add a TOML configuration and export EDF path (being used when running the container):
+```console
+echo "image = \"$(pwd)/c2sm+probtest+${PROBTEST_TAG}.sqsh\"" > probtest.toml
+echo "mounts = [ \"$(pwd)\" ]" >> probtest.toml
+echo "workdir = \"$(pwd)\"" >> probtest.toml
+echo "writable = true" >> probtest.toml
+export EDF_PATH=$(pwd)
+```
+
+Create and activate Python environment:
+```console
+python3 -m venv .venv
+source .venv/bin/activate
+pip install pyyaml pandas click toml
+```
+
+### Every Time You Reconnect to the Server
+If the container and environment are already set up, simply re-run:
+```console
+export EDF_PATH=$(pwd)
+source .venv/bin/activate
+```
+
+Set experiment name, e.g.:
+```console
+export EXPERIMENT=c2sm_clm_r13b03_seaice
+```
+
+Export required environment variables:
+```console
+export BB_NAME=santis_cpu_nvhpc
+export UENV_VERSION=$(cat config/cscs/SANTIS_ENV_TAG)
+```
+
+## 3. Run perturbed ensemble on CPU
+Navigate to your CPU build directory and generate and run a 10-member ensemble (this may take time):
+```console
+cd nvhpc_cpu
+./make_runscripts $EXPERIMENT
+uenv run ${UENV_VERSION} -- python3 scripts/cscs_ci/probtest_container_wrapper.py ensemble $EXPERIMENT --build-dir $(pwd) --member-ids $(seq -s, 1 10)
+```
+
+This generates:
+
+- `stats_${EXPERIMENT}_<member_id>.csv`
+- `${EXPERIMENT}_reference.csv`
+
+## 4. Generate Tolerance from Ensemble
+
+Create reference and tolerance files using the 10 ensemble members:
+```console
+python3 scripts/cscs_ci/probtest_container_wrapper.py tolerance $EXPERIMENT --build-dir $(pwd) --member-ids $(seq -s, 1 10)
+```
+
+This generates:
+
+- `${EXPERIMENT}_tolerance.csv`
+
+## 5. Run the test case on GPU and collect statistics
+Navigate to your GPU build folder and run the same test case, e.g.:
+```console
+cd ../nvhpc_gpu
+./make_runscripts $EXPERIMENT
+cd run && sbatch --uenv ${UENV_VERSION} ./exp.c2sm_clm_r13b03_seaice.run && cd ..
+```
+
+Navigate back to ICON root folder and collect the GPU statistics:
+```console
+cd ..
+python3 scripts/cscs_ci/probtest_container_wrapper.py stats $EXPERIMENT --stats-file-path stats_gpu.csv --build-dir nvhpc_gpu
+```
+
+This saves the GPU stats as `stats_gpu.csv` in your ICON root directory.
+
+## 6. Check GPU Statistics Against Reference and Tolerance
+
+From your ICON root directory, run the check using the generated reference and tolerance:
+```console
+python3 scripts/cscs_ci/probtest_container_wrapper.py check $EXPERIMENT --input-file-cur stats_gpu.csv --input-file-ref nvhpc_cpu/${EXPERIMENT}_reference.csv --tolerance-file-name nvhpc_cpu/${EXPERIMENT}_tolerance.csv --build-dir $(pwd)
+```
+
+## 7. Increase Ensemble Size if Validation Fails
+A 10-member ensemble may not capture the full variability, causing false negatives. Increase to 49 members for better coverage from your CPU build directory:
+
+Run additional members (11–49):
+```console
+cd nvhpc_cpu
+./make_runscripts $EXPERIMENT
+uenv run ${UENV_VERSION} -- python3 scripts/cscs_ci/probtest_container_wrapper.py ensemble $EXPERIMENT --build-dir $(pwd) --member-ids $(seq -s, 11 49)
+```
+
+Regenerate reference and tolerance using all 49 members:
+```console
+python3 scripts/cscs_ci/probtest_container_wrapper.py tolerance $EXPERIMENT --build-dir $(pwd) --member-ids $(seq -s, 1 49)
+```
+
+*If the test still fails, the GPU result is likely incorrect.*