diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md index ecd4d4191..2c0206e2c 100644 --- a/docs/SUMMARY.md +++ b/docs/SUMMARY.md @@ -4,7 +4,7 @@ * [Events](events/) * [Tasks](tasks/) * [User Support](support/) -* [Alps](alps/) +* [HPC](hpc/) * [Models](models/) * [Tools](tools/) * [Datasets](datasets/) diff --git a/docs/alps/SUMMARY.md b/docs/alps/SUMMARY.md deleted file mode 100644 index d51401b43..000000000 --- a/docs/alps/SUMMARY.md +++ /dev/null @@ -1,3 +0,0 @@ -* [Alps](index.md) -* [Santis](santis.md) -* [Eiger](eiger.md) \ No newline at end of file diff --git a/docs/datasets/ecmwf_data_cube.md b/docs/datasets/ecmwf_data_cube.md index 159a0c135..ff3d4d436 100644 --- a/docs/datasets/ecmwf_data_cube.md +++ b/docs/datasets/ecmwf_data_cube.md @@ -13,7 +13,7 @@ which is a storage system at the ECMWF Data Center in Bologna. - 0.5 PB of fast SSD storage - 2 PB of “slow” storage - Build via work package of SwissTwins project -- Physically installed at ECMWF Bologna with a direct connection to Alps Lugano [ALPS](../alps/index.md) +- Physically installed at ECMWF Bologna with a direct connection to Alps Lugano [ALPS](../hpc/index.md) - Part of the multi-site distributed infrastructure (Lugano, Lausanne, Bologna) ## Usage diff --git a/docs/events/icon_meetings/2022-4.md b/docs/events/icon_meetings/2022-4.md index 1a75fd150..b57a1fa96 100644 --- a/docs/events/icon_meetings/2022-4.md +++ b/docs/events/icon_meetings/2022-4.md @@ -89,6 +89,6 @@ BG says EXTPAR can process the CORINE dataset. She didn’t get any errors but h ### General WS asks if there are any news about the ICON seamless effort. MA cannot give any more stable information. -JB will send mail. There is information on the [COSMO website :material-open-in-new:](https://www.cosmo-model.org/content/tasks/workGroups/wg3b/default.htm){:target="_blank"}. +JB will send mail. There is information on the [COSMO website :material-open-in-new:](https://www.cosmo-model.org/content/tasks/workGroups/wgPHY/default.htm){:target="_blank"}. DB sends a reminder about the COSMO/ICON user workshop on 2 February. Please send him your registration and indicate if you want to present a poster. diff --git a/docs/events/icon_meetings/2024-3.md b/docs/events/icon_meetings/2024-3.md index e3371f1d6..a60c32d85 100644 --- a/docs/events/icon_meetings/2024-3.md +++ b/docs/events/icon_meetings/2024-3.md @@ -49,7 +49,7 @@ AH asks whether Daint or Säntis will be the long-term solution for their applic DF then asks if they can expect the images from Tödi to be available on Daint. ML replies that although it's not a long-term solution, those images should eventually be deployed on Daint. However, it won’t be as straightforward. DF points out that many people need to complete benchmarking by October. AH asks if ICON can be compiled by loading the user environment without those missing images. ML explains that the software stack image is the same as the user environment and can be accessed on each cluster, but using it isn't always straightforward. Currently, the images are interchangeable, though they still depend on the operating system. In the future, user environments on one cluster may not work on another. MK confirms that the risk of incompatibility exists and they've already seen this with MCH. Regarding the missing images on Daint, MK explains that the NVIDIA part wasn’t provided completely, which is why he created another environment. These images are not preloaded on the system, so users need to search for them using the user environment tool. MK suggests trying the command `uenv image find -s todi` to locate the images. -BG mentions using Tödi and that while the instructions on C2SM initially worked, two weeks later the executables stopped working and the instructions disappeared. ML responds that there is now an Alps section on the landing page. It should be possible to do the same tasks as before, but without using absolute paths to his environment. Instead, tags can be used. The updated instructions are available on the [workshop materials page :material-open-in-new:](https://c2sm.github.io/alps/#introductory-workshop-material){:target="_blank"} as well as the [user landing page :material-open-in-new:](https://c2sm.github.io/alps/uenvs/#the-uenv-command-line-tool){:target="_blank"}. ML encourages anybody to reach out to him on Slack if any issues arise. +BG mentions using Tödi and that while the instructions on C2SM initially worked, two weeks later the executables stopped working and the instructions disappeared. ML responds that there is now an Alps section on the landing page. It should be possible to do the same tasks as before, but without using absolute paths to his environment. Instead, tags can be used. The updated instructions are available on the [workshop materials page](../../hpc/index.md#introductory-workshop-material) as well as in the CSCS Documentation under [uenv :material-open-in-new:](https://docs.cscs.ch/software/uenv/#downloading-uenv){:target="_blank"}. ML encourages anybody to reach out to him on Slack if any issues arise. AD from Empa asks if ML has managed to run ICON-ART. ML responds that he has only been testing regular ICON, but Erik was successful in compiling it. AH confirms that Erik followed a workaround. AD asks if there is a plan for C2SM to support ICON-ART. ML explains that ICON-ART is not in the C2SM pipeline and suggests that users create a test case, and C2SM can help set up the CI infrastructure. AH mentions they have an old test case (icon-kit) that needs updating. MJ adds that ICON-ART is now part of ICON-NWP, and a test case could be set up and added. diff --git a/docs/hpc/SUMMARY.md b/docs/hpc/SUMMARY.md new file mode 100644 index 000000000..52e7faff1 --- /dev/null +++ b/docs/hpc/SUMMARY.md @@ -0,0 +1,6 @@ +* [CSCS](index.md) + * [Santis](santis.md) + * [Balfrin](balfrin.md) + * [Eiger](eiger.md) +* ETHZ + * [Euler](euler.md) \ No newline at end of file diff --git a/docs/hpc/balfrin.md b/docs/hpc/balfrin.md new file mode 100644 index 000000000..a721282e7 --- /dev/null +++ b/docs/hpc/balfrin.md @@ -0,0 +1,6 @@ +# Balfrin + +Balfrin is an Alps cluster used by MeteoSwiss. + +C2SM does not officially support Balfrin. Nevertheless, you can find instructions on how +to set up ICON on Balfrin at the [Compile section of ICON](../models/icon/compile.md#balfrin). \ No newline at end of file diff --git a/docs/alps/eiger.md b/docs/hpc/eiger.md similarity index 100% rename from docs/alps/eiger.md rename to docs/hpc/eiger.md diff --git a/docs/hpc/euler.md b/docs/hpc/euler.md new file mode 100644 index 000000000..712138f51 --- /dev/null +++ b/docs/hpc/euler.md @@ -0,0 +1,45 @@ +# Euler + +Euler is ETH Zurich's central high-performance computing (HPC) cluster, providing computational resources for research across all disciplines. + +## Access + +Groups at IAC have shares for storage and CPU via the Euler **Climate** group. As a group member, you should have an existing account and access to Euler. If this is not the case, contact your group leader or Urs Beyerle. + +## Useful Links + +- [Euler documentation :material-open-in-new:](https://docs.hpc.ethz.ch/){:target="_blank"} +- [Climate Euler users :material-open-in-new:](https://wiki.iac.ethz.ch/Collaboration/EulerUsers){:target="_blank"} (:material-lock: IAC login required) +- [Accounting :material-open-in-new:](https://wiki.iac.ethz.ch/Collaboration/EulerAccounting){:target="_blank"} (:material-lock: IAC login required) +- [Euler Climate members :material-open-in-new:](https://wiki.iac.ethz.ch/Collaboration/EulerClimateMembers){:target="_blank"} (:material-lock: IAC login required) + +## Software Stack + +Euler provides a [software stack :material-open-in-new:](https://docs.hpc.ethz.ch/software/software-stack/){:target="_blank"} via the module command: + +```bash +module load stack openmpi +module list +# Currently Loaded Modules: +# 1) gcc/12.2.0 2) stack/2025-06 3) openmpi/4.1.7 +``` + +Afterwards, one can load specific software such as: + +```bash +module load cdo nco ncview netcdf-c +module list +# Currently Loaded Modules: +# 1) gcc/12.2.0 2) stack/2025-06 3) openmpi/4.1.7 4) cdo/2.4.4 5) nco/5.2.4 6) ncview/2.1.9 7) netcdf-c/4.9.2 +``` + +With that, one has access to the most commonly used tools for climate applications: + +```bash +which cdo ncdump ncview ncrcat +# /cluster/software/stacks/2025-06/linux-ubuntu22.04-x86_64_v3/gcc-12.2.0/cdo-2.4.4-spns4mysmzmbj7eh37zswj64efou4xvr/bin/cdo +# /cluster/software/stacks/2025-06/linux-ubuntu22.04-x86_64_v3/gcc-12.2.0/netcdf-c-4.9.2-sekz6xps6vd4zyacqlj6e4gesed7hi7t/bin/ncdump +# /cluster/software/stacks/2025-06/linux-ubuntu22.04-x86_64_v3/gcc-12.2.0/ncview-2.1.9-hszhkgti42fealjswfivjjk5r3i3xcob/bin/ncview +# /cluster/software/stacks/2025-06/linux-ubuntu22.04-x86_64_v3/gcc-12.2.0/nco-5.2.4-zhb3mn5upr3rniqoebeyfreb7uabpohi/bin/ncrcat + +``` \ No newline at end of file diff --git a/docs/alps/index.md b/docs/hpc/index.md similarity index 94% rename from docs/alps/index.md rename to docs/hpc/index.md index ae2f46315..9715b0fc3 100644 --- a/docs/alps/index.md +++ b/docs/hpc/index.md @@ -1,4 +1,4 @@ -# The Alps System +# The Alps System at CSCS [Alps :material-open-in-new:](https://www.cscs.ch/computers/alps){:target="_blank"} is a distributed high-performance computing (HPC) infrastructure managed by CSCS. Unlike traditional HPC systems, it is composed of several logical units called vClusters (versatile clusters). @@ -20,15 +20,14 @@ The following table shows current clusters distribution on Alps at CSCS | Santis | Weather & Climate | ~ 430 Grace-Hopper nodes | :white_check_mark: | | Balfrin | MeteoSwiss | ~ 40 A100 GPU nodes | :yellow_circle: | | Eiger | CPU-only workloads | ~ 580 multicore nodes | :yellow_circle: | -| Daint | User Lab | ~ 600 Grace-Hopper nodes | :x: | -| Clariden | Machine Learning | ~ 800 Grace-Hopper nodes | :x: | :white_check_mark: Full C2SM support
:yellow_circle: Partial or limited C2SM support (help available on request)
-:x: No C2SM support +Additional vClusters without C2SM support include Daint (User Lab) and Clariden (Machine Learning). + More information about clusters on Alps is available on the [official CSCS documentation :material-open-in-new:](https://docs.cscs.ch/clusters/){:target="_blank"}. @@ -44,7 +43,7 @@ Host ela User cscsusername IdentityFile ~/.ssh/cscs-key -Host santis* daint* +Host santis* eiger* Hostname %h.alps.cscs.ch User cscsusername IdentityFile ~/.ssh/cscs-key @@ -62,9 +61,9 @@ e.g., `ssh santis-ln002`. Replace `cscsusername` with your actual username. ## User Environments -Software stacks at CSCS are now accessible through the so-called User Environments (uenv). +Software stacks at CSCS are now accessible through the so-called User Environments (uenvs). User environments contain the minimal software stack required for a certain activity, say, building and running ICON. -They are generated by `spack`, packed into single `squashfs` file and then mounted by the user. +They are generated by `spack`, packed into single `squashfs` files and then mounted by the user. In a way, they can be considered as poor man's containers. !!! success "Main Advantages of Uenvs" diff --git a/docs/alps/santis.md b/docs/hpc/santis.md similarity index 100% rename from docs/alps/santis.md rename to docs/hpc/santis.md diff --git a/docs/index.md b/docs/index.md index 3cef08df5..38babecd8 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,6 +1,6 @@ **Welcome to the C2SM User Landing Page!** Here we collect helpful information for scientists and partners -about the [C2SM core team](about/index.md), [tasks](tasks/index.md), [support](support/index.md), the [Alps system](alps/index.md), [models](models/index.md), [tools](tools/index.md), [datasets](datasets/index.md) and much more. +about the [C2SM core team](about/index.md), [tasks](tasks/index.md), [support](support/index.md), supported [HPC systems](hpc/index.md), [models](models/index.md), [tools](tools/index.md), [datasets](datasets/index.md) and much more. ## News \ No newline at end of file diff --git a/docs/models/icon/compile.md b/docs/models/icon/compile.md index fa74566b1..0a04d8a6e 100644 --- a/docs/models/icon/compile.md +++ b/docs/models/icon/compile.md @@ -2,41 +2,35 @@ ## Access -The [ICON repository :material-open-in-new:](https://github.com/C2SM/icon){:target="_blank"} is hosted on the C2SM GitHub organisation. If you do not have access, please follow the instructions under [How to get Access](../../about/index.md#how-to-get-access). +Since 2024, ICON is open-source and comes with semi-annual, public releases, which +can be accessed via the public [icon-model :material-open-in-new:](https://gitlab.dkrz.de/icon/icon-model){:target="_blank"} repository. - If you do not already have an SSH key set up for GitHub, but would like to do so, follow the [instructions :material-open-in-new:](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent){:target="_blank"}. - -Since 2024, ICON is open-source and comes with semi-annual releases, which -can be accessed via [this public repository :material-open-in-new:](https://gitlab.dkrz.de/icon/icon-model){:target="_blank"}. - -If you are an ICON developer, you should have access to the DKRZ GitLab, where the original ICON repository is hosted. All developments related to GPU go -into the [`icon-nwp` repository :material-open-in-new:](https://gitlab.dkrz.de/icon/icon-nwp){:target="_blank"}. +If you are an ICON developer, you should have access to the DKRZ GitLab, where the original ICON repository is hosted. All developments related to GPU porting and NWP settings go +into the [icon-nwp :material-open-in-new:](https://gitlab.dkrz.de/icon/icon-nwp){:target="_blank"} repository. -## Configure and compile +## Configure and Compile + +### Cloning ICON Repository Below you find instructions on how to compile different flavors of ICON on C2SM-supported machines. Clone the ICON repository: -=== "C2SM (latest release)" - ```console - git clone -b release-2025.10 --recurse-submodules git@github.com:C2SM/icon.git - ``` - -=== "DKRZ (latest release)" +=== "Latest release at DKRZ GitLab" ```console git clone -b release-2025.10-public --recurse-submodules https://gitlab.dkrz.de/icon/icon-model.git ``` -=== "DKRZ (icon-nwp master)" +=== "icon-nwp master branch at DKRZ GitLab" ```console git clone --recurse-submodules git@gitlab.dkrz.de:icon/icon-nwp.git ``` +### Compiling -### Säntis +#### Säntis -Run the following after navigating into ICON root folder: +Run the following after navigating into the ICON root folder: === "CPU compilation" ```console @@ -62,8 +56,7 @@ Run the following after navigating into ICON root folder: uenv image pull $UENV_VERSION ``` - -#### Building out-of-source +##### Building out-of-source Out-of-source builds are useful if you want to have two or more compiled versions of ICON in the same repository. To achieve that, you simply need to create separate folders in the ICON root folder @@ -91,39 +84,24 @@ Then, navigate into the corresponding folder and source the configure wrapper fo uenv run ${UENV_VERSION} -- ./../config/cscs/santis.gpu.nvhpc ``` -### Euler - -Navigate into the ICON root folder. - -Now, set up your spack instance: +#### Balfrin -```bash -# Setup spack -SPACK_TAG=$(cat "config/ethz/SPACK_TAG_EULER") -git clone --depth 1 --recurse-submodules --shallow-submodules -b ${SPACK_TAG} https://github.com/C2SM/spack-c2sm.git -. spack-c2sm/setup-env.sh -``` +Run the following after navigating into the ICON root folder: -Euler Support recommends to compile code on compute nodes. There, -we can take advantage of multi-core compiling. -However, we need to load the module `eth_proxy`, which enables connecting from a compute node -to an external service, e.g. GitHub or GitLab. +=== "CPU compilation" + ```console + ./config/cscs/alps_mch.cpu.nvidia + ``` -```console -module load eth_proxy -``` +=== "GPU compilation" + ```console + ./config/cscs/alps_mch.gpu.nvidia + ``` -Now, activate the spack environment and build ICON: +You can run the above scripts out-of-source also. -```bash -# Build ICON -# For out-of-source builds: navigate into the build folder and -# adapt the path to the Spack environment in the following -spack env activate -d config/ethz/spack/${SPACK_TAG}/euler_cpu_gcc -srun -N 1 -n 12 --mem-per-cpu=1G spack install -j 12 -``` -### Eiger +#### Eiger Pull and start the Environment: @@ -229,4 +207,37 @@ spack concretize spack install ``` -`concretize` resolves dependencies, and `install` builds the packages. \ No newline at end of file +`concretize` resolves dependencies, and `install` builds the packages. + + +#### Euler + +Navigate into the ICON root folder. + +Now, set up your spack instance: + +```bash +# Setup spack +SPACK_TAG=$(cat "config/ethz/SPACK_TAG_EULER") +git clone --depth 1 --recurse-submodules --shallow-submodules -b ${SPACK_TAG} https://github.com/C2SM/spack-c2sm.git +. spack-c2sm/setup-env.sh +``` + +Euler Support recommends to compile code on compute nodes. There, +we can take advantage of multi-core compiling. +However, we need to load the module `eth_proxy`, which enables connecting from a compute node +to an external service, e.g. GitHub or GitLab. + +```console +module load eth_proxy +``` + +Now, activate the spack environment and build ICON: + +```bash +# Build ICON +# For out-of-source builds: navigate into the build folder and +# adapt the path to the Spack environment in the following +spack env activate -d config/ethz/spack/${SPACK_TAG}/euler_cpu_gcc +srun -N 1 -n 12 --mem-per-cpu=1G spack install -j 12 +``` diff --git a/docs/models/icon/icon-clm.md b/docs/models/icon/icon-clm.md index 144b17b4a..53d788015 100644 --- a/docs/models/icon/icon-clm.md +++ b/docs/models/icon/icon-clm.md @@ -11,7 +11,7 @@ tailored for ICON-CLM simulations. [official docs :material-open-in-new:](https://hereon-coast.atlassian.net/wiki/spaces/SPICE/pages/983065){:target="_blank"}. !!! note - It is assumed that the [uenvs](../../alps/index.md#user-environments) for ICON and pre-/postprocessing are in place. For that, the following has + It is assumed that the [uenvs](../../hpc/index.md#user-environments) for ICON and pre-/postprocessing are in place. For that, the following has to be executed once: ```bash diff --git a/docs/posts/2024-08-14_alps.md b/docs/posts/2024-08-14_alps.md index c074a543d..b5b331197 100644 --- a/docs/posts/2024-08-14_alps.md +++ b/docs/posts/2024-08-14_alps.md @@ -8,7 +8,7 @@ categories: # New information about the Alps system available Since the transition from Piz Daint to the new Alps infrastructure is already taking place, -we have added a new navigation section [Alps](../alps/index.md) to collect all necessary information there. +we have added a new navigation section [Alps](../hpc/index.md) to collect all necessary information there. diff --git a/docs/posts/2024-12-17_alps_update.md b/docs/posts/2024-12-17_alps_update.md index 57c0c87b2..5c318bca3 100644 --- a/docs/posts/2024-12-17_alps_update.md +++ b/docs/posts/2024-12-17_alps_update.md @@ -7,7 +7,7 @@ categories: # Update on the current status on Alps -Last week, CSCS deployed the Climate and Weather vCluster `santis`. As some fine-tuning is still ongoing, the [Santis section](../alps/santis.md) provides an overview of how to transition from `todi` to `santis`. +Last week, CSCS deployed the Climate and Weather vCluster `santis`. As some fine-tuning is still ongoing, the [Santis section](../hpc/santis.md) provides an overview of how to transition from `todi` to `santis`. diff --git a/docs/tools/icontools.md b/docs/tools/icontools.md index 67a03191a..2855c2f13 100644 --- a/docs/tools/icontools.md +++ b/docs/tools/icontools.md @@ -17,7 +17,7 @@ please contact your group's technical contact. They will be responsible for addi ### 1) Load via module command -On [Säntis](../alps/santis.md), DWD ICON Tools is provided via the `climtools` uenv: +On [Säntis](../hpc/santis.md), DWD ICON Tools is provided via the `climtools` uenv: ``` uenv start --view=modules /capstor/store/cscs/userlab/cwd01/leclairm/uenvs/images/climtools_25.2_v2.sqfs