- Infos im HLRS Wiki sind nicht rechtsverbindlich und ohne Gewähr -
- Information contained in the HLRS Wiki is not legally binding and HLRS is not responsible for any damages that might result from its use -

Extending the software stack on Hunter with Spack

From HLRS Platforms
Revision as of 10:39, 25 November 2024 by Hpcjgrac (talk | contribs)
Jump to navigationJump to search

This page is work in progress!!!

In its current form, this page is meant for HLRS staff. Expect thing to change and break at any time.

Introduction

Most of the software stack on Hunter is build on top of Spack. Once released, a software stack is immutable. Also, during development of a new software stack, working directly on the files and directories is strongly discouraged. The normal workflow is to use Spack's capabilities to "chain" a private Spack instance to an upstream instance (i.e. the HLRS release). This allows to do modifications, tests, and adjustments to your private Spack instance without interfering with the upstream instance. Once you are satisfied with your modifications, it is possible to incorporate them and thus extend the upstream instance without mutating existing state.

Pre-requisites

  1. working access to internet from Hunter, e.g. ssh reverse tunnel with proxy setup


Spack Module

Spack is available after loading the module spack-user. The module however, can only be loaded if the environment variables TMPDIR and SPACK_USER_PREFIX are exported and point to existing directories. Spack will be configured to expect and place most of its data in the directory $SPACK_USER_PREFIX.

$ export TMPDIR=/localscratch/${UID}

$ export SPACK_USER_PREFIX=${HOME}/my-spack
$ mkdir -p $TMPDIR; mkdir -p $SPACK_USER_PREFIX
$ module use /sw/hunter-rh9/hlrs/testing/release/main/modulefiles

$ module load spack-user


Essentially, the module does two things:

  1. set the environment variable SPACK_ROOT pointing to the base of HLRS' Spack instance, and
  2. provide the command alias spack which wraps the original Spack command and adds some additional parameters to pick up relevant configuration files.
$ alias spack
alias spack='spack -C $SPACK_ROOT/../config-hlrs -C $SPACK_ROOT/../config-user'


The first time you use Spack it is recommended to bootstrap it. This may take up to several minutes

$ spack bootstrap now


Your private Spack environment

All your private configuration and customization needs to be done in a so-called Spack environment. These are essentially directories below $SPACK_USER_PREFIX/environments. You may create multiple environments, but only one can be active at any given time. Creating an environment takes a somewhat lengthy command:

spack env create --include-concrete $SPACK_ROOT/../environments/hlrs --without-view my-tools

==> Created environment my-tools in: .../my-spack/environments/my-tools

==> Activate with: spack env activate my-tools

This will create the environment labeled my-tools in the indicated directory.

The option --include-concrete is what ties your environment to the HLRS software stack and must therefore not be omitted.

Next, you need to activate your environment. Unlike the message above, activation requires a slightly different command

$ eval $(spack env activate my-tools --sh)

Deactivate a Spack environment with the command despacktivate.

Note, that activating a Spack environment conveniently set the shell environment (yes, too many "environments") variable SPACK_ENV, which points to the environments directory. In our case this is $SPACK_USER+PREFIX/environments/my-tools.

All configuration is done in the file spack.yaml. You may take inspiration from our template $SPACK_ROOT/../templates/user-env-spack.yaml.

$ cat $SPACK_ROOT/../templates/user-env-spack.yaml <\br>