The Secure Shell (SSH) protocol is used to access nodes on
Levante. SSH client programs are available for all major operating systems.
To access one of the Levante login nodes you can use the following command:
ssh -X <userid>@levante.dkrz.de
in which <userid> must be replaced by your individual user account.
After having logged into Levante, you will find yourself on one of
four login nodes: levante[3-6]. The login nodes serve as an
interface to the compute nodes of the HPC cluster. They are intended
for file editing and compilation of source code, as well as for
submission, monitoring and canceling of batch jobs. They can also be used for simple and
not time nor memory intensive serial processing tasks.
All DKRZ systems are managed by a central directory service
(LDAP). The password can be changed through the user and project
portal. A user defined password must
contain at least eight nonblank characters and has to be a combination
of upper and lowercase letters, numbers and special
characters. Additional checks are made to ensure that the password is
not in a dictionary and not overly simple. The online service will
give you detailed error messages if and why a new password was
rejected.
In case you do not remember your password please request a
password reset.
The default password authentication is neither comfortable nor very
secure. In order to use public key authentication, you have to
generate a key pair and upload the public key to DKRZ. The command for
key generation is ssh-keygen. It supports different key types. We
recommend ed25519 keys.
ssh-keygen -t ed25519
Please use a strong passphrase to secure your key. By default, this
created two files named id_ed25519 and id_ed25519.pub.
The public key can be selected from a file by pressing the “Browse”
button or pasted directly into the Key input field. Do not select UFTP
unless you want to use this key for UFTP exclusively. After pressing
“Register key”, the key is uploaded to the server. In order to use it
on Levante, you have to provide your LDAP password.
After that your key should be active and ready to use.
Levante does not yet support ecdsa-sk or ed25519-sk keys (FIDO).
The validity or lifetime of the keys is six weeks. You should receive
an email one day before the key expires. You then have to upload a
newly created key to continue using public key authentication. The
old key is blocked from further use at DKRZ and cannot be uploaded
again.
You may require multiple SSH keys for different computer
centers. Reasons for this are added security and the fact that
policies for key properties and lifetime may differ from site to site.
To prevent your SSH client from trying out all available keys, you
should tell it exactly where to use which key. For this purpose you
can create or edit the configuration file in ~/.ssh/config.
The default login shell for new DKRZ users is bash. You can
change your login shell to tcsh, zsh etc. using the
user and project portal. The settings you
would like to use every time you log in can be put into special shell setup
files. A login bash shell looks for .bash_profile, .bash_login or
.profile in your home directory and executes commands from the first file
found. A non-login bash shell or bash subshell reads .bashrc file. Tcsh
always reads and executes .tcshrc file (or .cshrc if .tcshrc is not
found). If tcsh is invoked as login shell, file .login is sourced
additionally. The typical tasks and setting that can be put in shell setup
files are for example:
Creation of a custom prompt
Modification of search path for external commands and programs
Definition of environment variables needed by programs or scripts
Definition of aliases
Execution of commands (e.g. moduleload<modname>/<version>)
Your primary Unix group will be set to the first project your account
is member of. This will not be modified until the project
expires. Even if you join other projects, the primary group will stay
the same. If you wish to have a different primary group (e.g. you are
working most of the time for a different project), you will have to
contact DKRZ user support, there is no option to
change the primary group on your own.
To cover the software needs of DKRZ users and to maintain different
software versions, the DKRZ uses the module environment. Loading a module adapts your
environment variables to give you access to a specific set of software
and its dependencies. The modulefiles are not organized hierarchically but
have internal consistency checks for dependencies and can uniquely be
identified by naming convention <modname>/<version>.
Each modulefile is named following the below pattern
i.e. the package name is followed by the version of the package, then,
if it applies, the MPI implementation (name and version), and the
compiler (name and version) used to build the package, for example:
Such long module file names are needed to avoid naming conflicts and
to support users in the selection of compatible software variants.
Note
If only the <modname> (i.e. without <version>) is supplied
to the module command, the lexicographically highest version or
one marked as default explicitly is loaded. In most cases this will
result in incompatible software combinations. It is important to
ensure that you use modules for required libraries built with the
same compiler and MPI that you chose to build your program.
In many cases it is unproblematic to use newer compiler and MPI
versions for your program than those used to build libraries in the
software tree, if those are compatible.
Users can load, unload and query modules through the module
command. The most important module sub-commands are:
moduleavail: Shows the list of all available modules
moduleshow<modulefile>: Shows the effect of loading the module
<modulefile> on your environment. For a deeper insight into the
modules, please use the
spack
command described below.
moduleload<modulefile>: Loads selected module; a default or
lexicographically highest version is chosen, if only <modname>
without <version> is specified
Apart from the module
command there is a
command called spack for further investigation of the installed
software. In fact it is the package manager which is used by DKRZ to
build and maintain the software tree. You can find more documentation
at https://spack.readthedocs.io.
A proper replacement for moduleavail is spackfind, which
represents the most basic information: package name and version as
well as compiler used to buid the package. To list all software
packages installed on Levante, use:
$ spack find
Let’s use the netcdf-c package to compare module and spack
commands:
netcdf-c has a lot of dependencies and can be built in different
ways (serial, parallel, w/o netcdf4, w/o thread safety). module on
the other hand does not provide much information on this. In the case
of netcdf-c it is possible to inspect the installation with the
nc-config tool, but not many package provide such a tool.
First let’s try to seperate the available netcdf-c package from each
other. Here the -l option of spackfind is useful:
Each installed package has a unique alphanumeric string (hash),
which can be used to identify this package. The first six characters
of the hash are used as part of the installation path. Let’s inspect
the first package in the list (f7hh57c). The flags -p and
-d will print the path and dependency information:
In this view you can see each individual runtime dependency and where
it is located. This is far more than the module command can do.
Obviously this is a serial version of netcdf-c without support for
parallel I/O useful for tools like CDO. Please note that the spack
hash must be prepended with a slash (/) so that spack can handle
it as special input instead of a normal package name.
This installation of netcdf is parallel and supports hdf4 and hdf5
with szip compression. This installation is useful for a climate or
NWP model if you want to build it using the Intel compiler and the
Open MPI library.
These differences in building a package are called variants and can
printed with the -v option:
Instead of using hash 2k3cmu3 it is possible to identify a package
by its name, version (@), compiler (%), variant
(+, -, ~), and dependencies (^), for example:
Note, that modulefiles are not generated for every installed software
package on Levante. The easiest way to add a package (without an
existing modulefile) to the user environment is to use the spackload command, for example:
To remove packages from the user environment, the spackunload command
can be used:
$ spack unload bison xterm
Loading software packages with the spackload command modifies a
number of environment variables, such as PATH,
LD_LIBRARY_PATH, MANPATH, PKG_CONFIG_PATH etc. To see
changes that will be introduced into your environment by loading a
software package with spack, you can use the -sh option:
spackload--shbison%gcc
To list packages loaded into your environment with spackload, you
can use