Python#

New concept under development

We are rolling out this approach to handling python packages. We might have to adjust details based on user feedback, so keep an eye on these pages.

We recommend that you build and manage your own Python environment. It is reproducible, it is under your control, and you can add packages to it as you need.

The three tools we support for this are:

uv – packages from PyPI
pixi – packages from conda-forge
micromamba – packages from conda-forge

Quick start

No preference yet? This gets you a Python interpreter with a typical analysis stack:

cd ~/my_awesome_project
wget -nc https://dkrz-sw.gitlab-pages.dkrz.de/python-envs/specs/km-scale/pyproject.toml
uv run python

Want a Jupyter kernel instead of a normal python in the shell? Just extend the uv run python line a bit, and it’s good for use in our Jupyterhub (you might need to reload in your browser).

uv run python -m ipykernel install --user --name my-kernel --display-name="My Kernel"

Add packages with the tool that built the environment – uv add, pixi add or micromamba install – never with pip, see why that breaks things for details.

Create your environments on the /home file system (the default for all three tools), not on /work. They consist of a huge number of small files, and creating an environment, starting Python and importing packages are all considerably faster on /home – see why, and what counts against your quota.

If you need GPU-enabled packages, there is an extra step – the conda-based tools need to be told which CUDA version to target, since Levante’s login nodes have no GPU to detect it from. For dask and mpi4py, see our parallel-Python guide – mpi4py in particular needs a rebuild against Levante’s own MPI, since neither the PyPI nor the conda-forge version speaks the PMIx interface Slurm uses here.

The system wide python3 modules still exist, but they are frozen and no longer updated. You can keep using them in existing workflows, but we recommend switching to your own environments built with one of the tools above – more on those further down.

Choosing a tool#

No strong preference? Use uv – it is the fastest to set up and needs no extra configuration for GPU-enabled packages.

Note

Pick one and stick with it. Each tool keeps its own package cache, so using all three costs three times the /home quota for largely the same packages. If you do switch, clean up the cache of the tool you left behind, see where those caches live.

tool	packages from	environment location	lockfile	use when
`uv`	PyPI	`.venv/` next to your project	`uv.lock`	your stack is pure Python, you want the fastest option, or you want GPU-enabled packages without extra configuration
`pixi`	conda-forge	`.pixi/` next to your project	`pixi.lock`	you need packages that only exist on conda-forge
`micromamba`	conda-forge	named environment under a root prefix	none	you want global, named environments

uv and pixi write their lockfile next to your project and pin every resolved package, so you and your colleagues can rebuild the exact same environment later – keep it under version control. micromamba has no lockfile: an environment.yaml is re-resolved on every install and may well give you different versions each time. See the uv and pixi documentation for what lockfiles contain, and the specification file documentation for the closest micromamba equivalent.

Ready-made environments#

Before you assemble your own package list, have a look at https://dkrz-sw.gitlab-pages.dkrz.de/python-envs/. It publishes curated specifications for the typical DKRZ analysis stack (xarray, dask, intake, healpix, easygems, cartopy, …).

Every specification is built and its workloads are tested with all three installers on Levante on a daily basis. The page shows the resulting pass / fail matrix. Specifications that can make use of a GPU node on Levante are marked in the GPU column.

Two files are published per environment: the conda environment.yaml and an auto-converted pyproject.toml. Download the one your tool needs:

SPEC=km-scale  # pick a specification from the page above
BASE=https://dkrz-sw.gitlab-pages.dkrz.de/python-envs/specs/$SPEC

# uv
wget -nc $BASE/pyproject.toml
uv sync

# pixi
wget -nc $BASE/environment.yaml
pixi init --import environment.yaml

# micromamba
wget -nc $BASE/environment.yaml
micromamba create -f environment.yaml

The specifications are maintained at https://gitlab.dkrz.de/dkrz-sw/python-envs. Have a look there if you want to contribute a specification; for installing an environment the download above is all you need.

Building your own environment#

All three examples below name a Python version. That is deliberate – see the pinning section for why it matters.

Using uv#

uv init --python 3.13 my_project
cd my_project
uv add xarray dask
uv run python

uv can also install the interpreter itself, independently of any module:

uv python install 3.13

Using pixi#

pixi init my_project
cd my_project
pixi add python=3.13 xarray dask
pixi run python
pixi shell # to get a shell with the current environment in the PATH

Add python yourself, as in the example: a fresh pixi project contains no interpreter at all.

Using micromamba#

If you know conda, this will feel familiar: same verbs (create, activate, install, list, env remove), same conda-forge packages – just faster, and without conda’s own paid-channel footgun. There is no plain conda/mamba on your default PATH – only a module provides those – so micromamba is the standalone replacement.

Once per account, add the shell hook:

micromamba shell init --shell bash # adjust to your shell if needed.

Then create and use environments:

micromamba create -n my_env python=3.13 xarray dask
micromamba activate my_env # puts the environment into the PATH
micromamba install netcdf4
python

Unlike the other two tools, micromamba does not put the environment next to your project: named environments live under $MAMBA_ROOT_PREFIX, together with the package cache, see the file-system section.

Note

micromamba shell init writes the hook to ~/.bashrc, which bash reads in interactive non-login shells only. If micromamba activate is not available in your login shells, see why; for batch jobs, where it is never available, see the section below.

Using your environment#

The uv and pixi commands below assume you are in your project directory; micromamba commands name the environment explicitly and work from anywhere.

To activate an environment in your interactive shell:

source .venv/bin/activate     # uv, in your project directory
pixi shell                    # pixi, in your project directory
micromamba activate my_env    # micromamba

Editors and IDEs usually want the path of the interpreter instead. Ask the tool rather than guessing, then paste the result into your editor’s Python interpreter setting:

uv run which python
pixi run which python
micromamba run -n my_env which python

To update the packages in an environment:

uv lock --upgrade && uv sync
pixi update
micromamba update -n my_env --all # WARNING: This will break pinnings you had in your environment.yaml file.

To rebuild somebody else’s environment exactly as they had it – this is what the lockfile is for, and it is the answer to “can I have the same environment as you”:

git clone <repository> && cd <repository>

uv sync --frozen        # uv
pixi install --frozen   # pixi

Registering a Jupyter kernel#

An environment and a Jupyter kernel are two different things:

The environment is the actual installation: a Python interpreter together with all its packages, in a directory on disk. This is what you use on the command line and in batch jobs, and it is what everything above creates.
A kernel is just a registration that tells Jupyter which interpreter to start for a given name / label. It contains no packages of its own, it only points at an environment.

Install ipykernel into the environment, then run the registration with that environment’s interpreter:

# uv, in your project directory
uv add ipykernel
uv run python -m ipykernel install --user --name my-kernel --display-name="My Kernel"

# pixi, in your project directory
pixi add ipykernel
pixi run python -m ipykernel install --user --name my-kernel --display-name="My Kernel"

# micromamba
micromamba install -n my_env ipykernel
micromamba run -n my_env python -m ipykernel install --user --name my-kernel --display-name="My Kernel"

--name is the internal name of the kernel, --display-name is the label shown in JupyterHub. Neither has to match the name of the environment, but keeping them recognisable helps. My Kernel can then be selected at https://jupyterhub.dkrz.de, which is available to all users who may submit batch jobs; reload the browser tab if the kernel does not show up yet.

Because the kernel only points at the environment:

Packages you install into the environment later are available in the kernel right away, just restart the kernel. You do not have to register it again.
If you delete, move or rename the environment, the kernel entry stays behind and will fail to start. Use jupyter kernelspec list to see your kernels and jupyter kernelspec remove my-kernel to clean up.

See our JupyterHub documentation and its kernels page for details.

Using an environment in a batch job#

A batch script runs in a shell that is neither interactive nor a login shell, so it reads neither ~/.bashrc nor ~/.bash_profile, see why. Slurm does pass the environment variables of your submitting shell into the job; micromamba activate is a shell function instead, and those are not passed along (yes, there are exceptions). This is the most common reason for jobs failing with command not found or picking up the wrong Python.

For a job that runs a single Python process, let the tool start it:

cd /home/x/xz0123/my_project

uv run python my_script.py
# or
pixi run python my_script.py
# or
micromamba run -n my_env python my_script.py

If you really need the environment activated in the job script, load the shell hook first:

eval "$(micromamba shell hook -s posix)"
micromamba activate my_env

python my_script.py

Parallel jobs#

Do not put uv run, pixi run or micromamba run behind srun. Each of the tasks would start its own copy of the tool, and they then contend for the same lockfile – with micromamba the job can stall on Cannot lock '~/.cache/mamba/proc'. Resolve the interpreter once, before srun, and let Slurm start it directly:

cd /home/x/xz0123/my_project

PY=$(uv run which python)                       # uv
# PY=$(pixi run which python)                   # pixi
# PY=$(micromamba run -n my_env which python)   # micromamba

srun -n 4 $PY my_script.py

This is what you want for mpi4py as well, but note that mpi4py needs an environment built specifically for Levante, see our parallel-Python guide.

See more batch script examples.

Pinning the Python version#

Without a version you get whatever your tool considers newest at the time, so which Python you work with becomes a side effect of when you happened to create the environment. How much that matters later differs per tool:

uv init --python 3.13 records the version twice: as a lower bound in requires-python in pyproject.toml, and as the exact request in .python-version. Keep the latter under version control – it is the only one of the three files that names a specific interpreter, since uv.lock is universal by design and stays valid across a range of Python versions. Without it, uv uses any compatible interpreter it finds, which on Levante can be newer than the one you asked for. uv python pin changes it later, but does not adjust requires-python.
pixi writes your request into pixi.toml, and pixi.lock then records the interpreter like any other conda package – so a colleague running pixi install --frozen gets exactly your Python. Do add python explicitly, though: if it only arrives as a dependency of something else, it comes in unpinned and nothing in pixi.toml says which version you got.
micromamba has no lockfile, so the version in your create command or environment.yaml is the only record. Without it, every re-creation resolves afresh.

File system, quota and caches#

Keep your project on ``/home``. All three tools link packages out of a package cache instead of copying them, and these links cannot cross file systems. Since uv puts .venv/ and pixi puts .pixi/ next to your project, a project on /work ends up with its environment on a different file system than its cache: the tools silently fall back to full copies, which is slower and uses more space. The same applies to micromamba environments outside $MAMBA_ROOT_PREFIX.

Environments and package caches both count against your /home quota, see the file-systems page for the current value and for the quota command to check your usage.

The default locations are already on /home: ~/.cache/uv for uv, ~/.cache/rattler/cache for pixi, and $MAMBA_ROOT_PREFIX/pkgs for micromamba, whose prefix defaults to ~/micromamba. So there is nothing you need to change. To find out what is eating your quota, ask the tools rather than guessing, and check every path they print, as there may be more than one:

uv cache dir
pixi info        # see 'Cache dir'
micromamba info  # see 'package cache'

If you do move a cache, keep it on /home for the reason above. See uv (UV_CACHE_DIR) and pixi (PIXI_CACHE_DIR). For micromamba, move MAMBA_ROOT_PREFIX instead of the cache alone: named environments and the package cache both live under it, so moving the prefix takes both along and cannot split them across file systems.

pixi will warn on every run that its repodata and conda-pypi-mapping caches sit on a network file system and redirect them to /tmp for that run. That is fine and needs no action: it concerns metadata only, not the package cache your environments are linked from. Set netfs-redirect = "never" under the [cache] table in pixi’s config.toml if you would rather not see it.

Reclaiming space#

Delete environments you no longer need:

rm -rf .venv  # uv, in the project directory
pixi clean    # pixi, in the project directory
micromamba env remove -n my_env

Cleaning a cache never breaks an existing environment – the environment holds hard links to those files, so the only thing you lose is a re-download later. uv and micromamba can drop just the unused entries:

uv cache prune
micromamba clean -p

pixi has no equivalent: its cache is cleaned by type, not by usage, so pixi clean cache --conda (where the package files are) is the closest thing. To wipe a cache completely:

uv cache clean
pixi clean cache
micromamba clean -a

Pitfalls#

Do not use `pip`#

We often see users who add custom Python packages on top of an existing environment with pip. Add them with the tool that built the environment instead – uv add, pixi add or micromamba install. Using pip causes trouble both inside and outside an environment, for two different reasons.

Inside an environment, the tools do not know what pip put there, so the next uv sync, pixi install or micromamba install may remove or break it, and the environment no longer matches its specification. For packages that exist on PyPI only, pixi has pixi add –pypi; micromamba has no equivalent, list those in the pip: section of your environment.yaml so that they are at least part of the specification.

Outside an environment, typically with pip install --user, the packages end up in your home directory in ~/.local/lib/python*. These paths take precedence over the packages installed in a conda-based environment, so pixi and micromamba environments – and the system wide python3 modules – can all pick them up. Sometimes this works, but in many cases an incompatible package version gets loaded and produces hard-to-debug errors that nobody else can reproduce. Environments created by uv are not affected, as they ignore ~/.local entirely.

Usually we recommend to remove such installations by simply deleting ~/.local/lib/python*. Note that this also removes packages that were not conflicting, and may therefore break other workflows. Do this at your own risk! As a less invasive check, set PYTHONNOUSERSITE=1, which keeps Python from adding ~/.local to sys.path, and see whether that makes the problem go away.

None of this applies to a plain virtual environment that you create with python -m venv and populate with pip yourself: no other tool manages it, so pip is exactly the right tool there. That is a perfectly good workflow, but uv installs the same PyPI packages considerably faster and writes a lockfile while doing so, so we would start a new project with uv. If you already have one, From pip to a uv project walks you through turning your requirements.txt into a pyproject.toml and a lockfile.

Package channels and licensing#

Channels are the locations where conda packages are stored. On Levante the system wide configuration in /etc/conda/condarc already selects the conda-forge channel, which is free for all to use, and disables the Anaconda defaults channel. You can read it with cat /etc/conda/condarc. pixi likewise defaults to conda-forge, which it records in your pixi.toml. You therefore do not need to pass -c conda-forge yourself.

Warning

Your own configuration overrides the system wide one. Check whether you already have a ~/.condarc from an earlier setup (cat ~/.condarc): if you write one now, or install Miniconda or Anaconda, you may end up on a channel that requires a paid license – you are not allowed to use those here. In the rare cases where you need to add a channel, please make sure that its license allows free use for all.

Troubleshooting#

When a program picks up the wrong packages, find out which Python is actually running and where it looks for packages:

which python
python -c 'import sys; print(sys.executable)'
python -m site
echo $PYTHONPATH                  # should normally be empty

python -m site reports ENABLE_USER_SITE: if it is True, the user site-packages directory under ~/.local is on sys.path and its packages can shadow the ones in your environment.

micromamba activate with no environment name activates base, which is normally empty. With no module loaded, that leaves you with no python at all, which is easy to spot. If a module like python3/... is still loaded in the same shell, its bin/ directory is still on PATH, so python silently keeps resolving there instead, which looks like it worked but did not. Either way, name the environment explicitly.

If uv add or pixi add fails while compiling a package from source, that is not necessarily a missing toolchain: Levante’s login nodes have /usr/bin/gcc on PATH by default, no module load needed. It is a dated system compiler (GCC 8.5.0), though, so a build that needs a newer C/C++ standard can still fail on it – module load gcc for a newer one. Either way this only gives you the compiler itself: MPI, HDF5 and similar libraries still come from their own modules, see our parallel-Python guide.

If an import fails immediately after you added the package, check that it really was added: a rejected uv add can still leave you with a working environment that simply does not contain the new package.

uv pip list          # or: pixi list, micromamba list -n my_env
cat pyproject.toml   # or: pixi.toml

If you are stuck, get in touch with our User Support.

Legacy: system wide `python3` modules#

Warning

The python3 modules are frozen. They receive no further updates, and we do not recommend using them for new work.

The modules follow the naming convention python3/YYYY.XX-compilerversion, for example:

module load python3/2023.01-gcc-11.2.0

Always use the full module name to avoid surprises when the default changes.

These environments are CPU-only and do not contain GPU-enabled packages. For GPU support, see the GPU page.

Looking for the same kind of curated, ready-to-use environment going forward? Our specification catalog publishes the equivalent, built and tested on Levante every day with uv, pixi and micromamba.

If you built a personal conda create --name my_env environment on top of one of these modules, it keeps working. You might want to try a new environment instead, as new python versions and the use of the home directory can improve your experience.

Python#

Choosing a tool#

Ready-made environments#

Building your own environment#

Using uv#

Using pixi#

Using micromamba#

Using your environment#

Registering a Jupyter kernel#

Using an environment in a batch job#

Parallel jobs#

Pinning the Python version#

File system, quota and caches#

Reclaiming space#

Pitfalls#

Do not use pip#

Package channels and licensing#

Troubleshooting#

Legacy: system wide python3 modules#

Do not use `pip`#

Legacy: system wide `python3` modules#