Warning
Deprecated, currently, singularity containers are not supported on Levante.
How to containerIze your jupyter kernel?#
We have seen in this blog post how to encapsulate a jupyter notebook (server) in a singularity container . In this tutorial, I am going to describe how you can run a jupyter kernel in a container and make it available in the jupyter*.
Possible use case for this is to install a supported PyTorch
version
and work with jupyter notebooks.
Requirements#
singularity image supported on Levante
software packages
ipykernel
kernel.json
start-singularity-kernel.sh
How it works?#
The concept is simple, usually, software packages are located in
conda
/venv
environements somwhere in Levante’s file system. The
idea is to allow working with packages from a singularity container.
Pull the image#
There are lot of a predefined images with installed software packages. You don’t need to build it from scratch. These images must be supported on Levante.
In this tutorial, I am using this image with the following software packages:
jupyter
jupyterlab
matplotlib
pandas
ipykernel
xarray
intake-esm
intake-xarray
You need to download this image to your (default) HOME
directory on
MIstral.
# load singularity module
module load singularity
# pull the image
singularity pull library://lev-hpc/jupyter/jupyter-kernel:latest
Create start-singularity-kernel.sh#
touch start-singularity-kernel.sh
chmod +x start-singularity-kernel.sh
The content should look like this:
#!/bin/bash
source /etc/profile
module purge
module load singularity
singularity exec --cleanenv /full/path/to/jupyter-kernel_latest.sif python -m ipykernel_launcher -f "$1"
Create kernel.json#
You need a kernel configuration file:
module load python3
python -m ipykernel install --user --name singularity-kernel --display-name="singularity-kernel"
This file must be modified to enable the connection to the singularity
container. My favorite method is via a bash script (let’s name it
start-singularity-kernel.sh
).
{
"argv": [
"/path/to/start-kernel-singularity.sh",
"{connection_file}"
],
"display_name": "singularity-kernel",
"language": "python"
}
Without a bash script#
If you like to write all modifications in kernel.json:
{
"argv": [
"singularity",
"exec",
"--cleanenv",
"/path/to/jupyter-kernel_latest.sif",
"python",
"-m",
"ipykernel",
"-f",
"{connection_file}"
],
"language": "python",
"display_name": "singularity-kernel"
}