Posts in Jupyterhub
JupyterDash on Jupyterhub @ DKRZ
- 24 October 2022
This content is based on this notebook.
The jupyter-dash package makes it easy to develop Plotly Dash apps
from the Jupyter Notebook and JupyterLab.
How to install R packages in different locations?
- 25 October 2021
The default location for R packages is not writable and you can not install new packages. On demand we install new packages system-wide and for all users. However, it possible to install packages in different locations than root and here are the steps:
create a directory in $HOME e.g. ~/R/libs
How to install jupyter kernel for Matlab
- 10 June 2021
In this tutorial, I will describe i) the steps to create a kernel for Matlab and ii) get the matlab_kernel working in Jupyterhub on Levante.
conda environment with python 3.9
How to re-enable the deprecated python kernels?
- 16 May 2021
Within the maintenance of Monday, May 15th, we will perform updates in our python installations (please see the details here).
Since the jupyterhub kernels are based on modules, the deprecated kernels will no longer be available as default kernels in jupyter notebooks/labs.
Requested MovieWriter (ffmpeg) not available
- 06 May 2021
Requested MovieWriter (ffmpeg) not available
conda env with ffmpeg and ipykernel
How to containerIze your jupyter kernel?
- 04 May 2021
Containers are not supported on Levante at this point.
We have seen in this blog post how to encapsulate a jupyter notebook (server) in a singularity container . In this tutorial, I am going to describe how you can run a jupyter kernel in a container and make it available in the jupyter*.
Create a kernel from your own Julia installation
- 23 March 2021
We already provide a kernel for Julia based on the module julia/1.7.0.
In order to use it, you only need to install ÌJulia:
Python environment locations
- 04 March 2021
Kernels are based on python environments created with conda,
virtualenv or other package manager. In some cases, the size of the
environment can tremendously grow depending on the installed packages.
The default location for python files is the $HOME directory. In
this case, it will quickly fill your quota. In order to avoid this, we
suggest that you create/store python files in other directories of the
filesystem on Levante.
The following are two alternative locations where you can create your Python environment:
Dask jobqueue on Levante
- 16 November 2020
According to the official Web site, Dask jobqueue can be used to
deploy Dask on job queuing systems like PBS, Slurm, MOAB, SGE,
LSF, and HTCondor. Since the queuing system on Levante is Slurm, we are
going to show how to start a Dask cluster there. The idea is simple as
described here. The difference is that the workers can be distributed
through multiple nodes from the same partition. Using Dask jobqueue you can launch
Dask cluster/workers as a Slurm jobs. In this case, Jupyterhub will play an interface role and the Dask
can use more than the allocated resources to your jupyterhub session
(profiles).
Load the required clients
Enable NCL Kernel in Jupyterhub
- 05 November 2020
Can’t use NCL (Python) as kernel in Jupyter
This tutorial won’t work
FileNotFoundError: [Errno 2] No such file or directory
- 07 October 2020
- 25 September 2020
- 18 September 2020
Jupyterhub log file
Each Jupyter notebook is running as a SLUM job on Levante. By default,
stdout and stderr of the SLURM batch job that is spawned by
Jupyterhub is written to your HOME directory on the HPC system. In
order to make it simple to locate the log file:
if you use the preset options form: the log file is named
jupyterhub_slurmspawner_preset_<id>.log.
Simple Dask clusters in Jupyterhub
There are multiple ways to create a dask cluster, the following is only an example. Please consult the official documentation. The Dask library is installed and can be found in any of the python3 kernels in jupyterhub. Of course, you can use your own python environment.
The simplest way to create a Dask cluster is to use the distributed module:
