Python environment locations#
Kernels are based on python environments created with conda,
virtualenv or other package manager. In some cases, the size of the
environment can tremendously grow depending on the installed packages.
The default location for python files is the $HOME directory. In
this case, it will quickly fill your quota. In order to avoid this, we
suggest that you create/store python files in other directories of the
filesystem on Levante.
Possible locations#
The following are two alternative locations where you can create your Python environment:
WORK (depends on the project) you can use this option if you are working on the same workflow and you want to share the same python env among the project members
SCRATCH (not recommended for long term use) for testing new packages/releases. Files are deleted 14 days since the last file access.
Example#
In this example, we will show you how to create conda environment in different location (SCRATCH).
$ module load python3
$ conda create --prefix /scratch/[a,b,g,k,m,u]/<userid>/<new_your_conda_env>
$ source activate /scratch/[a,b,g,k,m,u]/<userid>/<new_conda_env>
That’s it, you can now install other packages.
Note
For different locations, you just need to change the prefix.
Automate the prefix#
Warning
To be used with caution. Further information on .condarc can be found here.
If you would like to set up the same prefix for future use, you need a
.condarc on your home directory with the following content:
envs_dirs:
- /scratch/[a,b,g,k,m,u]/<userid>/
Now, you can create a conda env with the standard command:
$ conda create -n my_conda_env
The new environment will be automatically created in
/scratch/[a,b,g,k,m,u]/<userid>/.
Note
Creating kernels for jupyterhub based on these conda env is possible. The location for the packages is automatically set when you run ipykernel install …