How to keep disk usage under control#
Below, we provide some tips how to monitor your disk usage on Levante and find large files and directories in case you are nearing or exceeding your disk storage quota.
You can check your disk quota on Levante with the lfsquota.sh
wrapper script stored in the /sw/bin directory. To display disk
usage and limits for the personal directories assigned to your user
account (i.e. /home and /scratch), use:
$ /sw/bin/lfsquota.sh -u $USER
If you are only interested in /home or /scratch disk usage, you can
use the lfs quota command directly, e.g.:
$ lfsquota -h -u $USER /home
$ lfsquota -h -u $USER /scratch
For total disk usage within a project, you can call the
lfsquota.sh script with the -p option and the project account:
$ /sw/bin/lfsquota.sh -p <prj-account>
If you are not sure where the storage space has gone, you can identify
the size and location of the largest files and directories using Linux
commands like du, find and sort. The du command allows
you to recursively estimate and summarize the disk usage. For a better
overview, it is often advisable to limit the directory depth to one
and to pipe the output of the du command to the sort commnad
to display the files and directories based on their size in descending
order:
$ du -a -d1 | sort -n -r
To show the file and directory sizes in human readable form, the
-h option can additionally be used:
$ du -a -d1 -h | sort -h -r
The find command can be used to locate and display files based on
their size. For example, to find files larger than 500MiB in the
specified directory, you can use:
$ find /path/to/search -type f -size +500M | xargs ls -lh
Detailed commands descriptions can be found in the corresonding manual pages:
$ man du
$ man sort
$ man find
Once you have identified large files or directories, you can delete unnecessary data and move important data elsewhere (usually to the /work storage area of a project you are participating in) or store them in the tape archive.
Sufficient storage space in the home directory is required to successfully operate multiple software packages. This is because your home directory is used for various small files (like configuration files, log files, lock files, named pipes, unix sockets etc.) during execution of programs. If such files cannot be saved, various seemingly unrelated error messages may result from an exhausted home quota. Regularly monitoring the disk space usage in your home directory is therefore crucial for the smooth use of the Levante HPC system. Due to the relatively small home quota (30GiB), we recommend to move the location of your own Conda environments to the work project space, as outlined here.