Posts tagged mistral
Data Migration Mistral to Levante
- 04 March 2022
So that not every user or every project has to copy its data from Mistral to Levante, DKRZ will take over this in a coordinated manner. In this way, the bandwidth between the systems can be better utilised.
We will only copy data located in /work - and only for projects that have
resources granted on the Levante system.
Why do I receive .Xauthority file error messages?
- 08 July 2019
- FAQ
When you open a new terminal session on mistral with X forwarding turned on (ssh -X ...),
the .Xauthority file in your home directory gets updated by the xauth program.
This file is used to keep X authentication keys in order to prevent unauthorized
connections to your local display.
Sometimes, the .Xauthority file cannot be updated due to issues with the lustre01 file system, where your home directory is located, on mistral and you might experience an error message like:
Python Matplotlib fails with “QXcbConnection: Could not connect to display”
- 18 September 2018
- FAQ
Matplotlib is useful for interactive 2D plotting and also for batch production of plots inside a job. The default behavior is to do interactive plotting which requires the package to open a window on your display. For this purpose you have to log into mistral with X11 forwarding enabled.
If you run matplotlib in a jobscript where you just want to create files of your plots, you have to tell matplotlib to use a non-interactive backend. See matplotlib’s documentation how to do that and which backends are available. Here is how to select the Agg backend (raster graphics png) inside your script. Add to the top of your imports
How can I avoid core files if my program crashes
- 30 May 2018
- FAQ
Core files can be very helpful when debugging a problem but they also take a long time to get written for large parallel programs. This will limit the core size to zero, i.e. no core files are written:
Note that due to a bug in our current installation of the slurm scheduler, the option
How to use SSHFS to mount remote lustre filesystem over SSH
- 19 June 2017
- FAQ
In order to interact with directories and files located on the lustre filesystem, users can mount the remote filesystem via SSHFS (SSH Filesystem) over a normal ssh connection.
SSHFS is Linux based software that needs to be installed on your local computer. On Ubuntu and Debian based systems it can be installed through apt-get. On Mac OSX you can install SHFS - you will need to download FUSE and SSHFS from the osxfuse site. On Windows you will need to grab the latest win-sshfs package from the google code repository or use an alternative approach like WinSCP.
How to Write a shell alias or function for quick login to a node managed by SLURM
- 19 June 2017
- FAQ
For tasks better run in a dedicated but interactive fashion, it might be advantageous to save the repeating pattern of reserving resources and starting a new associated shell in an alias or function, as explained below.
If you use bash as default shell you can place the following alias definition in your ~/.bashrc file and source this file in the ~/.bash_profile or in the ~/.profile file:
How to Set the default SLURM project account
- 19 June 2017
- FAQ
On Mistral, specification of the project account (via option -A or –account) is necessary to submit a job or make a job allocation, otherwise your request will be rejected. To set the default project account you can use the following SLURM input environment variables
SLURM_ACCOUNT - interpreted by srun command
How can I run a short MPI job using up to 4 nodes?
- 19 June 2017
- FAQ
You can use SLURM Quality of Service (QOS) express by inserting the following line into your job script:
or using the option –qos with the sbatch command:
How can I check my disk space usage?
- 19 June 2017
- FAQ
Your individual disk space usage in HOME and SCRATCH areas as well as project quota in the WORK data space can be checked in DKRZ online portal. The numbers are updated daily.
How can I access my Lustre data from outside DKRZ/ZMAW?
- 19 June 2017
- FAQ
For data transfer you can use either sftp:
or rsync command:
Can I run cron jobs on Mistral?
- 19 June 2017
- FAQ
For system administration reasons users are not allowed to shedule and execute periodic jobs on Mistral using the cron utility. Our recommendation is to use the functionality provided by the workload manager SLURM for this purpose. With the option –begin of the sbatch command you can postpone the execution of your jobs until the specified time. For example, to run a job every day after 12 pm you can use the following job script re-submitting itself at the beginning of the execution:
A variety of different date and time specifications is possible with the –begin option, for example: now+1hour, midnight, noon, teatime, YYYY-MM-DD[Thh:mm:ss], 7AM, 6PM etc. For more details see manual pages of the sbatch command:
Is a FTP client available on mistral?
- 18 June 2017
- FAQ
LFTP is installed on mistral for download and upload of files from/to an external server via File Transfer Protocol (FTP):
The user name for authentication can be provided via option ‘-u’ or ‘–user’, for example: