Accessing Mistral’s GPU Nodes

../../../_images/Remote_Rendering_2016_preview.png

At the DKRZ, we are using TurboVNC / VirtualGL for remote 3D rendering on the GPU servers. These systems contain powerful graphics boards, that render to a virtual desktop. This desktop is then forwarded to the user via TurboVNC. On the client side, a vnc client, such as TurboVNC is needed in order to connect to the system. TurboVNC is available for Windows, Linux and other Unix platforms.

The instructions below work on most Linux and macOS installations. The use under Windows 10 with the Windows Subsystem for Linux (WSL) is also possible. For other windows versions, see VNC from Windows computers without Windows Subsystem for Linux (WSL).

Script mode

Script mode automates the manual process described below. Additional security and privacy is ensured by sending all communication with the server through an encrypted ssh tunnel. The VNC server is not directly exposed to the network in this case. The script works on most Linux and macOS installations. Windows 10 with the Windows Subsystem for Linux (WSL) is also possible. For other windows versions, see VNC from Windows computers without Windows Subsystem for Linux (WSL).

Download the script here

and add execute mode (Linux and macOS).

chmod a+x start-vnc

The most simple way to connect to a visualization node is by providing an account code of a DKRZ project you belong to.

./start-vnc -A uh9999

This assumes that your username on mistral is the same as your local username and that vnc client software is named vncviewer and can be found in your search path (on MPI/CEN-IT systems try module load turbovnc to use turbovnc as vncviewer). The script can be further configured by providing additional command line options or by directly editing the script.

Additional examples

./start-vnc -A uh9999 -C m6000 -v /opt/TurboVNC/bin/vncviewer

Use the TurboVNC client and request a m6000 GPU.

wsl bash start-vnc -A uh9999 -u u202020 -v /mnt/c/Program\ Files/TurboVNC/vncviewer.exe

Use WSL on Windows 10 with cmd or Powershell to connect with the TurboVNC client. Note that WSL accesses Windows drive C: through /mnt/c.

./start-vnc -A uh9999 -u u202020 -v /opt/TurboVNC/bin/vncviewer -s "-securitytypes none -geometry 2560x1600"

The -securitytypes none option for the server solves a problem on macOS where TurboVNC fails with the error message “javax.net.ssl.SSLHandshakeException: No appropriate protocol”. Note that because of global ssh encryption, this creates no security vulnerability. You can also add additional server options like -geometry in this case which sets the initial size of your desktop.

Graphics quality / speed

If you are not connected via high-speed internet, you can reduce the image quality in VNC to improve your life. On a mac go to the vnc settings via the drop-down in the top of your screen

../../../_images/getsettings.png

and set the Picture quality to low (vnc tends to assume a perfect connection when running via an ssh tunnel - automatic quality fails then).

../../../_images/picturequality.png

Manual mode

The reservation has to be done by hand using a console on one of Mistrals login nodes. Furthermore, the VNC server needs to be started by hand. You should also as cleanup your session after your work is done.

To access Mistral, simply ssh into the machine.

somewhere:~> ssh user_name@mistral.dkrz.de

Reservation/allocation of a GPU node:

On Mistral, we have to allocate a GPU node. This is done using the SLURM command “salloc”, in which you have to provide your account group (“-A <your project id>”),  the number of nodes (“-N 1”), the maximum number of parallel tasks (“-n 24”), as well as the partition type (“-p gpu”). The option “-t hh:mm:ss” sets the duration of the session. Currently, the maximum time allowed is 12 hours and is set automatically. If 12 hours are not sufficient for your specific purposes please contact our Help Desk.

More information on SLURM and salloc can be found in our SLURM Introduction.

After login to Mistral, you can allocate a GPU node and automatically ssh into the node reserved.

mistral:~> salloc -N 1 -n 12 --mem=128000 -p gpu -A <project> -t12:00:00 -- /bin/bash -c 'ssh -X $SLURM_JOB_NODELIST'
salloc: Granted job allocation 284896

user_name@mg100's password:

Shared/exclusive usage:

The option “-n 12” requests SLURM to allocate 12 physical cores. A total of 24 physical cores is available in the phase-1 nodes (mg100-mg111), respectively 36 phyiscal cores in the phase-2 nodes (mg200-mg208). The fraction of the node’s main memory you could use is limited according to fraction of the physical cores you request, but the default memory reserved per core is only 1.25 GB. With the option “–mem=MB” you could request a larger fraction of the memory, i.e. with “mem=128000” 128 GB. By requesting only 12 cores on phase-1 nodes, you could use up to half of the the total memory and the GPU node can be shared, i.e. another user might work on the same node at the same time. If you need more resources, i.e. the full memory of the node, you can allocate all physical cores or simply by using the “–exclusive” option.

mistral:~> salloc --exclusive -N 1 -p gpu -A <project> -t12:00:00 -- /bin/bash -c 'ssh -X $SLURM_JOB_NODELIST'
salloc: Granted job allocation 284896

user_name@mg100's password:

As soon as you have made your allocation, the project given in the salloc command will be charged according to the resources allocated for the duration of your session.

Starting a VNC-Server

Now you have to start a VNC server to connect to the virtual desktop. Otherwise it is not possible to use any of the visualization applications!

mg100:~> /opt/TurboVNC/bin/vncserver -geometry 1920x1200
Desktop 'TurboVNC: mg100:1 (user_name)' started on display mg100:1

At the first time, you have to supply a password, that you later need to access your VNC session remotely. Please choose a new password for this service and NOT your LDAP authentication.

Starting a VNC-Viewer on the client:

If everything is set, start a vncviewer in the console on your workstation (linux), or using a windows client application such as TurboVNC.

somewhere:~> vncviewer mg100.dkrz.de:1

Now a window opens, showing you the virtualized X11 session running on the GPU node. The system runs a Gnome desktop environment. Here you need to open a console window to start your visualization application.