Data Migration Mistral to Levante#

So that not every user or every project has to copy its data from Mistral to Levante, DKRZ will take over this in a coordinated manner. In this way, the bandwidth between the systems can be better utilised.

What data will be copied?#

We will only copy data located in /work - and only for projects that have resources granted on the Levante system.

Personal data in /home should be copied by each user. This is very little data in total and it gives the chance to start with a new, clean environment. Please note that Levante is not Mistral and therefore adjustments in e.g. ~/.ssh are useful.

How should I copy my home from Mistral to Levante?#

You should take the opportunity to clean up your home directory and archive or copy only data you actually want to keep.

One way to copy your entire home directory would be by using the rsync tool. Log into Mistral and change into the top level of your home directory.

cd
rsync -av ./ levante.dkrz.de:mistral_home

This will recursively copy all files from your current directory into the directory mistral_home on Levante.

What if I do not want my data to be copied?#

This is fine. Please talk to your project admin and let us know which data should not be copied.

Where will the copied data be located?#

All copied data will be placed in

/work/<project>/from_Mistral/<project>

Where <project> is the same project acronym used on Mistral and Levante.

When will the migration happen?#

This depends on the current number and size of files in your project’s /work.

A detailed schedule is given as download PDF

After a project’s data had been copied and the last sync happened, we will create a file of the following format

/work/<project>/from_Mistral/last-sync-YYYY-MM-DD

Then you will know that all of the project’s data up to this point in time had been copied from Mistral to Levante.

What happens if data productions runs in parallel on Mistral and Levante?#

The migration plan (see pdf) indicates a period where we will try to synchronise the data between both systems. After that period you are responsible for managing the consistency of data.

How should I synchronise data between Mistral and Levante after migration?#

For small amount of data (less than 5 TiB), one can simply use rsync -av.

Caution:

  • You first have to mv the already copied data on Levante from directory from_Mistral to the new place you like, because from_Mistral is read-only!

  • Do not cp data from from_Mistral to a new location - use mv.

If you want to synchronise more than 5 TiB (or a very high number of files), please contact support@dkrz.de for guidance.

Bus error in jobs   libGL.so.1 missing
On this page