Data Migration Mistral to Levante¶
So that not every user or every project has to copy its data from Mistral to Levante, DKRZ will take over this in a coordinated manner. In this way, the bandwidth between the systems can be better utilised.
What data will be copied?¶
We will only copy data located in
/work - and only for projects that have
resources granted on the Levante system.
Personal data in
/home should be copied by each user. This is very little
data in total and it gives the chance to start with a new, clean environment.
Please note that Levante is not Mistral and therefore adjustments in
~/.ssh are useful.
How should I copy my home from Mistral to Levante?¶
You should take the opportunity to clean up your home directory and archive or copy only data you actually want to keep.
One way to copy your entire home directory would be by using the
rsync tool. Log into Mistral and change into the top level of your
cd rsync -av ./ levante.dkrz.de:mistral_home
This will recursively copy all files from your current directory into
mistral_home on Levante.
What if I do not want my data to be copied?¶
This is fine. Please talk to your project admin and let us know which data should not be copied.
Where will the copied data be located?¶
All copied data will be placed in
<project> is the same project acronym used on Mistral and Levante.
When will the migration happen?¶
This depends on the current number and size of files in your project’s
A detailed schedule is given as
After a project’s data had been copied and the last sync happened, we will create a file of the following format
Then you will know that all of the project’s data up to this point in time had been copied from Mistral to Levante.
What happens if data productions runs in parallel on Mistral and Levante?¶
The migration plan (see pdf) indicates a period where we will try to synchronise the data between both systems. After that period you are responsible for managing the consistency of data.
How should I synchronise data between Mistral and Levante after migration?¶
For small amount of data (less than 5 TiB), one can simply use
You first have to
mvthe already copied data on Levante from directory
from_Mistralto the new place you like, because
from_Mistralto a new location - use
If you want to synchronise more than 5 TiB (or a very high number of files), please contact email@example.com for guidance.