slk wrappers: SLURM wrapper scripts to simplify StrongLink-tasks#

file version: 04 Feb 2025

current software version: slk_wrappers version 2.0.1

Available Wrapper Scripts#

slk_wrapper_version: print the version of the scripts
slk_wrapper_daily_login_check: start a SLURM job which runs daily and notifies you when your login token is due to expire
slk_wrapper_recall_wait_retrieve (DEPRECATED since v2.0.0): recall files to the cache and start retrieval as SLURM job when the recall is finished; please use instead: start_recall_wrapper.sh and start_retrieve_wrapper.sh
slk_wrapper_wait_retrieve (DEPRECATED since v2.0.0): retrieve files which are currently copied from tape to cache (see in this command’s description for details); please use instead: start_recall_wrapper.sh and start_retrieve_wrapper.sh
start_recall_wrapper.sh: start SLURM job chain which repeatedly starts recall jobs to copy requested files from tape to cache; requires that slk_helpers gfbt ... ran before
start_retrieve_wrapper.sh: start SLURM job chain which repeatedly tries to retrieve/transfer requested files from the cache to you/Lustre/Levante; requires that slk_helpers gfbt ... ran before

slk_wrapper_version#

$ slk_wrapper_version
slk wrapper version: 2.0.1
slk_helpers version: Version 1.13.2
slk version: SCLI Version 3.3.91

Prints the version of the slk wrapper scripts and of the used version of the slk_helpers and slk.

slk_wrapper_daily_login_check#

$ slk_wrapper_daily_login_check

Submit a SLURM job which notifies you via email when your login token is due to expire. This job re-submits itself with one day delay so that the status of your login token is checked once a day. If you wish to stop this again, please cancel the SLURM job with name slk_token_expire_notify via scancel.

slk_wrapper_weekly_verify_job#

wrapper will be removed soon

slk_wrapper_recall_wait_retrieve (DEPRECATED since v2.0.0)#

$ slk_wrapper_recall_wait_retrieve --help
$ slk_wrapper_recall_wait_retrieve <account> <source_path> <destination_path> <suffix_logfile>

* ``--help``: print short help
* ``<account>``: DKRZ project with compute time (e.g. gg0302, ab0995 or mh0287)
* ``<source_path>``: resource path or search id of archived data which should be obtained
* ``<destination_path>``: destination directory on Levante
* ``<suffix_logfile>``: a log file with the name ``logfile_<suffix>.log`` is automatically created

Start a recall job in StrongLink (copy files from tape to cache) and submits a SLURM job which repeatedly submits itself until the recall job is finished. When the recall job is finished, another SLURM job is submitted which retrieves the files from the cache to the Lustre filesystem / Levante.

slk_wrapper_wait_retrieve (DEPRECATED since v2.0.0)#

$ slk_wrapper_wait_retrieve --help
$ slk_wrapper_wait_retrieve <account> <source_path> <destination_path> <job_id> <suffix_logfile>

* ``--help``: print short help
* ``<account>``: DKRZ project with compute time (e.g. gg0302, ab0995 or mh0287)
* ``<source_path>``: resource path or search id of archived data which should be obtained
* ``<destination_path>``: destination directory on Levante
* ``<job_id>``: id of a running StrongLink recall job (not SLURM job id); recall job ids are printed into ``~/.slk/slk-cli.log``
* ``<suffix_logfile>``: a log file with the name ``logfile_<suffix>.log`` is automatically created

Useful when a recall job is already running and you want to retrieve the files from the cache as soon as they are there.

Submits a SLURM job which repeatedly submits itself until the recall job with given job_id is finished. When the recall job is finished, another SLURM job is submitted which retrieves the files from the cache to the Lustre filesystem / Levante.

start_recall_wrapper.sh#

Please see improved retrieval workflow v01

start_retrieve_wrapper.sh#

Please see improved retrieval workflow v01