Switching: pftp to slk#
file version: 24 Jan 2023
current slk version: 3.3.81
Note
This page is deprecated and will be removed soon.
Some common pftp workflows and their counterparts using slk are listed below to simplify switching from pftp to slk. A general description of slk is given in StrongLink Command Line Interface (slk) and additional example calls are provided in slk Usage Examples.
Nomenclature#
We use some abbreviated file and folder names in the code examples for simpler readability. These are
/lustre_dir: a directory path on the lustre file system (e.g/work/bm0146/k204221)/tape_dir: a directory path in the tape archive (e.g today/hpss/arch/bm0146/k204221)file.nc: a netCDF file
Login#
pftp#
$ pftp
...
ftp>
slk#
Login to another shell like to the ftp shell via pftp is not necessary. Instead, the user has to generate a login token every 30 days. This procedure is comparable to the Kerberos authentication. The slk commands are run on the normal Linux shell.
$ slk login
Username: XYZ
Password:
Login Successful
Note
slk stores the login token in the home directory of each user (~/.slk/config.json). By default, this file can only be accessed by the respective user (default permissions: -rw-------/600). However, users should be careful when doing things like chmod 755 * in their home directory so that other users cannot access the token. If you assume that your slk login token has been compromized please contact support@dkrz.de .
Archival of a single file#
pftp#
$ cd /lustre_dir
$ pftp
# possibly `mkdir tape_dir`
ftp> cd /tape_dir
ftp> put file.nc
ftp> quit
slk#
$ slk archive /lustre_dir/file.nc /tape_dir
Archival of a whole directory#
pftp#
$ cd /lustre_dir
$ pftp
# possibly `mkdir /tape_dir`
ftp> cd /tape_dir
ftp> prompt
ftp> mput *
ftp> quit
slk#
$ slk archive -R /lustre_dir /tape_dir
All files located in /lustre_dir are archived.
Note
You need to add * to the source path as would be necessary for cp (cp /lustre_dir/* /tape_dir/`basename /lustre_dir`) in order to archive only the files directly location in /lustre_dir. If you have sub-folders 1900 until 1999 (one output folder per model year) and you only want to archive the 1980s then you simply do slk archive /lustre_dir/198? /tape_dir.
Retrieval of a single file#
pftp#
$ cd /lustre_dir
$ pftp
ftp> cd /tape_dir
ftp> get file.nc
ftp> quit
slk#
$ slk retrieve /tape_dir/file.nc /lustre_dir
Retrieval of a whole directory#
pftp#
$ cd /lustre_dir
$ pftp
ftp> cd /tape_dir
ftp> prompt
ftp> mget *
ftp> quit
slk#
$ slk retrieve -R /tape_dir/ /lustre_dir
All files located in /tape_dir and its subdirectories are retrieved.
Note
The notation of the source directory in slk retrieve is similar to rsync: slk retrieve /tape_dir/ /lustre_dir is different to slk retrieve /tape_dir /lustre_dir (no slash). Only the folders content is retrieved when / is appended (like cp /tape_dir/* /lustre_dir). The source folder itself (with its content) is retrieved when / is not appended (like cp /tape_dir/* /lustre_dir/`basename /tape_dir`).
Listing the content of a folder in the archive#
pftp#
$ pftp
ftp> cd /tape_dir
ftp> ls
200 PORT command successful.
150 Opening ASCII mode data connection for file list.
a.nc
b.nc
c.nc
d.nc
c.nc
226 Transfer complete.
312 bytes received in 0.0054 seconds (0.056 MBytes/sec)
ftp> quit
slk#
$ slk list /tape_dir
-rw-r--r--- k204221 bm0146 4 04 Feb 2021 10:32 a.nc
-rw-r--r--- k204221 bm0146 6.5K 04 Feb 2021 10:32 b.nc
-rw-r--r--- k204221 bm0146 10 07 Feb 2021 04:17 c.nc
-rw-r--r--- k204221 bm0146 16.1K 01 Feb 2021 14:13 d.nc
-rw-r--r--- k204221 bm0146 1.2M 04 Feb 2021 10:32 e.nc