Changelog: slk v3.3.67#

Changes from slk v3.3.21 to v3.3.67

Changlog updated on 2022-10-21 (changes in slk chmod extended)

Highlight Fixes#

  • slk archive, partly archived files (slk archive cancelled during archival): During the archival process of a file, this file’s size is printed as 0 byte. If the archival process is canceled while a file has not been fully copied, this file will remain as 0 byte file in StrongLink. In the past, partly archived files could be retrieved; e.g. partly archived tar balls could be retrieved and extracted to rescue a few of the packed files.

  • slk retrieve got options to skip -s and overwrite -f existing files

  • slk archive got a double verbose mode -vv

  • slk list: pagination removed; list individual files; warning: might take a very long time when many items should be listed

  • slk move: overwrite existing target file by default

  • slk search is activated

  • slk chmod: skips file(s) and returns exit code 1 if slk chmod set the same permissions as the target file(s) already has(/ve)

  • slk group: skips file(s) and returns exit code 1 if slk group set the same group as the target file(s) already has(/ve)

All Fixes#

  • slk in general:

    • If the StrongLink instance is not accessible (server responds but no access to StrongLink; e.g. maintenance), this message is printed: ERROR: Connection Unavailable

    • If slk receives no reply at all, this message is printed by slk login: Host not reachable: archive.dkrz.de. All other commands print ERROR: Unhandled error occurred, please check logs.

    • The process id and hostname are prepended to each log message (location of log file: ~/.slk/slk-cli.log)

    • Removal of confusing verbose output from the log.

    • // in file paths are sustituted to /

    • slk bad_input return exit code 1

  • slk archive

    • no archival status is printed by default

    • -v prints a one-line archival status

    • -vv prints out the name of each processed file per line and appends the status: SUCCESSFUL, SKIPPED or FAILED

    • During the archival process of a file, this file’s size is printed as 0 byte. If the archival process is canceled while a file has not been fully copied, this file will remain as 0 byte file in StrongLink. In the past, partly archived files could be retrieved; e.g. partly archived tar balls could be retrieved and extracted to rescue a few of the packed files.

    • does not hang anymore when one or more empty files are archived

  • slk chmod:

    • can be applied on more than one file at once (i.e. slk delete /arch/ab1234/file1 /arch/ab1234/file2)

    • fixed a bug related to recursive chmod on a nested folder hierarchy

    • improved speed

    • skips file(s) and returns exit code 1 if slk chmod set the same permissions as the target file(s) already has(/ve) => e.g. slk chmod 770 on a file with rwxrwx--- (==``770``) will result in exit code 1

  • slk delete:

    • can be applied on more than one file at once (i.e. slk delete /arch/ab1234/file1 /arch/ab1234/file2)

  • slk group:

    • improved speed

    • skips file(s) and returns exit code 1 if slk group set the same group as the target file(s) already has(/ve) => e.g. slk group bm0146 on a file with group bm0146 will result in exit code 1

  • slk list

    • pagination mode is removed (all results are printed at once to the command line)

    • automatically returns to the command line (no key has to be pressed)

    • can be applied on individual files now

    • user and group names longer than 10 characters are not cropped anymore

    • the column width of the output is not fixed anymore but variable

    • a size of 0 byte is printed for each namespace/folder; this simplifies parsing the output of slk list

    • not only the creation date but also the time (hh:mm) is printed

    • does not hang anymore when a non-existing search id is provided but prints The search id could not be found

    • when a search with search id 12345 does not find any results, then slk list 12345 will print ERROR: No resources found for given search id: 12345

    • nothing is printed when the content of a folder is listed for which the user has no read permissions; this message is printed to the log: ERROR Get namespace children error: {"errors":[{"status":"403","detail":"You do not have permission to access this namespace.","title":"Forbidden"}],"suppressed":[],"localizedMessage":null}

  • slk move: changed behaviour when a target file does already exist:

    • new default: overwrite existing target file

    • -i: prompt user (interactive mode) or do not overwrite (batch mode; exit code is 1)

  • slk retrieve

    • -f overwrite target file if it exists (otherwise create a DUPLICATE file)

    • -s skip retrieving file if it exists in target location (otherwise create a DUPLICATE file)

    • -ns restore the full HSM path; useful when we performed a search, which found several files of the same name, and want to retrieve the search results

  • slk search is available

  • slk tag

    • can be applied on individual files

    • names of metadata fields are case sensitive; Setting the metadata field Title of the schema netcdf requires slk tag FILE netcdf.Title="blub". In the past, this would have worked: slk tag FILE netcdf.title="blub" (lowercase t).

    • -display prints the metadata of files; currently, does not work with metadata of netCDF files because of on special field

New Bugs#

  • slk list wildcard evaluation does not work for namespaces anymore but only for files

  • slk tag -display does not work on netCDF files because they have a special metadata field which slk tag -display cannot handle

  • slk list SEARCH_ID collects all search results, first, and, then, prints them. This might take a while if many search results are found. Alternatively, you can run slk_helpers list_search on the same SEARCH_ID which will print search results continuously.

How big are my files?   JupyterDash on Jupyterhub @ DKRZ