Changelog: slk_helpers v1.12.10#

Update from slk_helpers v1.10.2 to 1.12.10

see here for changes from slk_helpers v1.9.7 to v1.10.2

see here for changes from slk_helpers v1.9.5 to v1.9.9

see here for changes from slk_helpers v1.9.3 to v1.9.5

see here for changes from slk_helpers v1.9.0 to v1.9.3

see here for changes from slk_helpers v1.8.10 to v1.9.0

see here for changes from slk_helpers v1.8.2 to v1.8.10

see here for changes from slk_helpers v1.7.4 to v1.8.2

see here for changes from slk_helpers v1.7.1 to v1.7.4

see here for changes from slk_helpers v1.6.0 to v1.7.1

see here for changes from slk_helpers v1.5.8 to v1.6.0

see here for changes from slk_helpers v1.5.7 to v1.5.7

see here for changes from slk_helpers v1.2.x to v1.5.7

Please have a look here for a detailed description of the new features and here for an incremental changelog.

Major changes#

  • new commands (partly hidden):

    • list_clone_file: print information on provided file

    • list_clone_search: list search results and print additional information to all listed files

    • resource_tape: print on which tape(s) a resource is stored on; can be used if gfbt / group_files_by_tape does not work due to high system load of StrongLink

    • search_immediately: submit a search query and return the search id immediately without waiting for the search to finish

    • search_status: print the status of a search; search id has to be provided as input

    • tape_library: print tape library in which a tape is stored (slk_helpers tape_library (<tape_id>|--tape-barcode <tape_barcode>))

    • tnsr: short form of total_number_search_results

  • basic logging into ~/.slk/slk-cli.log as slk also does

  • if files are in an unclear caching state, commands like is iscached and is_on_tape will inform the user

  • if a search has a timeout or returns an unexpected search status, commands which run searches or print search results will react more specifically

  • fixed a bug which caused tape_status and group_files_by_tape / gfbt to exit with an error when a tape was blocked by a system-job

  • multiple commands received a flag --json to print output as JSON

Note

unclear caching state should only occur while a file is copied from tape to HSM-cache or when there is a connection issue between StrongLink and HSM-Cache. Please contact support@dkrz.de when this happens.

Details on new commands#

  • output format for list_clone_file and list_clone_search; 10 columns:

    • col 1: permissions and storage location

    • col 2: uid / user id

    • col 3: gid / group id

    • col 4: size in byte

    • col 5: mtime of the file (time stamp of “last modification of the file prior to the archival”)

    • col 6: time stamp of first file version in StrongLink

    • col 7: stamp of current file version in StrongLink

    • col 8: time stamp of last StrongLink-internal copy process of this file (e.g. last recall)

    • col 9: tape id if file is stored on one tape

    • col 10: full path

  • exit codes of search_status:

    • exit code 0: search ended successfully

    • exit code 1: running or failed (incl. search timeout)

    • exit code 2: any error returned by StrongLink which is not captured by 1 or 3

    • exit code 3: timeout while communicating with StrongLink

All fixes and new features of existing commands#

  • generally improved error and warning messages

  • basic logging into ~/.slk/slk-cli.log as slk also does

  • iscached: * if file(s) in an unclear caching state: iscached will inform the user via a warning (if -v or -vv is not set) * if -v or -vv is set, the files in unclear caching state will be listed accordingly

  • is_on_tape: * if file(s) in an unclear caching state: is_on_tape will inform the user via a warning (if -v or -vv is not set)

  • gen_search_query: minor fix in generation of search query

  • gfbt / group_files_by_tape:

    • fixed a bug which caused that a wrong tape id / barcode was used when a file was migrated from an old HPSS tape to a new tape

    • properly deal with search timeouts

    • improvements in the output and program flow when search does not finish or is not successful

    • does not exit with an error anymore if tape is in BLOCKED state

  • print_rcrs updates:

    • prints tape id and barcode

    • got the parameter --json to print output as JSON

  • result_verify_job:

    • does not throw error anymore but warning when numbers of submitted and checked files do not agree

    • throws no error anymore when target namespace does not exist

    • new parameter --json

    • remove resources which do not exist anymore from report

    • captures a StrongLink bug when a file had the same name as its parent namespace

  • submit_verify_job
    • new parameter:

      • --end-on-page

      • --results-per-page

      • --json

    • updated restart information when command timeouts

    • updated estimation of number of verify jobs to submit

Known issues#

  • list_search and list_clone_search may list already deleted files (not checked for performance reasons)

HSM: check storage location of files   HSM module cleanup April 2024