Changelog: slk_helpers v1.10.2#

TODO from which version?

Update from slk_helpers v1.9.7 to v1.10.2

see here for changes from slk_helpers v1.9.5 to v1.9.9

see here for changes from slk_helpers v1.9.3 to v1.9.5

see here for changes from slk_helpers v1.9.0 to v1.9.3

see here for changes from slk_helpers v1.8.10 to v1.9.0

see here for changes from slk_helpers v1.8.2 to v1.8.10

see here for changes from slk_helpers v1.7.4 to v1.8.2

see here for changes from slk_helpers v1.7.1 to v1.7.4

see here for changes from slk_helpers v1.6.0 to v1.7.1

see here for changes from slk_helpers v1.5.8 to v1.6.0

see here for changes from slk_helpers v1.5.7 to v1.5.7

see here for changes from slk_helpers v1.2.x to v1.5.7

Please have a look here for a detailed description of the new features and here for an incremental changelog.

Major changes#

  • command size can calculate the size of a namespace incl. its content when -R is set

  • slk_helpers allow to submit verify jobs (command submit_verify_job) and collect their results (command results_verify_job); a verify job compares the actual size of cached files with the file size stored in the StrongLink metadata database

  • timeout errors of StrongLink are properly cached and the slk_helpers re-try all operations multiple times when timeouts occur

Changes that might break established workflows#

  • new job states (unclear whether tthey appear in your workflows): BLOCKED, PAUSED, STOPPED, WAITING, OTHER

  • access constraints for requesting job information; slk_helpers job_status does not work for any job id anymore (details below)

  • command size returns 0 as size when namespace is targeted and -R is not set; previously, an error was returned

All fixes and new features#

  • slk_helpers allow to run verify jobs which check whether archived files are smaller than expected

  • important new commands:
    • is_admin_session: Check if the use is currently logged in as normal user or admin user; mainly useful for admin users working on multiple accounts

    • job_report: fetch raw verify job report; please use result_verify_job instead if possible

    • print_rcrs: print size and checksums of file parts; some HPSS files are stored as two parts on two tapes; mainly useful for users who retrieve large amounts of old data

    • result_verify_job: list relevant errors found by a verify job

    • search_immediately (hidden): Creates search and returns search id immediately, even if search is not finished; less error checks than in slk search => strange error message possible

    • search_incomplete: Prints whether the search is incomplete (still running)

    • search_successful: Prints whether the search was successful

    • submit_verify_job: run a verify job for a provided set of files

    • total_number_search_results (hidden): print total number of search results of a search id; results which are not visible to the current user are counted as well

  • catch all HTTP status code >= 400 everytime an HTTP request is send

  • timeout errors should always cause an exit code of 3

  • new job states: BLOCKED, PAUSED, STOPPED, WAITING, OTHER

  • access constraints for requesting job information; this affects slk_helpers job_status and job_report; non-admin users may only access:
    • VERIFY jobs of the current user or

    • COPY jobs, which do retrievals/recalls and were started by slk

    • list number of errors and checked files (--number-errors and --number-sources)

  • extended command size:
    • new parameter: -R / --recursive for requesting the size of the content of folders recursively

    • new parameter: --pad-spaces-left for space padding to the left in order to align file/namespaces sizes when the command is called multiple times

    • new parameters: -v and -vv for verbose and double-verbose mode; only reasonable in combination with -R

    • returns 0 as size when namespace is targeted and -R is not set

    • updated textual output

Known issues#

  • list_search may list already deleted files (not checked for performance reasons)