Changelog: slk_helpers v1.16.5#

Update from slk_helpers v1.13.2 to v1.16.5

Please have a look here for a detailed description of the new features and here for an incremental changelog.

Major changes#

multiple improvements in these commands:
- iscached: pipe into command, accept multiple resource ids, new output options
- exists: accepts a resource id as input
- recall and retrieve: few bug fixes and adaptions
new commands to simplify recalls and retrievals for you:
- init_watchers: a simplified version of gfbt; generate input files for new generation of SLURM recall and retrieval job scripts, called recall watcher and retrieve watcher
- start_watchers: start recall and retrieve watchers
- stop_watchers: end running recall and retrieve watcher jobs
new commands (other than recall/retrieve):
- resource_tapes: prints tape information for multiple resources (resource_tape could not be updated while keeping it downward compatible)
- resource_features (experimental): prints selected details/features of a resource like caching status, size, permissions

init_watchers:
- simplified version of gfbt -wf1 ...
- generates files required by recall and retrieve watchers
start_watchers: start recall and/or retrieve watcher scripts in the current directory
stop_watchers: stop recall and/or retrieve watcher scripts
resource_tapes:
- same information as resource_tape but made for printing tape information of multiple resources
- per line, one resource path is printed together with barcodes of tapes onto which this resource is stored on
- --print-sizes appends size of each file part and the total size of this resource (HPSS split some files into multiple parts)
resource_features:
- accepts multiple resources as input and prints requested details/feeatures on these resources as CSV table: one resource per row and requested features/details as columns
- select which details/features would be printed:
  
  --which-features: print list of possible features
  
  --features, -f: comma separated list of features to print
  
  --full: print all resource features except for detailed tape information which would require additional API calls; please run with --full-with-tapes to include tape information
  
  --full-with-tapes: print all resource features; also those which require additional API calls
- command is experiment and arguments might change in future version without notice
- details here

command gfbt:
- fixed bug in slk_helpers gfbt –gen-search-query which printed debug output
- writes more variables into config.sh when -wf1 / –retrieval-workflow-1 is set
commands recall and retieve:
- split HPSS files which are only partially recall are skipped
- retrieve got --slurm as short version of run-as-slurm-job-with-account
command exists got new argument –resource-id: provided resource considered as resource id instead of resource path
command iscached was extended:
- accepts multiple resource ids as input when –resource-ids is set
- resources (path or id) can be piped into the command (e.g.: cat file_list.txt | slk_helpers iscached …)
- prints resource ids instead of resource paths in the verbose output when –print-resource-ids is set
fixed bug in representation of POSIX permissions by list_clone_* commands

list_search and list_clone_search may list already deleted files (not checked for performance reasons)