Storage Access#

Web-based Browser#

Access to the S3 Object Storage is available via web browser by logging into the web-based interface.

Once logged in, navigation through buckets, personal sub-folders, and shared sub-folders can be done using the Object Browser. Data can be stored, previewed, deleted, and downloaded using the web-based interface.

While access to the object storage is provided through the web-based interface, integration into scripts and computing environments is enabled by leveraging the API.

Access via Command Line#

To access the S3 storage via the command line, the MinIO client (CLI command mc) can be used.

On Levante, the MinIO client is pre-installed as the minio-client module. The module can be loaded with the following command:

module load minio-client

Note: The minio-client module should not be confused with the Midnight Commander module (module name mc).

For installation of mc on a personal machine, the download and quickstart guide can be found here.

First you need to generate your access key pair, by using your LDAP credentials.

mc idp ldap accesskey create-with-login --name <USER_ID> --description <DESCRIPTION> https://s3.dkrz.cloud

Note: Access key pair data (<ACCESS_KEY> + <SECRET_KEY>) should be noted, as the <SECRET_KEY> cannot be displayed again for security reasons.

Additionally, it’s good practice to assign a name to each key pair for identification in the future, particularly if separate keys are used across different machine.

To access the storage system, create an <ALIAS> using the previously generated key pair (<ACCESS_KEY> and <SECRET_KEY>):

mc alias set <ALIAS> https://s3.dkrz.cloud <ACCESS_KEY> <SECRET_KEY>

Replace:

  • <ALIAS> with a suitable name, such as S3DKRZ.

  • <ACCESS_KEY> with the access key from the key pair.

  • <SECRET_KEY> with the secret key from the key pair.

Once the alias is set, it can be used to access a <BUCKET>. For instance, to list the content of <BUCKET>:

mc ls <ALIAS>/<BUCKET>

To list the content of a sub-folder belonging to a <USER_ID>:

mc ls <ALIAS>/<BUCKET>/<USER_ID>

To copy a <FILE> into the respective subfolder:

mc cp <FILE> <ALIAS>/<BUCKET>/<USER_ID>

For more details and commands, refer to the MinIO-mc documentation.

Note: Mounting the S3 storage via software like s3fs should be avoided, as it results in an excessively high number of active connections.

On this page