API for Accounting Data

As a project admin or even as a normal project user you may look from time to time into accounting data for resources we provide (Levante, archive, etc.).

For each project you participate in, you can find a table on https://luv.dkrz.de like the one shown below. Usually you only see your own entries. Other user’s names are hidden.

utilization table

Utilization for compute nodes of Mistral

While this is okay for many use cases, you may want to collect the data on a regular basis with a script where an API makes things much easier.

Authentication

You need to log into luv to view the table for your projects. This information is not public. So we also need authentication to manage access through the API. One way would be to use the session you create when logging in with the browser. This actually works but it is not convenient when you want to collect data in an automated fashion.

Token authentication solves this problem. Be aware that the token is as valuable as your password when using the API. So take measures to protect it.

To get a token, use your browser to visit https://luv.dkrz.de/api/new-token/ after logging in. The token is valid until you visit the url again and get a new token. You can also delete the token by visiting https://luv.dkrz.de/api/remove-token/.

Fetching data

Now you can fetch accounting data from luv.dkrz.de. One example would be

curl https://luv.dkrz.de/api/utilization/project/2022-05-20/22/ac5555/ \
-H 'Authorization: Token XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'

You have to put in your actual token. Then you would get the compute time utilization (ID 22) of project ac5555 on Levante for the date of 2022-05-20. The output format is JSON which can be parsed by many scripting languages. The example uses curl for the GET request but this can also be done in most scripting languages. In Python, the requests package works well for this kind of task.

You can also get data for other resources. Here is a list of currently available and also of retired resources. We keep accounting data for a pretty long time.

Resource

ID

Levante CPU nodes

22

Levante GPGPU nodes

24

Levante storage

23

Archive project

25

Archive long term

26

Swift Object Storage

12

Mistral compute time

13

Mistral Lustre work

14

HPSS arch

3

HPSS doku

4

Blizzard CPU time

1

GPFS work

2

Hurrikan CPU time

17