API for Accounting Data#
As a project admin or even as a normal project user you may look from time to time into accounting data for resources we provide (Levante, archive, etc.).
For each project you participate in, you can find a table on https://luv.dkrz.de like the one shown below. Usually you only see your own entries. Other user’s names are hidden.
Utilization for compute nodes of Mistral#
While this is okay for many use cases, you may want to collect the data on a regular basis with a script where an API makes things much easier.
Authentication#
You need to log into luv to view the table for your projects. This information is not public. So we also need authentication to manage access through the API. One way would be to use the session you create when logging in with the browser. This actually works but it is not convenient when you want to collect data in an automated fashion.
Token authentication solves this problem. Be aware that the token is as valuable as your password when using the API. So take measures to protect it.
To get a token, use your browser to visit https://luv.dkrz.de/api/new-token/ after logging in. The token is valid until you visit the url again and get a new token. You can also delete the token by visiting https://luv.dkrz.de/api/remove-token/.
Fetching data#
Now you can fetch accounting data from luv.dkrz.de. One example would be
curl https://luv.dkrz.de/api/utilization/project/2022-05-20/22/ac5555/ \
-H 'Authorization: Token XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'
You have to put in your actual token. Then you would get the compute time utilization (ID 22) of project ac5555 on Levante for the date of 2022-05-20. The output format is JSON which can be parsed by many scripting languages. The example uses curl for the GET request but this can also be done in most scripting languages. In Python, the requests package works well for this kind of task.
You can also get data for other resources. Here is a list of currently available and also of retired resources. We keep accounting data for a pretty long time.
Resource |
ID |
|---|---|
Levante CPU nodes |
22 |
Levante GPGPU nodes |
24 |
Levante storage |
23 |
Archive project |
25 |
Archive long term |
26 |
Swift Object Storage |
12 |
Mistral compute time |
13 |
Mistral Lustre work |
14 |
HPSS arch |
3 |
HPSS doku |
4 |
Blizzard CPU time |
1 |
GPFS work |
2 |
Hurrikan CPU time |
17 |