• Freva instances available at DKRZ

Freva

The Free Evaluation System Framework (Freva) is an open source data search and analysis platform developed at the FUB and DKRZ and provided by the Climate Informatics and Technologies research group (CLINT) at the Data Analysis department. You can install the freva package via pip or conda/mamba.

With the help of Freva researchers can, among other things:

• quickly and intuitively search for data stored at typical data centers that host many datasets (databrowser for data already in freva, esgf for data at esgf nodes).

• create a common interface for user defined data analysis tools (plugins).

• apply data analysis tools in a reproducible manner (history).

Freva can be accessed either via command line interface (CLI), via web browser or with a python module, with similar functionalities in all the cases.

For a complete overview of the Freva functionalities please refer to its official documentation page, which is regularly updated.

Freva access

Freva is a framework whose usage is (mainly) limited to projects, i.e. there are different instances for different projects. All the instances share the same core functionalities but have been personalized to cater the necessities of each project hosting their own tools, history database and data that can be found through the databrowser.

As such, in order to access them you will need to either load a particular module (cli), go to a website (web), or indicate a specific configuration file (python):

Module	Website	Configuration file
gems	https://gems.dkrz.de/	/work/ch1187/clint/nextgems/freva/evaluation_system.conf

Warmworld -GEMS instance is open to anyone with a dkrz account. Through this instance you will be able to search most of the data hosted at levante: CMIP6 data hosted at DKRZ’s ESGF node, CMIP5, Cordex, and many reanalysis products. However, take in account that there are no plugins available for data analysis here.

Note

Have an overview of all Freva instances available at DKRZ here.

Below, we indicate shortly how to access Freva via command line and in a python environment, the web access is self-explanatory. This a short summary of Freva’s documentation FAQ.

1. Command-line interface

In order to access via command-line you will, first, need to connect to levante and then load clint + your module (module column):

% ssh <username>@levante.dkrz.de
$USER@levante $ module load clint <freva-instance>
$USER@levante $ freva --help
usage: freva [-h] [-V] {esgf,user-data,plugin,history,databrowser} ...

Free EVAluation system framework (freva)

positional arguments:
  {esgf,user-data,plugin,history,databrowser}
...

2. Python module

As mentioned before, Freva can be directly used within a python environment. This can be done via Freva’s python API. Here we explain 3 ways to do it.

a. Using available Freva environment

The Freva environment at levante ships with the most common libraries for climatological data processing and plotting (numpy, xarray, pandas, matplotlib, cartopy).

The simplest way to use the Freva python module is 1) load a freva instance and 2) execute your python scripts with the loaded environment.

% ssh <username>@levante.dkrz.de
$USER@levante $ module load clint <freva-instance>
$USER@levante $ python your-script.py

Naturally, in the python script you will need to import the freva package:

import freva
freva.count_values()

b. Using DKRZ’s Jupyterhub

DKRZ provides its own Jupyterhub, a multi-user server to serve Jupyter Notebooks to a large number of users.

You can create your own freva-kernel out of the available freva instances, for that:

$USER@levante $ module load clint <freva-instance>
$USER@levante $ jupyter-kernel-install python --name freva --display-name <freva-instance>

And then load the freva package in your jupyter notebook.

Alternatively, to create a jupyter kernel from your own environment (see below) you will need to also include the appropriate environment variable, for that:

(myenv) $USER@levante $ python -m ipykernel install --user --name my-kernel-name --env EVALUATION_SYSTEM_CONFIG_FILE </path/to/freva/evaluation_system.conf>

c. Using own environment

If you want to run Freva in your own (e.g. python or conda) environment you will need to install it first either via pip or conda/mamba:

(myenv) $USER@levante $ python3 -m pip install freva # with pip
(myenv) $USER@levante $ conda install -c conda-forge freva # with conda

In the python script, you will need to load freva + a configuration file corresponding to any of the freva instances:

import freva
freva.config(config_file="/path/to/freva/evaluation_system.conf")

3. Freva-client (🚨 NEW!!)

We are working on a complete new, REST API based Freva (freva-nextgen). You can already use its freva-client python module, cli tool, and REST API.

Important

Although it doesn’t allow for all the functionality of the current Freva, yet, you can already use it - especially if you’re just interested in searching and accessing data!

🚨 We are constantly updating the lib. So stay tuned for any features 🚨

a. Using available Freva environment

The available Freva environment at levante already ships with the tool. As before, you only need to load the module corresponding to a particular freva instance. For the cli tool:

$USER@levante $ module load clint <freva-instance>
$USER@levante $ freva-client --help

And python:

import freva_client

help(freva_client)

The syntax of the queries are very similar to the current Freva tool.

b. Using own environment

You will need to install the library via pip:

(myenv) $USER@levante $ pip install freva-client

If you want to use the databrowser you’d need to set the host name (see “website” column) to make a connection to the freva server. There are 2 options:

i. setting the host in the query

This is done by explicitly setting the host key in the databrowser call, e.g. for cli:

(myenv) $USER@levante $ freva-client databrowser data-search --host <hostname> [query]

and python:

from freva_client import databrowser

db = databrowser(host="<hostname>",...)

ii. creating a config file:

You will need to create a directory:

(myenv) $USER@levante $ mkdir -p ~/.config/freva

And then add the content of the following file to ~/.config/freva/freva.toml:

## The new freva configuration file.
## This configuration files follows toml (https://toml.io) syntax and replaces
## the old evaluation_system.conf file.
#
[freva]
## This section configures the freva client. All settings related to freva
## are set here.
##
## The name of the specific freva instance. If you use multiple configurations
## for different freva instances the `project_name` entry allows you to
## later identify which configuration you were using.
project_name = "<freva-instance>"

##
## The host that runs the freva api system. In most cases this is just the
## url of the freva webpage, such as https://www.freva.dkrz.de.
## You can set a port by separating <hostname:port>
## for example freva.example.org:7777
host = "<hostname>"