WeatherForce Tech Blog

WeatherForce Tech Blog

A notebook container for meteorology

At WeatherForce, we do a lot of data analysis with Jupyter notebooks. Since setting up an environment for meteorological data can take a bit of time, we've decided to create a Docker image containing a suitable ready-to-use environment.

The Jupyter project encourages this approach by providing a hierarchy of ready-made Docker images. Among those images, the Data Science Stack image provides a notebook installation with Python 3, R and Julia. We extended that image with useful libraries to create the Weather Notebook image. We've also included Python 2, which is still required by GRIB libraries.

In order to use our weather-notebook image, you'll need to have Docker installed. If you haven't installed it yet, you should follow instructions from the Docker website.

Starting a weather-notebook container

Once you have Docker installed, open a terminal and go to a directory where you're going to create notebooks:

$ cd <you working directory>

Now run a container based on the weather-notebook image with a command such as:

$ docker run -ti --rm -p 8888:8888 --name weather-notebook  \
             -v "$PWD":/home/jovyan weatherforce/weather-notebook

You should now be able to visit http://localhost:8888 with your web browser and create notebooks. In fact, the page you're reading now has been generated from a notebook served by a weather-notebook container.

Working with weather data

To illustrate what we can do with a weather-notebook container, we'll download a GFS GRIB file.

In [1]:
!cd data && wget ftp://ftp.ncep.noaa.gov/pub/data/nccf/com/gfs/prod/gfs.2018030806/gfs.t06z.pgrb2.0p25.f000

--2018-03-08 15:48:21--  ftp://ftp.ncep.noaa.gov/pub/data/nccf/com/gfs/prod/gfs.2018030806/gfs.t06z.pgrb2.0p25.f000
           => ‘gfs.t06z.pgrb2.0p25.f000’
Resolving ftp.ncep.noaa.gov (ftp.ncep.noaa.gov)... 140.90.101.61
Connecting to ftp.ncep.noaa.gov (ftp.ncep.noaa.gov)|140.90.101.61|:21... connected.
Logging in as anonymous ... Logged in!
==> SYST ... done.    ==> PWD ... done.
==> TYPE I ... done.  ==> CWD (1) /pub/data/nccf/com/gfs/prod/gfs.2018030806 ... done.
==> SIZE gfs.t06z.pgrb2.0p25.f000 ... 190894193
==> PASV ... done.    ==> RETR gfs.t06z.pgrb2.0p25.f000 ... done.
Length: 190894193 (182M) (unauthoritative)

gfs.t06z.pgrb2.0p25 100%[===================>] 182.05M  4.25MB/s    in 45s     

2018-03-08 15:49:08 (4.01 MB/s) - ‘gfs.t06z.pgrb2.0p25.f000’ saved [190894193]

We're going to use xarray with the pynio backend to open the GRIB file. Since pynio hasn't been ported to Python 3 at the time of writing, we must create a Python 2 notebook. (You may also convert the GRIB file to NetCDF if you'd rather work with Python 3.)

Let's import xarray with the standard alias:

In [2]:
import xarray as xr

Now we open the file with the pynio engine:

In [3]:
ds = xr.open_dataset("data/gfs.t06z.pgrb2.0p25.f000", engine="pynio")

Let's take a look at a specific variable:

In [4]:
temperature = ds['TMP_P0_L1_GLL0']
temperature
Out[4]:
<xarray.DataArray 'TMP_P0_L1_GLL0' (lat_0: 721, lon_0: 1440)>
[1038240 values with dtype=float32]
Coordinates:
  * lat_0    (lat_0) float32 90.0 89.75 89.5 89.25 89.0 88.75 88.5 88.25 ...
  * lon_0    (lon_0) float32 0.0 0.25 0.5 0.75 1.0 1.25 1.5 1.75 2.0 2.25 ...
Attributes:
    production_status:                              Operational products
    center:                                         US National Weather Servi...
    forecast_time_units:                            hours
    level:                                          [ 0.]
    forecast_time:                                  [0]
    long_name:                                      Temperature
    parameter_template_discipline_category_number:  [0 0 0 0]
    initial_time:                                   03/08/2018 (06:00)
    grid_type:                                      Latitude/longitude
    units:                                          K
    level_type:                                     Ground or water surface
    parameter_discipline_and_category:              Meteorological products, ...

We can see that this is surface temperature. Let's draw a map:

In [5]:
%matplotlib inline
temperature.plot(figsize=(12,5));

On top of xarray and pynio, we've also included these fantastic libraries in the image:

We hope that this image will allow you to easily start exploring meteorological data.

Also feel free to contact us if you wish to discuss how to integrate meteorological data within your organization.