Converting GRIB to CSV or to binary data

Question

The challenge is to extract multidimensional data (weather and ocean data in this case) from .grib files, without having to

1. download and install any new software on the computer by running installation programs,
2. download and compile any new source code,
3. rely on pre-installed software on the computer (like java, python or installation packages)

so that

1. Windows 10 is supported (without Linux emulators),
2. float64 is supported,
3. it does it "bulk", ie. not only one record ("band") at time (which can take a full day for a large dataset, at speed of for example 1 band/second). Instead, it shall do the full extract with one single command (or only a handful of commands), and
4. it outputs the data from the grib to a named file, as binary or as readable ASCII.

Ie. needing a downloadable command-line tool(set) that (after virus testing) extracts the data from .grib files, for further processing!

The GRIB format

GRIB (GRIdded Binary or General Regularly-distributed Information in Binary form1) is a concise data format commonly used in meteorology to store historical and forecast weather data.

https://en.wikipedia.org/wiki/GRIB

Answer 1

There are numerous packages available on the net (see the wikipedia page in the question) but after two days of investigating practically everything I could find, there was only one good tool (fit for me) available:

NOAA / The Climate Prediction Center's (CPC) and it's wgrib or wgrib2 tool.

(I have only tried wgrib, and the grib data is Copernicus ERA5 ECMWF reanalysis).

The wgrib tool is a handful of downloadable files including one executable. Seems one can place them in any directory and the .exe (compiled in 2017) runs autonomously. The documentation is good.

The command line

wgrib source.grib | -i source.grib -text -nh -o target.txt

will write a concatenation of the multidimensional grib-file's data values as readable text (one value per row) into target.txt.

The command line

wgrib source.grib | -i source.grib -bin -nh -o target.bin

will write a concatenation of the same as binary into target.bin.

There are multiple options.

---

Edit 26.7 2020: In response to user "xunilks" comment below, I'm adding some detail about the output formats of the conversion.

Example: If the grib file contains the following dimensions:

1. latitude (0.25 degree intervals)
2. longitude (0.25 degree intervals)
3. time (one hour interval)

with the following cell data

1. Wind U component
2. Wind V component
3. Significant wave height

and the dimensional spans are

1. latitude; 10-11 degrees (with 0.25 deg intervals, gives 1x4+1 = 5 values; 10, 10.25, 10.5, 10.75, 11)
2. longitude; 45-47 degrees (with 0.25 deg intervals, gives 2x4+1 = 11 values; 45, 45.25, 45.5, 45.75, 46, 46.25, 46.5, 46.75, 47)
3. time; 25.7 2020 - 27.7 2020 (one hour intervals, gives 2x24+1 = 49 values)

then there will be an output of 5x11x49 = 2695 cells and each cell will contain 3 numbers (windu, windv, waveh). Hence, the total number of numbers in the data file will be 2695x3 = 8085. wgrib writes them either as a readable text file (with 8085 rows) or a binary file of 32-bit (=4 byte) floats (note: not 64-bit, unfortunately, so we get 7 significants instead of 14 - not a huge problem), hence the file size will be 8085x4 = 32340 bytes.

(For example in C#, one can use BitConverter to convert bytes into float-32's)

The order seems to be (time(data(lat(lon)))) repetitions, where data consists of windu,windv,waveh. This means that

• the first 11x5=55 numbers in the binary (or text) file are windu values for the 1st hour, as a concatenation of lon columns and lat rows.
• the second 11x5=55 numbers (ie. numbers 56-110) are windv values for the 1st hour, same concatenation
• the third 55 numbers are waveh values for the first hour, ...
• the fourth 55 numbers are windu values for the second hour, ...
• the fifth 55 numbers are windv values for the second hour, ...
• etc.

wgrib writes a nice housekeeping file that shows detail about the order of the written numbers. In Windows, one gets that into a text file by adding >[filename] to the command line, eg.

wgrib source.grib | -i source.grib -text -nh -o target.txt >myindexhelpfile.txt

See wgrib documentation!