1

gdal_translate is able to convert a multidimensional GRIB data file (Copernicus ERA5 reanalysis) with the following command-line command (gdal_translate.exe from 11/2016):

gdal_translate source.grib target1.xyz

gdal_translate will in this case create two files:

  1. target1.xyz.aux.xml
  2. target1.xyz

target1.xyz.aux.xml is rather clear, it contains readable data like this:

<PAMDataset>
  <PAMRasterBand band="1">
    <Metadata>
      <MDI key="GRIB_COMMENT">10 metre u wind component [m/s]</MDI>
      <MDI key="GRIB_ELEMENT">10U</MDI>
      <MDI key="GRIB_FORECAST_SECONDS">0 sec</MDI>
      <MDI key="GRIB_REF_TIME">310262400 sec UTC</MDI>
      <MDI key="GRIB_SHORT_NAME">0-SFC</MDI>
      <MDI key="GRIB_UNIT">[m/s]</MDI>
      <MDI key="GRIB_VALID_TIME">310262400 sec UTC</MDI>
    </Metadata>
  </PAMRasterBand>
  <PAMRasterBand band="2">
    etc.

target1.xyz is a mystery for me. It's definitely not readable ASCII, but binary. It's not possible to paste the first 100 bytes into this SE edit window. Reading from file start as 64-bit doubles produces crazy values (huge exponents), so it definitely has a structure and most likely a header.

No solution:

Once I have the xml file, I can parse it's headers and get the corresponding band numbers. Then I can call gdal_translate again, this time with a band number (note "target2" file name). Example:

gdal_translate source.grib target2.xyz -b 12345

In this case, I do get readable ASCII, example:

22    60     4.953857421875
22.25 60     5.292724609375
22.5  60     5.439208984375
22.75 60     5.296630859375
23    60     5.023193359375
23.25 60     4.396240234375
23.5  60     3.891357421875
23.75 60     4.203857421875
24    60     4.513427734375
24.25 60     4.732177734375
24.5  60     4.951904296875
etc.

However, each conversion takes one second and there can well be 30000 bands to extract => 8 hours! I would very much like to read the target1.xyz file using own code, but it seems impossible to find any information about it's format.

Tried in the first conversion ("target1", with no band argument):

  • Using other target1 file name extension, in order to get another output format - gdal_translate ignores.
  • Specifying the output format with the -of [format] parameter, gdal_translate ignores.
  • Reanaming target1.xyz => target1.tiff and loading into a program with capabilities - no go (says invalid file format). Seemed like a waste of time to try more renaming + loading, blind shooting.

Tried in the second conversion ("target2", with band argument):

  • Specifying multiple bands simultaneously (as the specification says it is possible, see https://gdal.org/programs/gdal_translate.html) - gdal_translate ignores all but the first band (this is also verified elsewhere in SE); in addition the DOS command line has a length limit. (I could have imagined that gdal_translate had created multiple output file name variations in this case). Example:

    gdal_translate source.grib target2.xyz -b 12345 12346 12347 etc.   
    

Question: What is the file format and structure of the target1.xyz file that

gdal_translate source.grib target1.xyz

creates?

3

If your GDAL is from 2016 it's time to update.

Your output file format is a GeoTIFF.

The default output format (in GDAL < 2.3) is a GeoTIFF. With your GDAL, if you want an XYZ ascii file, you need to tell GDAL that by specifying the format with the -of flag.

gdal_translate -of XYZ source.grib target1.xyz

Starting with GDAL 2.3 released in 2018, if not specified, the format is guessed from the file extension.

Note that while gdal_translate can output multiple bands in general, it can only output multiple bands to formats that support this. However, the XYZ format only supports a single band. So if you try to output multiple bands (with a recent GDAL) you will get the message:

Warning 6: XYZ driver only uses the first band of the dataset.

You can of course specify a different band using a -b flag, but you still can only output a single band at a time to the XYZ format

gdal_translate -b 1 -of XYZ source.grib target1.xyz
gdal_translate -b 2 -of XYZ source.grib target2.xyz
gdal_translate -b 3 -of XYZ source.grib target3.xyz
gdal_translate -b 4 -of XYZ source.grib target4.xyz
etc...
gdal_translate -b 999 -of XYZ source.grib target999.xyz
| improve this answer | |
  • Thank you for your reply! Yes, I did indeed try -of XYZ too, and it outputs the first band only (and that with no -b specified). While no -of parameter outputs a full bandset, but as binary (GeoTiff you say, probably this PaintShop 2020 is incapable - earlier versions did read). Using the -b flag was an initial problem because of slowness (as I wrote). But I understand this 11/2016 version is outdated, that's a good point. See comment below! – Stormwind Jul 23 at 22:28
  • Could you by any chance point me at the good and sufficiently updated executables, as i neither use C nor Python, and also do not compile. I am somewhat reluctant to running installation programs on this devbox, so it would be a download of (trusted) exe's + antivirus tests. Thx! – Stormwind Jul 23 at 22:31
  • 1
    Yes, as I noted, XYZ format only supports a single band so GDAL defaults to the first band if you don't specify a band. You can download GDAL from gisinternals.com/release.php (download the compiled binaries in a single zip, no installation required for 2.4.4 or 3.0.4 ). – user2856 Jul 24 at 2:17
  • Am I right then, in that GDAL won't translate the entire grib file (or netcdf, as an other source option) in one operation into xml or json - including the actual data? As these two formats are easy to read into the natively multidimensional APL language. – Stormwind Jul 24 at 11:15
  • Marking Answered and placing new Q. – Stormwind Jul 24 at 19:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.