2

I have a set of rasters that are in the "*.nc" format and I noticed that gdalinfo (and subsequently other GIS packages based on GDAL like QGIS) is opening them with two different drivers.

Linux:

gdalinfo A20133052013312.L3m_8D_CHL_chl_ocx_9km.nc 
Driver: netCDF/Network Common Data Format

Windows:

gdalinfo.exe A20133052013312.L3m_8D_CHL_chl_ocx_9km.nc
Driver: HDF5/Hierarchical Data Format Release 5

The problem is that under Linux I get the proper/expected range of values, while under Windows the range not only is not the same, but is completely off.

  • Are the versions of GDAL different? And is one driver (netCDF probably) compiled as a plugin? netCDF 4 stores data as HDF5 internally, IIRC, and the projection info will differ. If netCDF is a plugin, it might not be tested before HDF5, which may cause the issue. – user10353 Feb 2 '16 at 20:19
  • I have 1.11.2 on Linux and 1.11.3 on Windows. On Windows I use the version installed via osgeo4w, not sure there the netcdf/hdf5 support is via plugins. – Giovanni Manghi Feb 3 '16 at 10:39
3

The correct answer was given yesterday on IRC in the GDAL channel.

The above *.nc file is a netCDF version 4 encapsulated into a HDF5 format.

Under Linux the libraries support the version 4 of netCDF, so the file opens with this (specific) driver and the medatadata (as min/max values) are correctly displayed.

Under Windows (at least with GDAL installed using the OSGeo4W installer) GDAL comes with an older library for netCDF that does not support version 4, so instead is used the HDF5 driver, that is a generic one and apparently not able to read correctly the metadata from the dataset.

  • this is only correct for 32bit OSGeo4W. I've updated my answer. – user2856 Feb 16 '16 at 4:04
  • This has just been fixed in 32bit OSGeo4W (just 1 hour after I reported it!). Try upgrading your OSGeo4W installation. – user2856 Feb 17 '16 at 0:35
2

I've been able to open your NetCDF file on Windows as NetCDF using the GDAL binaries distributed by GIS Internals. My version is a couple of releases old.

C:\Temp>gdalinfo --version
GDAL 1.11.2, released 2015/02/10

I think there are 2 issues:

1. The order of drivers is different between Windows and Linux:

Windows:

gdalinfo --formats | findstr "netCDF HDF5"

HDF5 (ros): Hierarchical Data Format Release 5
HDF5Image (ro): HDF5 Dataset
netCDF (rw+s): Network Common Data Format

Linux:

gdalinfo --formats|grep 'netCDF\|HDF5'
netCDF (rw+s): Network Common Data Format
HDF5 (ros): Hierarchical Data Format Release 5
HDF5Image (ro): HDF5 Dataset

The order the drivers appear in the gdalinfo --formats list is the order they are tried. So Windows will try the HDF5 driver before the netCDF.

This is possibly (likely?) due to those drivers being built as plugins and the HDF* driver getting loaded/tried before NetCDF as it is alphabetically first in the filesystem (suggested by @kyle)

2. The Windows NetCDF driver isn't trying to open your .nc file

Your answer that says "Under Windows (at least with GDAL installed using the OSGeo4W installer) GDAL comes with an older library for netCDF that does not support version 4" was partially correct. OSGeo4W x32 shipped an old netcdf 3.6.1 from ~2008 (until 17/02/2016 when it was upgraded. However, OSGeo4W x64 ships netcdf 4.3.0 and my various GDAL installs from GIS Internals also ship NetCDF 4.3.0.

However, even after skipping the HDF5 driver, I still couldn't open your *.nc file...

C:\Temp>set GDAL_SKIP=HDF5 HDF5Image

C:\Temp>gdalinfo A20133052013312.L3m_8D_CHL_chl_ocx_9km.nc
ERROR 4: `A20133052013312.L3m_8D_CHL_chl_ocx_9km.nc' not recognised as a supported file format.

gdalinfo failed - unable to open 'A20133052013312.L3m_8D_CHL_chl_ocx_9km.nc'.

I don't know why the NetCDF driver wouldn't attempt to open the file, but it can be forced to by using the NETCDF: prefix

C:\Temp>gdalinfo NETCDF:"A20133052013312.L3m_8D_CHL_chl_ocx_9km.nc"
...
Subdatasets:
  SUBDATASET_1_NAME=NETCDF:"A20133052013312.L3m_8D_CHL_chl_ocx_9km.nc":chl_ocx
  SUBDATASET_1_DESC=[2160x4320] mass_concentration_chlorophyll_concentration_in_sea_water (32-bit floating-point)
  SUBDATASET_2_NAME=NETCDF:"A20133052013312.L3m_8D_CHL_chl_ocx_9km.nc":palette
  SUBDATASET_2_DESC=[3x256] palette (32-bit floating-point)

C:\Temp>gdalinfo NETCDF:"A20133052013312.L3m_8D_CHL_chl_ocx_9km.nc":chl_ocx -stats
Driver: netCDF/Network Common Data Format
Files: A20133052013312.L3m_8D_CHL_chl_ocx_9km.nc
       A20133052013312.L3m_8D_CHL_chl_ocx_9km.nc.aux.xml
Size is 4320, 2160
Coordinate System is `'
Origin = (-179.999994916092280,90.000002544309424)
Pixel Size = (0.083333338043927,-0.083333335689175)
Metadata:

<SNIP...>

Corner Coordinates:
Upper Left  (-179.9999949,  90.0000025)
Lower Left  (-179.9999949, -90.0000025)
Upper Right ( 180.0000254,  90.0000025)
Lower Right ( 180.0000254, -90.0000025)
Center      (   0.0000153,  -0.0000000)
Band 1 Block=4320x1 Type=Float32, ColorInterp=Undefined
  Min=0.001 Max=97.772
  Minimum=0.001, Maximum=97.772, Mean=0.321, StdDev=1.265
  NoData Value=-32767
  Metadata:
    _FillValue=-32767
    add_offset=0
    display_max=20
    display_min=0.01
    display_scale=log
    long_name=Chlorophyll Concentration, OC3 Algorithm
    NETCDF_VARNAME=chl_ocx
    reference=O'Reilly, J.E., and 24 Coauthors, 2000: SeaWiFS Postlaunch Calibration and Validation Analyses, Part 3. NASA Tech. Memo. 2000-206892, Vo
sfc.nasa.gov/REPROCESSING/R2009/ocv6/
    scale_factor=1
    standard_name=mass_concentration_chlorophyll_concentration_in_sea_water
    STATISTICS_MAXIMUM=97.772338867188
    STATISTICS_MEAN=0.3208483871507
    STATISTICS_MINIMUM=0.0010000000474975
    STATISTICS_STDDEV=1.2654627276218
    units=mg m^-3
    valid_max=100
    valid_min=0.001
  • It's plugins, I believe they are loaded first. Maybe alphabetically? – user10353 Feb 2 '16 at 23:41
  • @kyle, that sounds logical, and yes in my windows GDAL, the HDF* and netCDF drivers are plugins. – user2856 Feb 3 '16 at 1:09
  • Thanks for the answers, that make sense, but it also raises new questions/issues: if I skip the HDF5 driver then I get (under Windows): ERROR 4: `A20133052013312.L3m_8D_CHL_chl_ocx_9km.nc' not recognised as a supp ed file format. gdalinfo failed - unable to open 'A20133052013312.L3m_8D_CHL_chl_ocx_9km.nc'. – Giovanni Manghi Feb 3 '16 at 10:37
  • @Luke yes, I just checked. So apparently the same dataset opens fine with the netCDF driver under Linux, but does not under Windows. If you want to give it a go: dropbox.com/s/0n92v1lksckc157/… – Giovanni Manghi Feb 3 '16 at 10:56
  • @Luke, I posted the correct answer down here. Thanks a lot for your feedback! – Giovanni Manghi Feb 4 '16 at 17:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.