GDAL: *.nc rasters, opened as "netCDF/Network Common Data Format" under Linux and as " HDF5/Hierarchical Data Format Release 5" under Windows

Question

I have a set of rasters that are in the "*.nc" format and I noticed that gdalinfo (and subsequently other GIS packages based on GDAL like QGIS) is opening them with two different drivers.

Linux:

gdalinfo A20133052013312.L3m_8D_CHL_chl_ocx_9km.nc 
Driver: netCDF/Network Common Data Format

Windows:

gdalinfo.exe A20133052013312.L3m_8D_CHL_chl_ocx_9km.nc
Driver: HDF5/Hierarchical Data Format Release 5

The problem is that under Linux I get the proper/expected range of values, while under Windows the range not only is not the same, but is completely off.

Answer 1

The correct answer was given yesterday on IRC in the GDAL channel.

The above *.nc file is a netCDF version 4 encapsulated into a HDF5 format.

Under Linux the libraries support the version 4 of netCDF, so the file opens with this (specific) driver and the medatadata (as min/max values) are correctly displayed.

Under Windows (at least with GDAL installed using the OSGeo4W installer) GDAL comes with an older library for netCDF that does not support version 4, so instead is used the HDF5 driver, that is a generic one and apparently not able to read correctly the metadata from the dataset.

Answer 2

I've been able to open your NetCDF file on Windows as NetCDF using the GDAL binaries distributed by GIS Internals. My version is a couple of releases old.

C:\Temp>gdalinfo --version
GDAL 1.11.2, released 2015/02/10

I think there are 2 issues:

1. The order of drivers is different between Windows and Linux:

Windows:

gdalinfo --formats | findstr "netCDF HDF5"

HDF5 (ros): Hierarchical Data Format Release 5
HDF5Image (ro): HDF5 Dataset
netCDF (rw+s): Network Common Data Format

Linux:

gdalinfo --formats|grep 'netCDF\|HDF5'
netCDF (rw+s): Network Common Data Format
HDF5 (ros): Hierarchical Data Format Release 5
HDF5Image (ro): HDF5 Dataset

The order the drivers appear in the gdalinfo --formats list is the order they are tried. So Windows will try the HDF5 driver before the netCDF.

This is possibly (likely?) due to those drivers being built as plugins and the HDF* driver getting loaded/tried before NetCDF as it is alphabetically first in the filesystem (suggested by @kyle)

2. The Windows NetCDF driver isn't trying to open your .nc file

Your answer that says "Under Windows (at least with GDAL installed using the OSGeo4W installer) GDAL comes with an older library for netCDF that does not support version 4" was partially correct. OSGeo4W x32 shipped an old netcdf 3.6.1 from ~2008 (until 17/02/2016 when it was upgraded. However, OSGeo4W x64 ships netcdf 4.3.0 and my various GDAL installs from GIS Internals also ship NetCDF 4.3.0.

However, even after skipping the HDF5 driver, I still couldn't open your *.nc file...

C:\Temp>set GDAL_SKIP=HDF5 HDF5Image

C:\Temp>gdalinfo A20133052013312.L3m_8D_CHL_chl_ocx_9km.nc
ERROR 4: `A20133052013312.L3m_8D_CHL_chl_ocx_9km.nc' not recognised as a supported file format.

gdalinfo failed - unable to open 'A20133052013312.L3m_8D_CHL_chl_ocx_9km.nc'.

I don't know why the NetCDF driver wouldn't attempt to open the file, but it can be forced to by using the NETCDF: prefix

C:\Temp>gdalinfo NETCDF:"A20133052013312.L3m_8D_CHL_chl_ocx_9km.nc"
...
Subdatasets:
  SUBDATASET_1_NAME=NETCDF:"A20133052013312.L3m_8D_CHL_chl_ocx_9km.nc":chl_ocx
  SUBDATASET_1_DESC=[2160x4320] mass_concentration_chlorophyll_concentration_in_sea_water (32-bit floating-point)
  SUBDATASET_2_NAME=NETCDF:"A20133052013312.L3m_8D_CHL_chl_ocx_9km.nc":palette
  SUBDATASET_2_DESC=[3x256] palette (32-bit floating-point)

C:\Temp>gdalinfo NETCDF:"A20133052013312.L3m_8D_CHL_chl_ocx_9km.nc":chl_ocx -stats
Driver: netCDF/Network Common Data Format
Files: A20133052013312.L3m_8D_CHL_chl_ocx_9km.nc
       A20133052013312.L3m_8D_CHL_chl_ocx_9km.nc.aux.xml
Size is 4320, 2160
Coordinate System is `'
Origin = (-179.999994916092280,90.000002544309424)
Pixel Size = (0.083333338043927,-0.083333335689175)
Metadata:

<SNIP...>

Corner Coordinates:
Upper Left  (-179.9999949,  90.0000025)
Lower Left  (-179.9999949, -90.0000025)
Upper Right ( 180.0000254,  90.0000025)
Lower Right ( 180.0000254, -90.0000025)
Center      (   0.0000153,  -0.0000000)
Band 1 Block=4320x1 Type=Float32, ColorInterp=Undefined
  Min=0.001 Max=97.772
  Minimum=0.001, Maximum=97.772, Mean=0.321, StdDev=1.265
  NoData Value=-32767
  Metadata:
    _FillValue=-32767
    add_offset=0
    display_max=20
    display_min=0.01
    display_scale=log
    long_name=Chlorophyll Concentration, OC3 Algorithm
    NETCDF_VARNAME=chl_ocx
    reference=O'Reilly, J.E., and 24 Coauthors, 2000: SeaWiFS Postlaunch Calibration and Validation Analyses, Part 3. NASA Tech. Memo. 2000-206892, Vo
sfc.nasa.gov/REPROCESSING/R2009/ocv6/
    scale_factor=1
    standard_name=mass_concentration_chlorophyll_concentration_in_sea_water
    STATISTICS_MAXIMUM=97.772338867188
    STATISTICS_MEAN=0.3208483871507
    STATISTICS_MINIMUM=0.0010000000474975
    STATISTICS_STDDEV=1.2654627276218
    units=mg m^-3
    valid_max=100
    valid_min=0.001