gdal_translate reading unexpected values from NetCDF

Question

I'm trying to generate separate files from the layers of a NetCDF downloaded from ECMWF's Copernicus data archive (https://cds.climate.copernicus.eu/cdsapp#!/dataset/reanalysis-era5-single-levels).

I can open the file in python and clearly see the values I want in each layer which I expect to be between 0 and 1:

>>> import netCDF4 as nc
>>> fn = "ecmwfGrab_S_.nc"
>>> ds = nc.Dataset(fn)
>>> ds['tcc'][:][0]
masked_array(
  data=[[0.97147217, 0.9679472 , 0.962469  , 0.9503071 , 0.94968146,
         0.91046427, 0.881822  , 0.8939076 , 0.89589135, 0.90016404,
         0.91324152, 0.92880632, 0.93549002],
        [0.99909205, 0.99724564, 0.99724564, 0.99546027, 0.99546027,
         0.98633502, 0.98633502, 0.98908175, 0.98908175, 0.99134017,
         0.99134017, 0.99343074, 0.99187426],
        [0.99999237, 0.99999237, 0.99999237, 0.99999237, 0.99999237,
         0.99999237, 0.99999237, 0.99999237, 0.99999237, 0.99999237,
         0.99999237, 0.99825277, 0.991859  ]],

When I try to get a seperate raster using gdal_translate:

gdal_translate -of AAIGrid -b 1 NETCDF:"ecmwfGrab_S_.nc":tcc test.txt

I get the following values in my test.txt file:

 30897 30666 30307 29510 29469 26899 25022 25814 25944 26224 27081 28101 28539
 32707 32586 32586 32469 32469 31871 31871 32051 32051 32199 32199 32336 32234
 32766 32766 32766 32766 32766 32766 32766 32766 32766 32766 32766 32652 32233

The following is the gdalinfo output for the first band:

Band 1 Block=13x1 Type=Int16, ColorInterp=Undefined
  NoData Value=-32767
  Unit Type: (0 - 1)
  Offset: 0.499996184895262,   Scale:1.5259604007058e-05
  Metadata:
    add_offset=0.4999961848952621
    long_name=Total cloud cover
    missing_value=-32767
    NETCDF_DIM_time=1055544
    NETCDF_VARNAME=tcc
    scale_factor=1.525960400705799e-05
    standard_name=cloud_area_fraction
    units=(0 - 1)
    _FillValue=-32767

What is wrong in my use of gdal_translate?

Answer 1

Maybe the data are scaled and you should tell gdal_translate to unscale https://gdal.org/programs/gdal_translate.html#cmdoption-gdal_translate-unscale

Apply the scale/offset metadata for the bands to convert scaled values to unscaled values. It is also often necessary to reset the output datatype with the -ot switch.

Gdalinfo shows possible offset and scale in the metadata like this:

Metadata:
    add_offset=1.5
    long_name=z
    NETCDF_VARNAME=z
    scale_factor=0.01