2

I'm trying to write a test for code that works with hycom netcdf global ocean circulation data. The source file 2 subdatasets:

gdalinfo hycom_glb_911_2015110200_t000_uv3z.nc | grep _NAME
  SUBDATASET_1_NAME=NETCDF:"hycom_glb_911_2015110200_t000_uv3z.nc":water_u
  SUBDATASET_2_NAME=NETCDF:"hycom_glb_911_2015110200_t000_uv3z.nc":water_v

With 40 depth bands:

gdalinfo NETCDF:hycom_glb_911_2015110200_t000_uv3z.nc:water_u | grep '^Band' | head -3
Band 1 Block=4500x1 Type=Int16, ColorInterp=Undefined
Band 2 Block=4500x1 Type=Int16, ColorInterp=Undefined
Band 3 Block=4500x1 Type=Int16, ColorInterp=Undefined

I've got some code written the does things like calculate the speed in m/s from the UV and creates a geotiff, etc. The original files are much too large for using in unittests. Is there an easy way with gdal, nco or ? to keep the structure and metadata, but create a much smaller file? I'm open to dropping all but the first two bands and writing a constant value into each band + turning on deflate compression.

Trying with GDAL:

gdal_translate --version  # At head -> r31584
GDAL 2.1.0dev, released 2015/99/99

gdal_translate hycom_glb_911_2015110200_t000_uv3z.nc hycom_glb_911_2015110200_t000_uv3z-try2.nc -of netcdf -co compress=deflate
Input file contains subdatasets. Please, select one of them for reading.

The originals are quite a bit large for tests:

wget ftp://ftp.hycom.org/datasets/GLBu0.08/expt_91.1/data/hindcasts/2015/hycom_glb_911_2015110200_t000_uv3z.nc

ls -lh hycom_glb_911_2015110200_t000_uv3z.nc
-rw-r----- 1 schwehr group 1.4G Nov 19 09:13 hycom_glb_911_2015110200_t000_uv3z.nc

The details of the source netcdf file:

ncdump -h hycom_glb_911_2015110200_t000_uv3z.nc
netcdf hycom_glb_911_2015110200_t000_uv3z {
dimensions:
    lat = 2001 ;
    lon = 4500 ;
    depth = 40 ;
    time = UNLIMITED ; // (1 currently)
variables:
    double time(time) ;
        time:long_name = "Valid Time" ;
        time:units = "hours since 2000-01-01 00:00:00" ;
        time:time_origin = "2000-01-01 00:00:00" ;
        time:calendar = "gregorian" ;
        time:axis = "T" ;
        time:NAVO_code = 13 ;
    double tau(time) ;
        tau:long_name = "Tau" ;
        tau:units = "hours since analysis" ;
        tau:time_origin = "2015-11-02 00:00:00" ;
        tau:NAVO_code = 56 ;
    double depth(depth) ;
        depth:long_name = "Depth" ;
        depth:standard_name = "depth" ;
        depth:units = "m" ;
        depth:positive = "down" ;
        depth:axis = "Z" ;
        depth:NAVO_code = 5 ;
    double lat(lat) ;
        lat:long_name = "Latitude" ;
        lat:standard_name = "latitude" ;
        lat:units = "degrees_north" ;
        lat:point_spacing = "even" ;
        lat:axis = "Y" ;
        lat:NAVO_code = 1 ;
    double lon(lon) ;
        lon:long_name = "Longitude" ;
        lon:standard_name = "longitude" ;
        lon:units = "degrees_east" ;
        lon:modulo = "360 degrees" ;
        lon:axis = "X" ;
        lon:NAVO_code = 2 ;
    short water_u(time, depth, lat, lon) ;
        water_u:long_name = "Eastward Water Velocity" ;
        water_u:standard_name = "eastward_sea_water_velocity" ;
        water_u:units = "m/s" ;
        water_u:_FillValue = -30000s ;
        water_u:missing_value = -30000s ;
        water_u:scale_factor = 0.001f ;
        water_u:add_offset = 0.f ;
        water_u:NAVO_code = 17 ;
    short water_v(time, depth, lat, lon) ;
        water_v:long_name = "Northward Water Velocity" ;
        water_v:standard_name = "northward_sea_water_velocity" ;
        water_v:units = "m/s" ;
        water_v:_FillValue = -30000s ;
        water_v:missing_value = -30000s ;
        water_v:scale_factor = 0.001f ;
        water_v:add_offset = 0.f ;
        water_v:NAVO_code = 18 ;

// global attributes:
        :classification_level = "UNCLASSIFIED" ;
        :distribution_statement = "Approved for public release. Distribution unlimited." ;
        :downgrade_date = "not applicable" ;
        :classification_authority = "not applicable" ;
        :institution = "Naval Oceanographic Office" ;
        :source = "HYCOM archive file" ;
        :history = "archv2ncdf3z" ;
        :field_type = "instantaneous" ;
        :Conventions = "CF-1.0 NAVO_netcdf_v1.0" ;
1
  • Please do a 'ncdump -h hycom_glb_911_2015110200_t000_uv3z.nc' so we can tell what the variables and dimensions are.
    – Dave X
    Nov 19, 2015 at 19:37

1 Answer 1

3

I'd use NCO's ncks:

ncks -v water_u,water_v -d,depth,1,2,1 hycom_glb_911_2015110200_t000_uv3z.nc out.nc 

with 'depth' being whatever the depth/band dimension is.

8
  • 1
    ncks -4 --deflate 9 -v water_u,water_v -d depth,1,2,1 hycom_glb_911_2015110200_t000_uv3z.nc out.nc Nov 19, 2015 at 20:22
  • 1
    I didn't realize that you can also use the step option to keep the same basic bounds, but only do every 10th sample on the longitude to get down to 270K: ncks -4 --deflate 9 -v water_u,water_v -d depth,1,2,1 -d lon,0,360,10 hycom_glb_911_2015110200_t000_uv3z.nc out.nc Nov 19, 2015 at 20:33
  • Yep. You could sample the latitudes as well.
    – Dave X
    Nov 19, 2015 at 20:37
  • Oh -- you might make sure your whether your sampling is in variable space or in index space. I think it is in index space, but I'm not sure.
    – Dave X
    Nov 19, 2015 at 20:43
  • Sadly, the subsampling with "-d lon,0,360,10" totally breaks the georeferencing. Nov 21, 2015 at 9:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.