0

Given a series of NetCDF datasets like fileA.nc, fileB.nc, fileC.nc, etc., all of which share the same dimensions, and the three dimensions are identical (i.e. same spatial and temporal extents), I am attempting to concatenate these files into one dataset.

What differs in each file is the non-coordinate variable (A, B, C, etc., for about 400 variables). Each dataset contains one of these only, and is named accordingly. These can be sorted, and these could be seen as an additional dimension (perhaps labelled scenario), but treating it as a simple array of variables is adequate, too.

Here is the structure of one input file:

netcdf fileA {
dimensions:
    time = 115 ;
    longitude = 243 ;
    latitude = 260 ;
variables:
    int64 time(time) ;
        time:units = "days since 2006-08-30 00:00:00" ;
        time:calendar = "proleptic_gregorian" ;
    float longitude(longitude) ;
        longitude:_FillValue = NaNf ;
        longitude:long_name = "longitude" ;
        longitude:units = "degrees" ;
        longitude:standard_name = "longitude" ;
        longitude:axis = "X" ;
    float latitude(latitude) ;
        latitude:_FillValue = NaNf ;
        latitude:long_name = "latitude" ;
        latitude:units = "degrees" ;
        latitude:standard_name = "latitude" ;
        latitude:axis = "Y" ;
    int64 A(time, latitude, longitude) ;
}

I have attempted to concatenate these files using the NCO tool ncks and the group aggregation method:

ncecat --gag /data/output/file*.nc concatenated.nc

However (as expected) the output structure is not ideal:

group: fileA {
  dimensions:
    time = 115 ;
    longitude = 243 ;
    latitude = 260 ;
  variables:
    int64 time(time) ;
        time:units = "days since 2006-08-30 00:00:00" ;
        time:calendar = "proleptic_gregorian" ;
    float longitude(longitude) ;
        longitude:_FillValue = NaNf ;
        longitude:long_name = "longitude" ;
        longitude:units = "degrees" ;
        longitude:standard_name = "longitude" ;
        longitude:axis = "X" ;
    float latitude(latitude) ;
        latitude:_FillValue = NaNf ;
        latitude:long_name = "latitude" ;
        latitude:units = "degrees" ;
        latitude:standard_name = "latitude" ;
        latitude:axis = "Y" ;
    int64 A(time, latitude, longitude) ;
  } // group fileA

group: fileB {
  dimensions:
    time = 115 ;
    longitude = 243 ;
    latitude = 260 ;
  variables:
    int64 time(time) ;
        time:units = "days since 2006-08-30 00:00:00" ;
        time:calendar = "proleptic_gregorian" ;
    float longitude(longitude) ;
        longitude:_FillValue = NaNf ;
        longitude:long_name = "longitude" ;
        longitude:units = "degrees" ;
        longitude:standard_name = "longitude" ;
        longitude:axis = "X" ;
    float latitude(latitude) ;
        latitude:_FillValue = NaNf ;
        latitude:long_name = "latitude" ;
        latitude:units = "degrees" ;
        latitude:standard_name = "latitude" ;
        latitude:axis = "Y" ;
    int64 B(time, latitude, longitude) ;
  } // group fileB
}

The output I am trying to achieve would be like:

netcdf concatenated {
  dimensions:
    time = 115 ;
    longitude = 243 ;
    latitude = 260 ;
  variables:
    int64 time(time) ;
        time:units = "days since 2006-08-30 00:00:00" ;
        time:calendar = "proleptic_gregorian" ;
    float longitude(longitude) ;
        longitude:_FillValue = NaNf ;
        longitude:long_name = "longitude" ;
        longitude:units = "degrees" ;
        longitude:standard_name = "longitude" ;
        longitude:axis = "X" ;
    float latitude(latitude) ;
        latitude:_FillValue = NaNf ;
        latitude:long_name = "latitude" ;
        latitude:units = "degrees" ;
        latitude:standard_name = "latitude" ;
        latitude:axis = "Y" ;
    int64 A(time, latitude, longitude) ;
    int64 B(time, latitude, longitude) ;
  } 

Or an alternative structure where A, B, etc., are some kind of record dimension, for a four dimensional output.

I presume that --gag (group aggregation) is incorrect since the desired output is not structured as groups. However using the default record aggregation results in an error since the variables do not share a name:

ncecat file*.nc concatenated.nc
ERROR: nco_inq_varid() reports requested variable "A" is not defined in file
nco_err_exit(): ERROR Short NCO-generated message (usually name of function that triggered error): nco_inq_varid()
nco_err_exit(): ERROR Error code is -49. Translation into English with nc_strerror(-49) is "NetCDF: Variable not found"
nco_err_exit(): ERROR NCO will now exit with system call exit(EXIT_FAILURE)

I'd like to find a way to concatenate these files using the NCO command line tools.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.