2

NetCDF seems to be a scientific standard which seems to be rather common in oceanography and atmospheric sciences. I'd like to know more about it.

How does the NetCDF format work, structure wise? Do you have a columned sturcture, like: X, Y, Z, ValueA

Can theses files be translated into something like a CSV format?

I'll paste an example from http://www.unidata.ucar.edu/software/netcdf/docs/netcdf-tutorial.html#pres_005ftemp_005f4D below. What would a single entry look like for a single point in time and space?

    netcdf pres_temp_4D {
 dimensions:
    level = 2 ;
    latitude = 6 ;
    longitude = 12 ;
    time = UNLIMITED ; // (2 currently)
 variables:
    float latitude(latitude) ;
        latitude:units = "degrees_north" ;
    float longitude(longitude) ;
        longitude:units = "degrees_east" ;
    float pressure(time, level, latitude, longitude) ;
        pressure:units = "hPa" ;
    float temperature(time, level, latitude, longitude) ;
        temperature:units = "celsius" ;
 data:

  latitude = 25, 30, 35, 40, 45, 50 ;

  longitude = -125, -120, -115, -110, -105, -100, -95, -90, -85, -80, -75, -70 ;

  pressure =
   900, 901, 902, 903, 904, 905, 906, 907, 908, 909, 910, 911,
   912, 913, 914, 915, 916, 917, 918, 919, 920, 921, 922, 923,
   924, 925, 926, 927, 928, 929, 930, 931, 932, 933, 934, 935,
   936, 937, 938, 939, 940, 941, 942, 943, 944, 945, 946, 947,
   948, 949, 950, 951, 952, 953, 954, 955, 956, 957, 958, 959,
   960, 961, 962, 963, 964, 965, 966, 967, 968, 969, 970, 971,
   972, 973, 974, 975, 976, 977, 978, 979, 980, 981, 982, 983,
   984, 985, 986, 987, 988, 989, 990, 991, 992, 993, 994, 995,
   996, 997, 998, 999, 1000, 1001, 1002, 1003, 1004, 1005, 1006, 1007,
   1008, 1009, 1010, 1011, 1012, 1013, 1014, 1015, 1016, 1017, 1018, 1019,
   1020, 1021, 1022, 1023, 1024, 1025, 1026, 1027, 1028, 1029, 1030, 1031,
   1032, 1033, 1034, 1035, 1036, 1037, 1038, 1039, 1040, 1041, 1042, 1043,
   900, 901, 902, 903, 904, 905, 906, 907, 908, 909, 910, 911,
   912, 913, 914, 915, 916, 917, 918, 919, 920, 921, 922, 923,
   924, 925, 926, 927, 928, 929, 930, 931, 932, 933, 934, 935,
   936, 937, 938, 939, 940, 941, 942, 943, 944, 945, 946, 947,
   948, 949, 950, 951, 952, 953, 954, 955, 956, 957, 958, 959,
   960, 961, 962, 963, 964, 965, 966, 967, 968, 969, 970, 971,
   972, 973, 974, 975, 976, 977, 978, 979, 980, 981, 982, 983,
   984, 985, 986, 987, 988, 989, 990, 991, 992, 993, 994, 995,
   996, 997, 998, 999, 1000, 1001, 1002, 1003, 1004, 1005, 1006, 1007,
   1008, 1009, 1010, 1011, 1012, 1013, 1014, 1015, 1016, 1017, 1018, 1019,
   1020, 1021, 1022, 1023, 1024, 1025, 1026, 1027, 1028, 1029, 1030, 1031,
   1032, 1033, 1034, 1035, 1036, 1037, 1038, 1039, 1040, 1041, 1042, 1043 ;

  temperature =
   9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20,
   21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32,
   33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44,
   45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56,
   57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68,
   69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80,
   81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92,
   93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104,
   105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116,
   117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128,
   129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140,
   141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152,
   9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20,
   21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32,
   33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44,
   45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56,
   57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68,
   69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80,
   81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92,
   93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104,
   105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116,
   117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128,
   129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140,
   141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152 ;
 }
  • Would you be able to edit your question to make clear whether you have already reviewed the NetCDF FAQ, please? If any of your questions are not already covered there then I recommend you ask them here (or maybe there is better) one at a time rather than as a "two for one" deal. – PolyGeo Aug 13 '14 at 8:46
  • 1
    It really depends on which files and sometimes which tools you need to usr, it is a very general format. R's raster package and GDAL have high level models of a wide variety of files, most other interfaces are very low level: gis.stackexchange.com/questions/109191/… – mdsumner Aug 13 '14 at 9:08
  • I guess I don't understand the core concept. I understand that it is a binary format, but why is it different to, let's use a csv of database format? – FredFury Aug 13 '14 at 9:23
  • The array format often has multiple dimensions, so you can slice or view, e.g. a 2D raster from any combination of times, heights, etc. – Mike T Aug 13 '14 at 9:41
  • How are the arrays then indexed? – FredFury Aug 13 '14 at 10:14
3

Oceanography and atmospheric sciences have large, interrelated multidimensional datasets, and it might be reasonable to think of NetCDF as an interoperable database format for storing data-mining-style extensible data cubes. The "time = UNLIMITED ; " dimension is often called a "record" dimension and identifies a dimension along which the data cube may be extended.

You posted a CDL of a file, which pretty much represents the full contents of the file.

1) The internal structure isn't columned, but a set of multidimensional array records parsed and accessed through a library. In your CDL, the vector of values after the "pressure = " line are the values of the "float pressure(time, level, latitude, longitude) ;" variable, unrolled and ordered with time varying slowest, and longitude varying fastest, so the '900' is the pressure at (longitude=-125,latitude=25,level=first level, and time=first time), with the subsequent entries scanning along longitudinally, etc... There's a lot of abstraction in the actual on disk format so that one can append a new timestep to the file without moving all the data around.

2) You can flatten portions of the file into a flat, columned, ASCII format with tools like 'ncks' from 'NCO', or many other language-specific tools (matlab, java, C, python,...) but the ability to have multiple arrays with differing dimensionality (for example, an additional variable defined as 'float surface_elevation(latitude, longitude)' makes a simple translation into CSV awkward.)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.