I'm trying to convert variables from NetCDF files to tabular data, but can't understand how to group observations by some index or key variable.
For example, when loading data from WOD queries using the xarray library
import xarray as xr
ds = xr.open_dataset(r'ocldb1573479266.24760_OSD.nc')
ds
I get something like
<xarray.Dataset>
Dimensions: (Alkalinity_obs: 32749, Ammonia_obs: 15252, CFC11_obs: 16256, CFC12_obs: 16317, Chlorophyll_obs: 58045, DeltaC14_obs: 1456, DeltaHe3_obs: 1929, Helium_obs: 2143, Neon_obs: 377, Nitrate_obs: 108954, Oxy18_obs: 409, Oxygen_obs: 300929, Phosphate_obs: 183091, Pressure_obs: 581435, Salinity_obs: 581435, Silicate_obs: 167977, Temperature_obs: 581435, Tritium_obs: 2015, biosets: 1149, casts: 51401, numberofpis: 6079, pCO2_obs: 8513, pH_obs: 42365, tCO2_obs: 24672, z_obs: 581435)
Coordinates:
lat (casts) float32 ...
lon (casts) float32 ...
time (casts) datetime64[ns] 1890-06-27T02:00:00.000214528 ... 1988-03-05
z (z_obs) float32 ...
Dimensions without coordinates: Alkalinity_obs, Ammonia_obs, CFC11_obs, CFC12_obs, Chlorophyll_obs, DeltaC14_obs, DeltaHe3_obs, Helium_obs, Neon_obs, Nitrate_obs, Oxy18_obs, Oxygen_obs, Phosphate_obs, Pressure_obs, Salinity_obs, Silicate_obs, Temperature_obs, Tritium_obs, biosets, casts, numberofpis, pCO2_obs, pH_obs, tCO2_obs, z_obs
Data variables:
country (casts) |S40 ...
WOD_cruise_identifier (casts) |S40 ...
(. . .)
Temperature (Temperature_obs) float32 ...
Temperature_sigfigs (Temperature_obs) int8 ...
Temperature_row_size (casts) float64 7.0 6.0 7.0 ... 58.0 45.0
Temperature_WODflag (Temperature_obs) int8 ...
Temperature_origflag (Temperature_obs) float32 ...
Temperature_WODprofileflag (casts) int8 ...
Temperature_Access_no (casts) float64 ...
Temperature_Scale (casts) |S170 ...
Temperature_Instrument (casts) |S170 b'' b'' b'' ... b'' b''
(. . .)
When inspecting shapes
print('CASTS : ', ds.casts.to_dataframe().shape )
print('Temperature : ', ds.Temperature.to_dataframe().shape )
print('Temperature_obs: ', ds.Temperature_obs.to_dataframe().shape )
print('time : ', ds.time.to_dataframe().shape )
I get the following
CASTS : (51401, 4)
Temperature : (581435, 1)
Temperature_obs: (581435, 1)
time : (51401, 4)
So, as expected, it seems for each cast
there is a set of Temperature
observations. But I'm not familiar with xarray (or NetCDF4-python) methods for mask, group, merge, etc. So my question is: how can I create something like this?
| | cast | lat | lon | temperature | ... |
|:---:|:----:|:---:|:---:|:-----------:|:---:|
| 0 | 0 | | | | |
| 1 | 0 | | | | |
| 2 | 0 | | | | |
| ... | | | | | |
| 21 | 0 | | | | |
| 22 | 1 | | | | |
| 23 | 1 | | | | |
I already tried to use to_dataframe()
method, but it did not work.