I'm currently working with a large number of CF-compliant NetCDF climate grids with global coverage that are organized as tiles across a file structure pertaining to particular "variables" and "years" (e.g.
tmax-year2008-650.nc, etc.). They're currently organized in a sensible file structure (:project/variable/year/files.nc)
For each variable*year, I would like to spatially merge the tiles (similar in call to
gdal_merge outputfile.tif /path/to/data/*.tif) using a walking method and I'm wondering what would be the most efficient/reliable way of doing so.
From my research, two (labour-intensive) methods come to mind:
Performing a walking loop in BASH that calls the Climate Data Operators (CDO) function
mergegridcan only accept two input files for each output, even if I were to merge them efficiently, I would need to call it > 400x for each variable*year.
Performing a walking loop in Python with the
NumPylibraries that creates an empty array the size of the the Earth, which reads each NetCDF to an
np.arrayand writes it out to the new NetCDF, appends the metadata, and iterates through the variables*years.
Ideally, I need a method that is easily replicated after I've written the function out and if I can stay in Python3, that would be great. What I would like to know is which is the better way of merging lots of nested NetCDF data or is there another method/modules that I haven't come across that would be better at performing large spatial merges of NetCDF data.