Below are three GDAL supported raster map processing commands, set inside a Bash shell helper function, defined inside a script that'd patch ERA5-Land with ERA5 data (see also: Merge ERA5-Land and ERA5 time series):
function gdalmerge_and_clean() {
VARIABLE=$1
YEAR=$2
BAND=$3
if [ ! -f "era5_and_land_${VARIABLE}_${YEAR}_band_${BAND}_patched.nc" ] ;then
if [ ! -f "era5_${VARIABLE}_${YEAR}_band_${BAND}.nc" ] ;then
gdal_translate \
-if netCDF \
-of netCDF \
-ot Float32 \
-tr 0.1 0.1 \
-unscale \
-b $BAND \
$ERA5_DATA_PATH/consolidated/${YEAR}/${YEAR}_cds_era5_${VARIABLE}.nc \
era5_${VARIABLE}_${YEAR}_band_${BAND}.nc
fi
if [ ! -f "era5_land_${VARIABLE}_${YEAR}_band_${BAND}.nc" ] ;then
gdal_translate \
-if netCDF \
-of netCDF \
-ot Float32 \
-unscale \
-b $BAND \
$ERA5LAND_DATA_PATH/cds_era5_land_${VARIABLE}_${YEAR}.nc \
era5_land_${VARIABLE}_${YEAR}_band_${BAND}.nc
fi
gdal_merge.py \
-of netCDF \
-ot Float32 \
-o era5_and_land_${VARIABLE}_${YEAR}_band_${BAND}_patched.nc \
era5_${VARIABLE}_${YEAR}_band_${BAND}.nc \
era5_land_${VARIABLE}_${YEAR}_band_${BAND}.nc
rm -rf era5_${VARIABLE}_${YEAR}_band_${BAND}.nc era5_land_${VARIABLE}_${YEAR}_band_${BAND}.nc ;
fi
}
Questions
How can I use in-memory (raster map) Python objects and GDAL's Python API to replicate the above commands?
While the
gdal_translate
commands seem easy to implement (after so many Q&As that demonstrate it), is it possible at all to use in-memory object withgdal_merge
in a Python script?The source files come in form of netCDF files. Would the use of the netcdf4-python Python interface to the netCDF C library to read-in data speed-up processing?
References
- https://gdal.org/api/python.html
- https://gis.stackexchange.com/a/236754/5256 : gdal_merge
- Is there a reason I should use gdal's Python library instead of just using os.system(gdal command)? : Why use GDAL's Python API
- https://gis.stackexchange.com/a/346716/5256 : gdal_translate, GDAL, Python
- https://gis.stackexchange.com/a/264798/5256 : VSIMEM filesystem (http://www.gdal.org/gdal_virtual_file_systems.html)
- https://gis.stackexchange.com/a/304762/5256 : gdal_translate, gdal.Translate
- https://gis.stackexchange.com/a/304336/5256 : gdal.Translate
- https://gis.stackexchange.com/a/351924/5256 : In-memory file, Create()
- https://gis.stackexchange.com/a/371863/5256 : gdal.Translate, Creation Options
- https://www.programcreek.com/python/example/85545/gdal.GDT_Float32 : gdal.GDT_Float32
- https://gis.stackexchange.com/a/290782/5256 : read/write numpy array, GDAL, Python
- https://gis.stackexchange.com/a/129868/5256 : rasterio