I have, for a while, been trying to make a terrain map using NASA's SRTM 3 dataset. I have a large (13 GB), geotiff projected to web mercator of all the elevation files provided. I have been able to make a color-relief Geotiff based on this one. The color-relief file was made in ~1 hour.
The problem I have is that the when I try to use the hillshading it is painfully slow, and produces around 50MB an hour, on a file that I suspect will be around 20GB. (which will take 10 days+)
I have come to a point where I might just let it run in the background for the time needed, but since we have power-outages somewhat frequently from people overloading the stove, this doesn't really seem feasible!
The command I am using is
os.system("gdaldem hillshade -compute_edges -combined -of GTiff -co BIGTIFF=YES -co compress=lzw -co predictor=2 -co TILED=YES -z 3 D:/gdal/rebase.tif D:/gdal/hill_projected.tif")
I have tried setting
--config CACHE_MAX and
wm, but they don't seem to bite, and the gdaldem process is still only using ~2% of the availabe memory.
I have also tried setting
--config GTIFF_NUM_THREADS=ALL_CPUS which didn't speed up the task either.
I have tried to split the hillshading up to different regions, to later combine them to export to tiles, but run into the same problem in the eurasia and islands region.