For writing large raster files (e.g. satellite image GeoTIFFs) to disk, the throughput rate tends to be limited by the compression algorithm rather than by the raw disk IO. (Benchmarks: 1, 2.) This is good news because data compression (computation) is far easier to parallelise than file IO generally is. Can we take advantage of this in python?
When using the
rasterio python library, is there any way to use multithreading to write a large file more quickly? (For example, is there a way to set