Does GDAL support multi-process ReadAsArray

Question

I have a very large TIFF file that I need to read and process block by block through ReadAsArray. If I have multiple processes calling this function at the same time, can I return the correct result?

Answer 1

Yes, it's no problem**. Just make sure you pass each process a filepath not gdal.Dataset object. That's not so much of a problem with multiple processes, as passing an open gdal.Dataset will just fail, but can cause issues if you try to use multiple threads:

One important point is that the same ​GDALDataset object should not be accessed by several threads at the same time. But of course, it is fine to use 2 different handles pointing to the same file in 2 threads.

You shouldn't try and write to the same output file from different processes though. Use your main process to farm out the tasks to the child processes, collect the results and write the results.

** Note that if your process is IO limited rather than CPU limited (e.g if the processing of each block is quick), you may find it faster to process sequentially in a single process rather than multiple processes.