It seems like processing a large raster dataset in QGIS can cause a crash, for example in QGIS failing to complete clip or intersect and crash dump in qgis. Is it possible to use GDAL parallel processing for a large raster dataset? Is there any example for this?
-
1This is a useful thread on the topic– dmciCommented Jan 28, 2016 at 21:39
-
Whenever I need to process large amounts of data I run gdal from the command line. Check gdal.org for info, examples and a couple tutorials to get you comfortable using it from the command line.– sig_renatoCommented Jan 28, 2016 at 21:46
-
That's right but when you want use in plugin, what do you do?– HMadadiCommented Jan 29, 2016 at 11:28
-
I have a bit of a plugin fobia… :(– sig_renatoCommented Jan 29, 2016 at 14:35
-
this does not really answer the question whether gdal can be used in parallel...– joaoalCommented Nov 13, 2017 at 10:30
|
Show 3 more comments
1 Answer
GDAL can use multiple threads for some processing. For example gdal2tiles https://gdal.org/programs/gdal2tiles.html can create base tiles in multiple processes. Gdalwarp https://gdal.org/programs/gdalwarp.html can also use multiple processes.
When it comes to clipping, I am not sure if multithreading could yield big benefits. Clipping is rather simple process, to find correct source pixels and to write them into a target file. Multithreading would probably require that it is possible to write data into several places of the target file. Perhaps it could be possible with very well known format like uncompressed tiff. But I think that there is now something in QGIS that is using unnecessary much resources and multithreading is not the right place to start with. For example, is your target file tiled?
