3

E.g. in SAGA GIS it is possible to set with one click the number of CPU cores to use during processing.

Is this also possible in Grass GIS?

3

GRASS hasn't been fully optimised with running in parallel as it doesn't lock files which are being processed. From the GRASS wiki:

GRASS doesn't perform any locking on the files within a GRASS database, so the user may end up with one process reading a file while another process is in the middle of writing it. The most problematic case is the WIND file, which contains the current region, although there are others.

If a user wants to run multiple commands concurrently, steps need to be taken to ensure that this type of conflict doesn't happen.

However, there is the GRASS Partial Differential Equations Library (GPDE) which is designed to work with OpenMP and is capable of multi-threading.

There has also been some development in the gpde and the gmath libraries where certain functions benefit from multi-core systems:

  • 1
    What a pity! My script would definitely be much faster if there was this opportunity. Thanks, however, for your answer! – countryman Dec 2 '15 at 6:13
  • @countryman - Most welcome buddy and yes, bit of a shame but they are still developing it so it may be possible soon hopefully :) – Joseph Dec 2 '15 at 10:29
5

There are many approaches to parallelization. Some GRASS GIS modules are parallelized internally using OpenMP or pthreads when GRASS GIS is compiled in the way that these are supported. This applies to modules written in C, the parallelized modules written in Python are using different Python ways for parallelization based on processes. These modules usually have option nproc to set the number of processes to use. Example of this is r.sun.daily in GRASS GIS Addons repository (note nproc=4):

r.sun.daily elevation=elev_lid792_1m glob_rad_basename=daily_rad \
    start_day=255 end_day=302 -t nproc=4

Some modules are not parallel because it is hard or impossible to parallelize the process or because the parallelized version doesn't give any performance advantage due to overhead related to handling different threads or processes. There are also other limits you can hit with parallelization, namely limited memory (RAM) or speed of I/O operations (which are usually not parallel).

In the comment to Joseph's answer, you are doing some scripting. This is really the best place to parallelize according to what I know. You can parallelize keeping in mind the particular analysis you are doing and you can even consider the given data and hardware.

Python has several ways of to help you with parallelizing things. GRASS GIS itself offers tools specialized for GRASS GIS and geospatial tasks, namely ParallelModuleQueue and GridModule:

GRASS wiki contains suggestions how to do parallelization yourself in Python and in Bash:

In Bash, the basic parallelization on the level of processes trivial: just adding & at the end of commands and using wait to connect the processes back:

sleep 10 &
sleep 10 &
sleep 10 &
sleep 10 &
wait

This causes four processes, here just dummy sleep for 10 seconds, to run in parallel and when they all are finished, the execution continues after the line with wait. Without doing some extra steps, this simple approach (unlike for example GridModule) won't work when you want to divide the area spatially, but it will work well if you need to compute, e.g. slope and aspect on 4 different digital elevation models.

Read also Let’s Get Parallel! by Graeme Bell (a presentation from FOSS4G-E Como) which gives a lot of good insights to parallel processing from the point of view of GIS and scripting:

  • I think this should be the accepted answer as it's far more informative than what I posted! +1 – Joseph Dec 14 '15 at 12:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.