3

Is it possible to run a standalone PyQGIS script using multiprocessing for better performance?

I'm creating 1,800 rendered map images and would like to run it across 4 processes to speed it up.

If I try to instantiate 4 QgsApplication objects using the following code (one in each process) - they fail silently (i.e. nothing happens and no CPU/hard drive/memory is being used).

QgsApplication.setPrefixPath("/Applications/QGIS3.8.app/Contents/MacOS", True)
app = QgsApplication([], False)
app.initQgis()

If I instantiate it once and then run the 4 processes - I get a "QgsApplication couldn't be pickled" error.

The code uses Python's multiprocessing module to create the parallel processes; using pool.imap_unordered

I'm using QGIS 3.8.2 and Python 3.7.3 on a Mac with MacOS 10.14.6 (Mojave).

Improve this question
2
  • 1
    Perhaps this post might help: gis.stackexchange.com/questions/330255/…
    – Joseph
    Commented Sep 10, 2019 at 10:03
  • Good to know it can be done using bash scripts, however I want to keep in all in a single Python script for simplicity.
    – minus34
    Commented Sep 18, 2019 at 3:42

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.