3

I've gone through a large number of forum threads and cannot make sense of how possible it is to easily run two tools at the same time in PyQGIS.

Basically what I'm wanting to do is:

def function1():
    processing.run("gdalsomething"...)
def function2():
    processing.run("gdalsomethingelse"...)

doAtTheSameTime( function1() , function2() )

I've tried the Python threading module (but learnt I was wanting multiprocessing, not multithreading?)

I've tried the multiprocessing module but that spits out an array of errors (including sys.argv not existing), perhaps relating to the fact that __name__ refuses to be __main__, and is __console__ instead (not that I know what that means)

Does anyone have a very simple example of running two QGIS tools at the same time in PyQGIS? (If it is even possible?)

6
  • 2
    Have you tried using subtasks? docs.qgis.org/3.28/en/docs/pyqgis_developer_cookbook/tasks.html Commented May 16, 2023 at 7:54
  • Yes I have, though I get an assertion error like in gis.stackexchange.com/questions/419922/…
    – Buff Fox
    Commented May 16, 2023 at 23:19
  • Also, I think it may be possible to run two QGIS tools at the same time by setting up multiple python scripts to run QGIS tools without QGIS being open (i.e passing a python script into "...\bin\qgis-ltr.bat"), though I was hoping to avoid having to do this...
    – Buff Fox
    Commented May 16, 2023 at 23:34
  • 1
    I was worried about splitting things apart too much with a new question. I think maybe this is all related. Looking at github.com/qgis/QGIS/issues/40472 made me think maybe this isn't currently possible unless you have two instances of QGIS open?
    – Buff Fox
    Commented May 17, 2023 at 23:41
  • 1
    @Taras instead of completely removing the user's solution from their question (yes, wrong place) without any respectful comment or notice, why not suggest to them to post it as answer? You just erase someone's hard work. Commented May 19, 2023 at 8:40

2 Answers 2

3

I strongly recommend you follow PyQGIS Developer Cookbook | 15. Tasks - doing heavy work in the background to create your tasks but below is the explanation of your error.


In Python functions are objects that you can pass around by name.

QgsTask.fromFunction requires you to pass a function object to it.

Your code currently passes the return values of your functions instead because you execute the functions (the ()s). In Python, functions return None if there is no explicit return statement with a different value.

So instead of getting a function object, QgsTask.fromFunction gets None and the assertion fails.

Try:

import random
from qgis.core import *

def one():
    processing.run("native:slope", {'INPUT':'...Dem.tif','Z_FACTOR':1,'OUTPUT':'...Slope1.tif'})
def two():
    processing.run("native:slope", {'INPUT':'...Dem.tif','Z_FACTOR':2,'OUTPUT':'...Slope2.tif'})

task1 = QgsTask.fromFunction('Please work', one)  # <- changed
task2 = QgsTask.fromFunction('Please work 2', two)  # <- changed

task1.addSubTask(task2)
QgsApplication.taskManager().addTask(task1)
2
  • 1
    Ok that makes sense that without the brackets you're not calling the function but simply referring to it. After that I also realised with what was in the link you sent that the functions must have a parameter called "task". Updated code above. Now when I look at my CPU usage I can see that they're definitely running in parallel, thanks.
    – Buff Fox
    Commented May 19, 2023 at 0:03
  • 1
    Awesome! The task is automatically passed to the function so that you can e.g. update the progress indicator from it or stop it if the user asked QGIS to interrupt the whole thing. To handle progression after a task has finished you can pass another function using the on_finished parameter. docs.qgis.org/3.28/en/docs/pyqgis_developer_cookbook/… Commented May 19, 2023 at 6:56
1

I've made some more progression on this.


INITIAL CODE UPDATE following on from bugmenot's help:

The below code works, and as far as I can tell this is the simplest possible way to run two tools at the same time in PyQGIS

Critically, the functions which are used in the QgsTask definition must have a parameter called task

from qgis.core import *

def one(task):
    processing.run("native:slope", {'INPUT':'...Dem.tif','Z_FACTOR':1,'OUTPUT':'...Slope1.tif'})
def two(task):
    processing.run("native:slope", {'INPUT':'...Dem.tif','Z_FACTOR':2,'OUTPUT':'...Slope2.tif'})

task1 = QgsTask.fromFunction('ItWorks', one)
task2 = QgsTask.fromFunction('ItWorks2', two)

task1.addSubTask(task2)
QgsApplication.taskManager().addTask(task1)

Please note that the script does not wait for addTask to complete, and simply powers on, hence why more advanced setups have handling to figure out when the task has completed


I've also attempted to try and run several tools in sequence, though with a set up such that I want one task to not start until after two other tasks have completed

def firstProcess(task):
    processing.run("qgis:rastercalculator", ...)
    time.sleep(0.1)

def secondProcess(task):
    processing.run("qgis:rastercalculator", ...)
    time.sleep(0.1)

firstProcessTask = QgsTask.fromFunction('FirstProcess', firstProcess)
secondProcessTask = QgsTask.fromFunction('SecondProcess', secondProcess)

firstProcessTask.addSubTask(secondProcessTask)
QgsApplication.taskManager().addTask(firstProcessTask)

    
####################################


def thirdProcess(task):
    try:
        firstProcessTask.waitForFinished(timeout = 20000000)
    except BaseException as e:
        print("...")
    try:
        secondProcessTask.waitForFinished(timeout = 20000000)
    except BaseException as e:
        print("...")
    processing.run("qgis:rastercalculator", ...)   

thirdProcessTask = QgsTask.fromFunction('ThirdProcess', thirdProcess)
QgsApplication.taskManager().addTask(thirdProcessTask)

It's a little clunky, but it's the best I've been able to do given that if two tasks are run together through addSubTask they appear to trigger task.waitForFinished independently

If anyone has any improvements that would be great

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.