I spent quite a bit of time looking on how to use the multiprocessing package, but couldn't find anything on how to use it inside a plugin in QGIS. I am developing a plugin that does some optimization for several elements. I would like to parallelize it.

I found a useful link on multi-threading inside a python plugin (http://snorf.net/blog/2013/12/07/multithreading-in-qgis-python-plugins/), but nothing on using the multiprocessing module, which might be easier?

I have been trying with a very basic example. I am only showing the run function from the plugin here:

def run(self):
    """Run method that performs all the real work"""
    # show the dialog

    # Run the dialog event loop
    result = self.dlg.exec_()

    # See if OK was pressed and run code
    if result:
        #Get number of cores
        nProcs = mp.cpu_count()

        #Start a Process
        p = mp.Pool(nProcs)

        #Define function     
            def cube(x):
                return x**3

        #Run parallel        
        results = p.map(cube, range(1,7))

When I run this code from the plugin in QGIS, it opens several QGIS windows, which then return errors (can't load layers, etc.).

What am I missing?

Do I need to start a worker first on another thread and then use multiprocessing there?

Or would we use another function from multiprocessing?

I am working under windows 7, using QGIS 2.10.

closed as too broad by PolyGeo Jul 27 '17 at 18:59

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • I'm not familiar with mp.Pool but have done a bit of multiprocesing in python, C# and C++... it's not easy. It sounds like your subprocesses don't have a reference to your application (QgisApplication) or are trying to modify the same resource at the same time; in C++ you would have a mutex to pass around the threads but your code is too simple for that - are you only trying to cube a number? – Michael Stimson Jul 28 '15 at 5:10
  • No, I am trying to do an optimization for several items, but essentially it would work the same, which is why I tested with a simple function first. Using "multiprocessing" in python directly is that simple, I am wondering how it can be adapted to work in python inside qgis. – Olivier Jul 29 '15 at 6:50