I am trying to troubleshoot a threading problem. QGIS version is 3.4.
I want to move a processing-intensive task (importing xyz data, applying hillshading, and saving as TIFF) to a separate thread. I am using threading.Thread() because it is (...theoretically...) simple to implement. However, this stops it from executing fully. I get different results when I call the function directly and when I call it via 'threading.Thread()'.
Earlier I wrote a bit of test code in an effort to reproduce the problem, but it runs successfully.
I quote it here because it is concise and self-contained and it demonstrates the method - although it does not in itself demonstrate the problem.
So this works:
import os import time class Utils: @staticmethod def runInThread(func, ar, parallel=False): import threading t = threading.Thread(target=func, args=ar) t.start() if not parallel: t.join() # wait for it to finish before continuing # # end class def makeFiles(outputFolder,a): for i in range(0,10): a = outputFolder + '/' + str(i) + '.txt' with open(a,'w+') as file: # create text file time.sleep(0.25) # def killFiles(outputFolder,a): for i in range(0,10): os.remove(outputFolder + '/' + str(i) + '.txt') time.sleep(0.25) # a = 0 outputFolder = 'D:/Users/[xyz]/Desktop/xyz' # BOTH OF THESE WORK: if 1 == 1: # with threading Utils.runInThread( makeFiles, (outputFolder, a), False ) Utils.runInThread( killFiles, (outputFolder, a), False ) else: # without threading makeFiles(outputFolder,a) killFiles(outputFolder,a) #
My original implementation calls the same 'runInThread' function. It is not so easy to reproduce here because it is part of a larger project, but this is the relevant part:
def xyzToTIFF(self, t, outputFolder, keepXYZ): import os self.StripDuplicatesForRaster(t, False) self.SortXYZForRaster(t) l = Qfunc.ImportRaster(t) # import file, return layer 'l' layerName = l.name() # store for later use if not l is None: Qfunc.setNoEnhancement(l) Qfunc.setRenderHillshade(l) # hillshade Qfunc.SaveRaster(l, outputFolder) # save TIFF <-- We get to this point in both cases QgsProject.instance().removeMapLayer( l.id() ) # remove the xyz file if keepXYZ == False: os.remove(t) # delete the xyz tile file <-- Does not happen when using threading! lTIFF = QgsRasterLayer(outputFolder + '/' + layerName + '.tif', layerName) if not lTIFF is None: QgsProject.instance().addMapLayer(lTIFF)
The above runs, and execution completes, and it generates TIFF images. However, when I call it using Utils.runInThread() the .xyz raster files (which should get deleted, and are represented by the 't' argument in 'xyzToTIFF' above) remain in the folder, and no layers get added to the QGIS project.
If I run it without threading, it works as intended. The .xyz files get deleted, and the TIFFs are loaded into the project.
It is called like this:
for t in tiles: if 1 == 1: # threading Utils.RunInThread( self.xyzToTIFF, (t, outputFolder, False), False ) else: # no threading self.xyzToTIFF(t, outputFolder, False)
I suspect the problem is that I am trying to reference the project instance from a different thread. I tried modifying the above code slightly to pass QgsProject.instance() into xyzToTIFF() but the result was unchanged.
Intuitively I can see that I probably cannot access the project instance from outside the original (UI) thread. Is this the case? If so, can I pass a reference to it somehow, or do I have to keep any manipulation of layers etc. within the original thread?