1

I am trying to troubleshoot a threading problem. QGIS version is 3.4.

I want to move a processing-intensive task (importing xyz data, applying hillshading, and saving as TIFF) to a separate thread. I am using threading.Thread() because it is (...theoretically...) simple to implement. However, this stops it from executing fully. I get different results when I call the function directly and when I call it via 'threading.Thread()'.

Earlier I wrote a bit of test code in an effort to reproduce the problem, but it runs successfully.

I quote it here because it is concise and self-contained and it demonstrates the method - although it does not in itself demonstrate the problem.

So this works:

import os
import time

class Utils:
  @staticmethod
  def runInThread(func, ar, parallel=False):
    import threading
    t = threading.Thread(target=func, args=ar)
    t.start()
    if not parallel: t.join() # wait for it to finish before continuing
  #
# end class

def makeFiles(outputFolder,a):
  for i in range(0,10):
    a = outputFolder + '/' + str(i) + '.txt'
    with open(a,'w+') as file: # create text file
      time.sleep(0.25)
#

def killFiles(outputFolder,a):
  for i in range(0,10):
    os.remove(outputFolder + '/' + str(i) + '.txt')
    time.sleep(0.25)
#

a = 0
outputFolder = 'D:/Users/[xyz]/Desktop/xyz'

# BOTH OF THESE WORK:
if 1 == 1: # with threading
  Utils.runInThread( makeFiles, (outputFolder, a), False )
  Utils.runInThread( killFiles, (outputFolder, a), False )
else: # without threading
  makeFiles(outputFolder,a)
  killFiles(outputFolder,a)
#

My original implementation calls the same 'runInThread' function. It is not so easy to reproduce here because it is part of a larger project, but this is the relevant part:

def xyzToTIFF(self, t, outputFolder, keepXYZ):

  import os

  self.StripDuplicatesForRaster(t, False)
  self.SortXYZForRaster(t)
  l = Qfunc.ImportRaster(t) # import file, return layer 'l'
  layerName = l.name() # store for later use
  if not l is None:
    Qfunc.setNoEnhancement(l)
    Qfunc.setRenderHillshade(l) # hillshade
    Qfunc.SaveRaster(l, outputFolder) # save TIFF <-- We get to this point in both cases
    QgsProject.instance().removeMapLayer( l.id() ) # remove the xyz file
    if keepXYZ == False:
      os.remove(t) # delete the xyz tile file <-- Does not happen when using threading!
    lTIFF = QgsRasterLayer(outputFolder + '/' + layerName + '.tif', layerName)
    if not lTIFF is None:
      QgsProject.instance().addMapLayer(lTIFF)

The above runs, and execution completes, and it generates TIFF images. However, when I call it using Utils.runInThread() the .xyz raster files (which should get deleted, and are represented by the 't' argument in 'xyzToTIFF' above) remain in the folder, and no layers get added to the QGIS project.

If I run it without threading, it works as intended. The .xyz files get deleted, and the TIFFs are loaded into the project.

It is called like this:

for t in tiles:
  if 1 == 1: # threading
    Utils.RunInThread( self.xyzToTIFF, (t, outputFolder, False), False )
  else: # no threading
    self.xyzToTIFF(t, outputFolder, False)

I suspect the problem is that I am trying to reference the project instance from a different thread. I tried modifying the above code slightly to pass QgsProject.instance() into xyzToTIFF() but the result was unchanged.

Intuitively I can see that I probably cannot access the project instance from outside the original (UI) thread. Is this the case? If so, can I pass a reference to it somehow, or do I have to keep any manipulation of layers etc. within the original thread?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.