I am processing a bunch of GPS data using a Qgis python script. The processing takes a LONG time to execute and when it is executing I notice that only a single CPU is pinned at 100%, the remaining CPUs are idle.
Before I get too deep into the details of my issue, I am trying to figure out how to create several processes to increase CPU utilization/processing speed for a task that is CPU-bound.
reading this question here: https://stackoverflow.com/questions/15143837/how-to-multi-thread-an-operation-within-a-loop-in-python I moved the contents of a loop in my script into a function then attempted to call it in using this form:
executor = concurrent.futures.ProcessPoolExecutor(10) futures = [executor.submit(try_my_operation, item) for item in items] concurrent.futures.wait(futures)
When I executed my script I got an error in the python error panel:
AttributeError: module 'sys' has no attribute 'argv'
After googling around a bit I found that using this at the top of my script should be a work around:
if not hasattr(sys, 'argv'): sys.argv = ['']
When I execute my script now another instance of QGIS opens up and an error message with title "Private qgis.db" displays a message:
Creation of missing tbl_srs in the private qgis.db failed. table tbl_srs already exists
Click OK and another error message witht he same title pops up:
Update of view in private qgis.db failed. table vw_srs already exists
This then repeats its self 9 more times and I end up with 10 instances of QGIS open, no of which are actually doing anything...