Questions tagged [multithreading]
Sharing a single CPU between multiple tasks (or "threads") in a way designed to minimise the time required to switch threads.
48 questions
1
vote
1
answer
137
views
Opening rasterio datasets in threads
I have an application that requires opening a lot of GeoTIFFs, and I would like to do it concurrently. Basically, I am calling rasterio.open() in each thread and storing the resulting DatasetReader ...
- 13
1
vote
0
answers
158
views
Using dask.distributed with rioxarray rio.to_raster results in `ValueError: Lock is not yet acquired` [closed]
I am trying to write some code using dask.distributed.Client and rioxarray to_raster that:
Concatenates two rasters (dask arrays)
Applies a function across all blocks in the concatenated array
Writes ...
- 31
0
votes
1
answer
43
views
Missing Outputs with ArcPy RasterToASCII and ThreadPoolExecutor
When I use ThreadPoolExecutor, I am missing a random number of expected outputs. I tried making an MRE to show what I am trying to do. The first MRE works but when I run a second MRE (which is closer ...
- 916
2
votes
0
answers
186
views
Using ThreadPoolExectuor doesn't work with ArcPy
I have a Python function that downloads a geopackage from somewhere then does some stuff with it and at some point converts it to an ESRI file geodatabase.
Now inside a python script I want to use ...
- 21
1
vote
1
answer
403
views
Conditional signals in QgsTask managing algorithm output
Last question from my QgsTasks questions trilogy...
I'd like to insert a new processing algorithm with a raster as an input (let's say 'fill sinks' for instance) to the code provided by @Ben W in his ...
- 341
2
votes
1
answer
554
views
Sharing class variables across thread (QgsTasks) with pyQGIS
In keeping with Setting up thread with busy progress indicator or GIF in PyQGIS where I asked about thread and busy indicator I now wish to ask understand how it is possible to use a variable declared ...
- 341
5
votes
1
answer
653
views
Setting up thread with busy progress indicator or GIF in PyQGIS
As discussed in the comments of this question I have troubles understanding how to to set up a thread with a busy progress bar (or a GIF) within a plugin in QGIS.
I would like the progress bar to turn ...
- 341
0
votes
0
answers
95
views
Utilise more cores for ArcPy script
Trying to optimise a script that I’ve made up to run a simple process of converting a cadastre (polygon) in a file geodatabase to a line feature class and then run some further processing on it. It’s ...
user203497
1
vote
0
answers
148
views
Multithreading single function using ArcPy with ArcMap
I am using ArcMap 10.5.
The problem
I have a huge DEM raster (more than 6 GB in size). I need to process this raster through Hydrology toolset in ArcGIS (fill, flow direction, flow accumulation...etc)....
2
votes
0
answers
246
views
PyQGIS task manager not working
I'm trying to do simple thing as running plugin that load layers and show loading bar at the same time as the loading process takes awhile.
I'm trying to use QGIS task manager with no success..
This ...
- 173
0
votes
0
answers
235
views
Realizing arcpy.mapping.MapDocument("CURRENT")
I am curious as to how arcpy.mapping.MapDocument("CURRENT") works. I used to execute highlight in the sub thread, but didn't work.
Does it only work in the main thread (including arcpy....
0
votes
0
answers
92
views
arcpy.mapping.MapDocument("CURRENT") not working in sub thread
I want to create a customize tool in ArcMap by Python Add-in wizards.I found that arcpy.mapping.MapDocument("CURRENT") does not work in the sub thread.
How can I call main thread to execute ...
7
votes
1
answer
263
views
GIF freezes when QGIS plugin runs
I have a QGIS plugin and would like to put loading GIF (spinner) somewhere in the Dialog Window. The problem is when I run the spinner with the following code and call next function, the spinner ...
3
votes
0
answers
182
views
Multi-computer distributed operations using QGIS
Although QGIS is multithreaded for some operations, There are many cases where I run large operations for cumulative viewsheds which take an extremely long time to calculate.
Is there a process ...
- 491
1
vote
1
answer
442
views
QgsTask: finished-function isn't executed
I am writing a plugin for QGIS. I created a task by subclassing QgsTask, it's basically copied from the PyQGIS Cookbook and looks like this:
import json
import urllib.request
from qgis.PyQt.QtCore ...
- 71
3
votes
0
answers
260
views
How do I run a constant background thread in a QGIS Plugin using Python?
I'm working on a plugin that involves parsing information, and then mapping it. I have a simple gui that allows me to start, stop and choose a file to parse.
I'm trying to separate the parsing ...
4
votes
1
answer
284
views
QgsVectorLayer creation in Qthread (PyQt)
Our QGIS project is dynamically generated on startup based on user roles. There are nearly 120 QgsVectorLayers that need to be generated. If created sequentially, the login process takes up to 2 ...
- 423
3
votes
1
answer
382
views
Using QObject inside thread in PyQGIS
In PyQGIS Cookbook, says:
Any background task (regardless of how it is created) must NEVER use
any QObject that lives on the main thread, such as accessing
QgsVectorLayer, QgsProject or perform any ...
- 445
1
vote
2
answers
189
views
QGIS loading a layer every 5 seconds (Threading)
Im trying to load a point layer from a CSV-file with a QGIS Plugin every 5 seconds. The Coordinates in this CSV-File which are used to locate the point layer are changed every few seconds.
I tried to ...
- 102
6
votes
0
answers
595
views
Open multiple QGIS projects via PyQGIS simultaneously in different threads
I'm writing a standalone PyQGIS application. When the method qgis() is called in a new thread, it's supposed to open a QGIS project (with one layer in it) and close it again:
from concurrent.futures....
- 161
7
votes
2
answers
2k
views
Definitive list of QGIS multiprocessing tools?
This question is not as straightforward as it seems, but can be summed up with the following:
Is there a definitive list of geoprocessing tools for QGIS that currently support multithreaded ...
- 300
2
votes
0
answers
110
views
Horizontal color lines artifacts / bug with GDAL Tiff
I would like to document a bug I encountered, and scratched my head against several hours:
I wrote a python script for extracting (cropping) many images from a big TIF image. I used both gdal....
- 181
2
votes
0
answers
152
views
Write Excel From ArcGIS Pro, Async
I am having trouble writing to an excel file from ArcGIS Pro Sdk add-in that I created, I have a basic understanding of multithread, I am using task but the main thread does not wait for the excel to ...
- 414
1
vote
0
answers
166
views
pyqgis - Code stops working when called using threading.Thread()
I am trying to troubleshoot a threading problem. QGIS version is 3.4.
I want to move a processing-intensive task (importing xyz data, applying hillshading, and saving as TIFF) to a separate thread. I ...
- 419
4
votes
0
answers
473
views
ogr2ogr manipulating cpu and memories
My OGR2OGR command is working very hard and I am kind of satisfied but not as fast as I expected.
What I am doing is Joining a table to a fgdb feature class and save it to another fgdb.
through my mac ...
- 1,050
2
votes
0
answers
375
views
Custom QGIS maptool to return current event
This is what I am trying to do.
I designed an interface which can switch between selection mode and placement mode.
Selection mode activates the selection tool. Placement mode is what I am ...
- 724
2
votes
1
answer
720
views
pyqgis threading - Exception when using QgsTask.fromFunction
I am trying to implement threading to some pyqgis code.
I have been trying to use QgsTask.fromFunction.
I have implemented code from this article but it isn't working. I used the first of the three ...
- 419
1
vote
0
answers
501
views
gdal cannot work when using openMP in c++ for reading a dataset?
void generate( GDALDataset *ds)
{
int tcount = (1 + abs(tmaxx - tminx))*(1 + abs(tmaxy - tminy));
GDALDriver* jpegDriver = GetGDALDriverManager()->GetDriverByName("PNG");
int tz = ...
- 113
2
votes
0
answers
114
views
Enable multiple GDAL threads for qgis2web?
I'm using qgis2web plugin for a project that has a raster base map and many layers. It works well but exporting is slow, taking 16 minutes. gdalwarp.exe seems to use only one thread (the CPU has eight)...
- 21
3
votes
1
answer
1k
views
QGIS plugin multithreading and loading memory layers
I've been working on a small plugin recently and I wanted to add a progress bar to it, since there are tasks in this plugin that take some time to complete. I looked at this and this, but so far I've ...
- 61
3
votes
0
answers
143
views
Will core functionality in QGIS 3.0 exploit multi-threading? [duplicate]
As mentioned in a recent QGIS blog by Tim Sutton, the major changes to QGIS 3.0 are:
Updating Qt4 to Qt5
Updating PyQt4 to PyQt5
Updating Python 2.7 to Python 3
Improving the QGIS API
Also, a couple ...
- 76.3k
4
votes
0
answers
1k
views
QGIS crash using QThread in a plugin script
I recently wrote a Python script that works with raster data and makes some time-consuming operations. My script uses QThread to update a QList and a QProgressBar and everything works fine if I run ...
- 41
7
votes
1
answer
12k
views
Is QGIS multi-threaded? [duplicate]
Is QGIS (2.16) multi-threaded?
I do not see multiple processes on the console while running it on Ubuntu.
It only uses one CPU.
Is there anything to do to enable it?
- 10.9k
2
votes
1
answer
906
views
Using PyQGIS to create images from the QGIS print composer each showing a different layer?
I have many vector layers which I want to display and print, one at a time. The following script works well for producing an image from the map canvas, but I want to produce an image from a print ...
- 53
2
votes
0
answers
482
views
Is GDAL/OGR (specifically CoordinateTransformation) thread-safe?
I'm using the Java bindings if it matters. I have a heavily multi-threaded program and this is one of the bottlenecks that I have in a synchronization block. If I could remove the synchronization, it ...
- 559
3
votes
1
answer
798
views
QGIS not interacting with iface from python class
So I'm working on something where i need to use threads from python console of QGIS.
The following line of code adds layer successfully
iface.addVectorLayer("D:\ArcGIS Misc\Atlas\points.shp", 'Points'...
- 323
2
votes
0
answers
1k
views
CPUs to process lidar data using ArcGIS for Desktop? [closed]
I am looking to build a computer to process data in ArcGIS 10.2.2.
The main use will be to rasterize 10-100's GBs of lidar data.
However, I am unclear as to the capabilities of ArcGIS 10.2.2 to ...
- 713
1
vote
1
answer
354
views
ArcMap Crashes when inserting new feature in multi-Threading add-in
I'm developing a ArcMap add-in that get dynamic feeds from the network and append the newly arrived data to a suitable featurelayer.
I'm using InMemoryWorkspace, FeatureBuffer, FeatureCursor, Thread.
...
- 395
11
votes
2
answers
2k
views
Will core functions in QGIS exploit multi-threading?
After reading Enabling multithreaded processing in QGIS?, I was wondering if QGIS 2.6 will include this when running various processes. I checked online and I could only find that multi-threaded ...
- 76.3k
0
votes
1
answer
263
views
ArcGIS / .Net - Load pyramids in thread
I need to display a large raster with ArcGIS. To do so, I'm using pyramids, and it is working, but they can be long to build. So I would like to build the pyramids in a separate thread. When doing so, ...
- 604
3
votes
0
answers
823
views
Why does multi-threaded rendering not use all the CPU potential in QGIS 2.4?
I have just downloaded QGIS 2.4 which is great and ran some tests. I have enabled the multi-threaded rendering but it seems that the program do not use all the potential of my CPU. Attached pic shows ...
- 41
14
votes
3
answers
11k
views
Enabling multithreaded processing in QGIS?
I have found multiple links that appear to say that this now exists in QGIS 2.2, but it appears that it's only using one core to full capacity.
This is running on Ubuntu 14.04
The current process is ...
- 495
4
votes
1
answer
2k
views
Multithreading in Arcpy
I am learning multithreading and thinking about using it with Arcpy. The example I am working on doesn't have much practical reasons. However, I don't know why it fails on the line "arcpy.Exists(...
- 287
2
votes
1
answer
998
views
Adding FeatureLayer on thread crashes ArcMap
I am creating an extension for ArcMap 10.2 using Visual Studio 2012 C# WPF. Consider this code snippet:
//Open the CSV file
IWorkspaceFactory WorkspaceFactory = new TextFileWorkspaceFactoryClass();
...
- 173
1
vote
0
answers
186
views
qgis 1.8.0 crash when starting qthread
I am trying to update a vector layer attribute table using QThread.
i connect a pushbutton click with a slot to start the thread:
def EGSupdateAttributeTable(self):
AttUpdateEGSThread = ...
- 1,013
6
votes
3
answers
882
views
What is the algorithm used by ESRI for finding loops in a geometric network?
I've been trying to develop some code for ArcMap (VB .net) that would take a river network, identify and label the loops. These loops can be simple bifurcations or highly thread braided channels.
I ...
- 44.6k
2
votes
0
answers
907
views
Problem with multi-threading and geoprocessor tool in c# [closed]
What I am doing: I am trying to run a geoprocessor tool in a background thread other than UI thread. I am passing string path of arcObjects and creating that arcObject in my background thread. Then I ...
- 2,425
3
votes
2
answers
2k
views
Accessing ArcGIS layers from separate thread using ArcGIS Engine?
I've got an ArcEngine application, referencing a MXD for it's data (and map display). I'm running a query against all layers, so can take quite some time.
I would normally run the query in a ...
- 1,133