Questions tagged [parallel-processing]

Use this tag when asking questions about multi or parallel processing - a programming paradigm used to speed up computationally intensive geoprocessing tasks by executing them over multiple CPU cores or threads.

15 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
5
votes
0answers
884 views

Is it possible to do parallel processing in GDAL and QGIS?

It seems like processing a large raster dataset in QGIS can cause a crash, for example in QGIS failing to complete clip or intersect and crash dump in qgis. Is it possible to use GDAL parallel ...
3
votes
0answers
265 views

Multiprocessing with geopandas

I am learning how to implement the multiprocessing with spatial data using the module multiprocessing. I am trying to implement a spatial intersection between a polygon file and a grid file. I am ...
3
votes
0answers
348 views

Multiprocessing in python qgis?

When i try to do mutliprocessing in python Qgis it doesn't work from multiprocessing import Process,Lock import multiprocessing def name1(l,i): print i if __name__=='__main__': lock=Lock() ...
2
votes
0answers
66 views

Parallel optimization for query over LiDAR dataset

I'm using PostGIS + pointcloud to store a huge dataset of LiDAR point (2M patches of 600points) I query this database to obtain the altimetic profile between two points. At the moment I have a huge ...
2
votes
0answers
68 views

Avoiding edges using GRASS generalization?

I'm trying to process the national wide land cover data into vector polygons. In order to be quick I separate the data into several tiles and process them in parallel. process flow is : 1. v.in.ogr ...
1
vote
0answers
33 views

Multiprocess Spatial Join with Spatial Index

I'm a newbie in implementing GIS on Python, but the lack of any GIS software at the office has forced me to learn about it. I have a huge grid (a multipolygon) that I'm currently filling out with ...
1
vote
0answers
41 views

Arcpy multiprocessing arcpy.Select_analysis

I have the following script: import arcpy from arcpy import env import datetime import os env.workspace = r"Q:\\SDE\Direct Connection to Contours.sde" dataFC = arcpy.ListFeatureClasses("","","...
1
vote
0answers
214 views

asyncio with multiprocessing

I am trying to run a multiprocessing on a zonal statistic in pure Python. I am new on it. I have to apply a zonal statistic with a shapefile with more than 6 million features on a taster of 6 GBs (...
1
vote
0answers
247 views

Spatial analysis and parallel processing in R

I am trying to do a point-in-poly analysis using R. I have 4.5 million points and 5k polygons. This is computationally intensive. I was pointed to parallel processing through another post and tried to ...
1
vote
0answers
15 views

Multiprocessing GALayerToRasters

I'm new to ArcPy and am creating a script to automate some interpolation work. I have multiple GA Layers outputted by an Empirical Bayesian Kriging interpolation (using arcpy....
1
vote
0answers
39 views

Sending commands into FGDB

I have a large .gdb of all blocks in the United States, available from Census. I would like to create a point shape file of all census tract centroids in the US. I have QGIS with FileGDB enabled, as ...
0
votes
0answers
21 views

ExecuteError when loading rasters using ArcPy Mosaic to Raster tool and Python multiprocessing

I'm attempting to merge thousands of rasters into a smaller more manageable number using the Mosaic to NewRaster_management tool in ArcPy. The code I've written works just fine when I run it the ...
0
votes
0answers
90 views

arcpy and multiprocessing: switch workflow to shapefiles to avoid schemalocks

I need to processe a large dataset through a bunch of interwoven scripts using arcpy. On a small test dataset my workflow functions fine so far but when it comes to the large dataset i see no other ...
0
votes
0answers
262 views

Multiprocessing Script Tool doesn't crash but will not function

I've made created a script that uses multiprocessing to break up a very large point dataset into pieces and extract values from 2 different rasters to them. When I tested it using the command line ...
0
votes
0answers
167 views

A Distributed Cost-Distance Algorithm

I'm currently trying to implement a distributed cost-distance algorithm, however I can't seem to invent my own and still make it fast. I've been googling around a bit but haven't found anything that ...