Skip to main content
19 votes
Accepted

Explaining long computing time of Python code

Here are some issues with your script: I've already learned that using GeoJSON for analyzing is a too bad idea. The script below that takes 20 seconds when I use GeoPackage takes about 8-10 hours ...
Kadir Şahbaz's user avatar
18 votes

Finding nearest neighbor for each point in the same table

This is essentially a duplicate question of multiple others, with the sole difference being a table self-join. However, all queries currently present in this post have delicate CRS misunderstandings, ...
geozelot's user avatar
  • 30.4k
16 votes

QGIS slow performance with PostgreSQL/PostGIS

This is a broad question... here are a few things I have gathered through the years: Make sure the geom/geog column is using a single type (ex avoid geometry but instead use geometry(point,4326)). ...
JGH's user avatar
  • 42.3k
15 votes

Faster calculation of cumulative sum in attribute table in QGIS

You can use the script below in QGIS Python Editor. It takes about 45 seconds (including saving to the source file) for 1.5 million features in SSD or HDD. Select the layer first. import time from ...
Kadir Şahbaz's user avatar
14 votes
Accepted

QGIS form performance

It seems to be a bug in QGIS that has existed for some time. I've tested versions 2.18, 3.4 and 3.10. When using a bigint as a primary key, QGIS will cast the primary key to a text field. This causes ...
Lennert De Feyter's user avatar
12 votes

Fastest way to convert big raster to polyline using R or Python?

Edit: In the latest version of sf (v0.9.1) and stars (v0.4.1), you need to specify the merge = TRUE argument in the call to sf::st_as_stars() function. For posterity, I've been having success with ...
mikoontz's user avatar
  • 315
12 votes
Accepted

What can be a reason of QGIS slow work?

First, I would try with a fresh profile. Go to Settings -> User Profiles - New Profile. This is basically like a fresh QGIS install - no Plugins are loaded, no (maybe wrong) settings you changed ...
sn1ks's user avatar
  • 3,042
11 votes
Accepted

How to improve performance of nested search cursors?

I have to keep posting that I will never use embedded search cursors ever for relating tables to each other. Only dictionaries should be used to do what you are doing. Performance will increase 100 ...
Richard Fairhurst's user avatar
11 votes
Accepted

Alias for postgis function is really slow

You should use an SQL wrapper instead. plpgsql wrappers tend to be slower since they are not inlined and you are forcing an intermediary step too which can get slow the bigger the geometry. So write ...
Regina Obe's user avatar
  • 10.6k
11 votes
Accepted

Only union/dissolve intersecting or adjacent features to speed up query

I would look into using ST_ClusterDBSCAN. I have had tremendous success using this function to solve many cluster like geometric problems. WITH clusters AS(select st_clusterdbscan(geom, 0, 2) over()...
ziggy's user avatar
  • 4,536
10 votes

QGIS rendering performance bottleneck?

There are two issues you have to consider: Whether your data has spatial index or not. If not, then try to create a spatial index to your data. The spatial index can improve the speed of rendering ...
ahmadhanb's user avatar
  • 41.1k
10 votes

Disjoint is very slow in Virtual Layer but fast on symbology

The issue is the cross join on non-intersection. Your query reads: for every row in g, return every row in s that does not intersect with the current g which seems in fact what you want, but about ...
geozelot's user avatar
  • 30.4k
9 votes

QGIS rendering performance bottleneck?

Make sure that under Settings -> Options -> Rendering you have "Render layers in parallel" checked.
ndawson's user avatar
  • 27.8k
9 votes

ArcPy UpdateCursor auto pausing after every 1000 records

By default the auto commit interval is set to 1000. You can change the interval using the following: arcpy.env.autoCommit = 5000
Lorne Dmitruk's user avatar
9 votes
Accepted

Measuring loading time of each layer when opening a QGIS project

Since QGIS 3.16 you can know the time it takes to load each layer from a QGIS project. Go to View --> Panels --> Debugging/Development Tools and click on the Profiler in the left menu. As soon ...
Germán Carrillo's user avatar
8 votes

Increasing speed of crop, mask, & extract raster by many polygons in R?

This question is very old so my answer is just for posterities. Since 2020-11-20 there is the new exactextractr package which implement the exact_extract() function, which is the new benchmark for ...
Elia's user avatar
  • 298
8 votes

Speeding up QGIS spatial join?

Add a spatial index to each of your shapefile layers: Layer > Properties... > General then Create spatial index. Another way is to create them for all the layers in one go: Vector > Data ...
Sorbus's user avatar
  • 885
8 votes
Accepted

Importing CSV (via VRT) to PostGIS is slow with -skipfailures?

You don't have to use a .vrt any more. ogr2ogr supports reading csv files with geometry directly since version 2.1. The ogr2ogr command: ogr2ogr -f "PostgreSQL" PG:"host=000.000.000.000 port=0000 ...
HeikkiVesanto's user avatar
8 votes
Accepted

Fastest way to extract a raster in R (improve the time of my reproducible code)

I get much faster results with velox if I crop the raster before running extract, e.g.: r <- velox("testras_so.tif") r$crop(poly) r$extract(poly) I've also been working on a package with ...
dbaston's user avatar
  • 13.2k
8 votes
Accepted

Accelerating GeoPandas for selecting points inside polygon

With the help of @Taras I could solve my efficiency problem. As the first try, I thought maybe the tqdm library is causing this delay. So I modified my code to within_points = [points.geometry[i]....
Vahid's user avatar
  • 251
8 votes

Multi-threaded compression in rasterio?

Yes, although this seems not to be mentioned in the documentation. If you include num_threads=8 or num_threads='all_cpus' as an argument to rasterio.open then multithreading will be enabled (for ...
benjimin's user avatar
  • 476
7 votes

Which is the faster way to copy data to another feature class: Feature Class to Feature Class or Insert Cursor

I created three sample random point datasets - 10 features, 100k features, 1m features - and tested the following options Copy (arcpy.Copy_management()) Insert Cursor (arcpy.da.InsertCursor()) ...
Midavalo's user avatar
  • 29.8k
7 votes

Prevent SAGA in QGIS 3.0 from creating certain outputs?

Should be possible to do if you feel like hacking your qgis install. look for this file in your qgis install: https://github.com/qgis/QGIS/blob/master/python/plugins/processing/algs/saga/description/...
ndawson's user avatar
  • 27.8k
7 votes

QGIS more than 3000 raster works slowly

You may see considerable benefit if you load them into a single virtual raster (vrt). You can do that through the processing toolbox, by searching for "build vrt"
ndawson's user avatar
  • 27.8k
7 votes
Accepted

PostGIS ST_Intersects is slow and doesn't seem to use spatial index

Try WITH poly AS ( SELECT geom FROM admin_boundary WHERE id = <id> ) UPDATE highway_only_motor AS a SET has_business_value = true FROM poly AS b WHERE ...
geozelot's user avatar
  • 30.4k
6 votes

Avoiding GeoServer/Java out of heap space error?

I've had the same error message on a Windows Server with GeoServer using Jetty. After some research I found the file C:\Program Files (x86)\GeoServer 2.10.1\wrapper\wrapper.conf where I edited these ...
Franka's user avatar
  • 161
6 votes
Accepted

Deleting many duplicate points using ArcGIS Desktop?

I found that this script copies unique points into separate originally empty shapefile about 5 times faster, compare to delete identical. I tested it on 200,000 points, with 100,000 of them being a ...
FelixIP's user avatar
  • 23.2k
6 votes
Accepted

LineString VS Multi-LineString Performance PostGIS

I wouldn't expect any significant difference in performance unless you have very unusual data. There are a few places in the code where optimizations are put in for edge cases like two-point ...
dbaston's user avatar
  • 13.2k
6 votes

Displaying huge geotiffs (or vrts) with QGIS?

You seem to have two main concerns: VRT id slow with browsing and it is slow to build global overviews. While I am sure that GDAL VRT used to be slow for me and my MapServer many years ago it may be ...
user30184's user avatar
  • 67.3k
6 votes
Accepted

Displaying huge geotiffs (or vrts) with QGIS?

Ok, well I solved both my problems...mainly by buying an NVMe SSD. My disk read/write has gone from 125 MB/s to 1200 MB/s. Programatically, there are a few things you can do to help your read/write ...
Jon's user avatar
  • 2,894

Only top scored, non community-wiki answers of a minimum length are eligible