Questions tagged [performance]
Performance concerns how effectively a computing system uses resources.
756
questions
47
votes
7
answers
13k
views
Tuning PostGIS for production environments
What PostGIS documentation have you come across that has been helpful when tuning PostGIS for a production environment?
I would like to get together with my DBA to setup a Postgresql/PostGIS ...
- 29.9k
8
votes
2
answers
3k
views
Finding nearest neighbor for each point in the same table
We've got a table of approximately 300,000 points and would like to find the nearest neighbor (within 10km) to each point in the same table. The most efficient query we've found so far is below, but ...
- 30.6k
0
votes
1
answer
942
views
Improving ArcGIS Pro performance on virtual desktop
I use ArcGIS Pro remotely and use a virtual desktop. I noticed the software works extremely slowly and would take time to load changes on my map. I took a look at the CPU and memory utilization. My ...
CommunityBot
- 1
6
votes
1
answer
223
views
Left joining CSV to GeoPackage file and outputting GeoJSON in scalable way with ogr2ogr
I have a large GPKG and a CSV file, and I'd like to (left) join the CSV to the GPKG on one of the columns. A visualisation of what I'm trying to achieve:
Reproducible example
We can create an example ...
- 33.7k
6
votes
2
answers
5k
views
Return closest element in GeoSeries
Is there any built-in function or quick hack to efficiently return the closest element (index) to a point for a given geopandas GeoSeries?
Something like
>> geopandas.GeoSeries(
>> {
...
- 76.2k
3
votes
1
answer
150
views
Efficiently check if polygon contains any point from a list
I have a large, static list of Shapely Points that I want to check against a large list of Shapely Polygons, filtering out any polygons that contain one or more of the points.
The polygons do not ...
- 3,804
12
votes
4
answers
11k
views
Rendering performance bottleneck in QGIS
When I load a vector with a large number of features into QGIS, the rendering time can become extremely slow. But when I look at my system resources, they all have plenty of excess capacity.
What is ...
- 33.7k
1
vote
0
answers
28
views
Loading Web Feature Service (WFS) takes too long in QGIS
I am trying to load WFS (URL for Cadastral data of a federal state in Germany) in QGIS LTR but it is extremely time-consuming. I know, it is a big dataset.
Whenever I try to load the layer, it shows ...
- 33.7k
0
votes
1
answer
349
views
Prevent sending too many requests while zooming/paning (OpenLayers 6.15)
I have a web page that displays WMS data from a WMS server, using a TileLayer. The images are fetched with a callback function called by the setTileLoadFunction(..) method of the associated TileWMS ...
CommunityBot
- 1
0
votes
1
answer
628
views
Making looping through line segments for vertex points faster?
I am trying to speed up some current code that I have for looping through lines, getting their segments and ultimately the vertex points in Python.
Can anyone recommend a faster method than what I ...
CommunityBot
- 1
2
votes
1
answer
36
views
Geopandas slow after vectorization
Processed an NDVI, created classes from it and vectorized the file.
The area is about 4 km * 4 km. I wanted to intersect, and that was slow. While creating a minimal example, it seems like any geo-...
- 3,804
0
votes
1
answer
142
views
Performance difference between xarray open_mfdataset and open_dataset
I'm dealing with CMIP6 Scenariomip data, I have eight ESMs precipitation grid that each one is composed by multiple netcdf files. There is one ESMs's netcdf files are seperately reserved every decade, ...
2
votes
0
answers
215
views
Large amount of time publishing to ArcGIS Online?
Publishing <5Mb tile packages to ArcGIS Online is taking more than 20 minutes. ESRI tech support has consistently given extremely vague answers that never solve and always reference server load. &...
10
votes
1
answer
396
views
Performance-limits of QGIS expressions
I use quite often QGIS expressions and I am wondering how performant they are compared with other possibilities. In many cases, the same problem can be resolved using different approaches like using ...
- 8,316
0
votes
1
answer
165
views
Speeding up GDAL gridding process
I have a shapefile containing 28 915 413 points.
I am trying to grid it using the minimum method and create a raster.
The survey was at 0.5 m, so I have set this as the radius and the output file ...
CommunityBot
- 1
0
votes
1
answer
275
views
Faster way to compute Moran’s Index from a very large distance matrix between geographic points
I am computing Moran’s Index from a very large distance matrix between geographic points. The time to execute the code is too high. I used the dism function in the geosphere package.
Here a ...
- 65.3k
2
votes
0
answers
595
views
Layer writing / exporting time in QGIS takes far too long
I am finding that QGIS takes quite some time when writing or exporting large datasets (approx 5 million records). A task I'm currently doing requires me to export 5 million points to a CSV, and this ...
- 1
0
votes
1
answer
4k
views
Speeding up Google Geocoding API Slow performance
I am using Google geocoding API service with their Java client, and my dev Key,
The problem is when I run up to 40 requests it does slow to 600ms. I am using it in a webservice that is supposed to run ...
CommunityBot
- 1
1
vote
1
answer
223
views
How to do non-blocking OWS requests in Python?
Is it possible to use OWSLib with aiohttp?
Is there a Python library for formatting/parsing WPS (and other OGC web services, such as WMS - web map service) requests, that also supports IO coroutines (...
CommunityBot
- 1
3
votes
1
answer
165
views
Why is it slow when deleting features in gpkg using PyQGIS?
Deleting features in a somewhat big GeoPackage takes a really long time and I am not sure if I do something wrong or if it is normal that it takes such a long time.
Data: line GeoPackage layer ...
- 3,804
4
votes
1
answer
161
views
Reasons of slow intersection of two GeoDataFrames using Python [closed]
I am trying to find the largest intersecting area for each Polygon in a layer1 compared to the geometries in a layer2 in python. I use geopandas.overlay for it.
However, the geometries in layer2 don't ...
- 3,804
2
votes
1
answer
119
views
Does ST_Simplify help on lines?
I have a function that selects all lines whose geometry fall into given tile envelope.
There are a couple millions of lines with proper indices on the table.
Would the query perform better if I would ...
- 722
2
votes
2
answers
79
views
Optimizing river network features queries in ArcGIS JavaScript SDK
I am working on a map to visualize river network data using ArcGIS SDK for JavaScript. The data I’m using is from this ArcGIS feature service published by the CLMS. The data includes a dozen feature ...
- 26.5k
6
votes
1
answer
2k
views
Improving speed of GeoPandas dissolve
I am trying to dissolve grid spaces by category using GeoPandas' dissolve function, but it doesn't finish after hours of running and I want to make sure I'm not doing something wrong and/or there isn'...
CommunityBot
- 1
1
vote
0
answers
75
views
Working with very large GeoTIFFs using QGIS
I would like to ask for advice on working with very large GeoTIFFs.
I have downloaded several large GeoTIFFs, then I load them into QGIS and combine them into a virtual raster.
Since I need the ...
- 20.2k
3
votes
1
answer
438
views
Clipping large shapefile in QGIS, GEOS geoprocessing error: intersection failed
I am working on a large shapefile sourced from https://www.iucnredlist.org/resources/spatial-data-download. I cannot clip Fish and Plants (global file) in Freshwater groups species to a smaller extent ...
- 107
0
votes
1
answer
108
views
Getting GEE data faster
I need country-month level nighttime lights data. I have already printed the data using the minimum example on the https://code.earthengine.google.com/ platform, but the current problem is that it ...
- 2,331
-2
votes
1
answer
104
views
Google Earth Engine Computation timed-out Error
I adapted the GEE script from the UN-Spider website, which now returns an computation timed-out error after 12 hours. The script basically calculates the flood extent and then overlays it with ...
- 17
0
votes
0
answers
184
views
Low performance when writing a large xarray.DataArray with dask
I'm preprocessing HMASR data, the data contains multiple data files with format of netcdf, and each tile (1X1 degree) is a nc file. So I read it through xarray open_mfdataset() to mosaic it spatially ...
- 31
4
votes
3
answers
4k
views
Slow IE8 responsiveness with OpenLayers and many features
Background
Adding about 700 sites to an OpenLayers Map. Firefox 11 and Google Chrome display the sites just fine. IE8, which is the organization's standard and will not be upgraded for the foreseeable ...
- 65.3k
2
votes
0
answers
71
views
PostGIS slow performances on small table and basic SELECT query
A few weeks ago, I received a query to install a PostGIS database for a few of my users. I'm not familiar with this but yet installed quite simply the extension using the sudo apt install postgresql-...
- 21
0
votes
1
answer
269
views
QGIS and remote PostgreSQL/PostGIS (in cloud) slow performance
I am using QGIS 3.28.11 version and in backend I am using PostgreSQL 15.0 with PostGIS 3.3.3.
I have total 36 layers which contained in 3 groups: A, B, C. Now X user has read only access on all layers ...
- 55
5
votes
1
answer
1k
views
Measuring loading time of each layer when opening a QGIS project
I often find myself working with QGIS projects that load too many layers. I'm talking about a mix of raster and vector layers, some with small file size, some very large file size. Most of them are ...
- 65.3k
0
votes
1
answer
75
views
Can I utilize vectorization to improve performance instead of iterating over QGIS Features?
I am using Python to write a standalone application with QGIS. The application uses requests to allow the user to update a shapefile from the internet on demand. After fetching the shapefile, I need ...
- 33.7k
39
votes
6
answers
29k
views
Increasing speed of crop, mask, & extract raster by many polygons in R?
I'm extracting the area and percent cover of different land use types from a raster based on several thousand polygon boundaries. I've found that the extract function works much faster if I iterate ...
- 298
3
votes
1
answer
121
views
Find all geometries in one table that do not exist in another
Using PostgreSQL and PostGIS, I have two tables of polygons, and I need to know which of the records in table p1 do not exist in table p2. I do not have additional information available (e.g. ids) - ...
- 30.6k
3
votes
0
answers
173
views
Speed up a query using ST_Within and ST_PointOnSurface
CREATE TABLE municipal_area (
id varchar NOT NULL,
"name" varchar NOT NULL,
state_id varchar NOT NULL,
state_abbr varchar NOT NULL,
state_name varchar NOT NULL,
...
- 153
3
votes
1
answer
190
views
QGIS Speed up slow virtual layer
In QGIS I have a virtual layer which does a line generation based on a point shape. The point shape has about 700 point geometries and 26 attribute fields.
SELECT a.Name, a.FromNode, a.ToNode, a....
- 61
1
vote
0
answers
88
views
Poor cache hit ratio for spatial tables
Most PostgreSQL tuning guides introduce the cache hit ratio as a metric to find out if PostgreSQL has enough heap space to cache data and reduce heavy disk i/o. To get a hit ratio for a certain table ...
- 20.2k
6
votes
2
answers
503
views
Stop Virtual Layer from automatically recalculation in QGIS
QGIS's Virtual Layer has big performance impact. Even when panning or zooming in the canvas, a virtual layer could take seconds to minutes to refresh. From my speculation, the reason might be the ...
- 33.7k
0
votes
0
answers
550
views
Slow performance of ST_AsMVT for huge PostGIS database
I am fetching MVT tiles from a PostGIS database in combination with pg_tileserve.
When fetching tiles from a fairly small dataset (some 10000 features) this works quite performant.
But when fetching ...
1
vote
1
answer
78
views
Browser hangs while parsing georaster in geoblaze
I've been trying to create a histogram using geoblaze with a COG from s3.
The georaster is as follows: (generated with geoblaze.parse(url))
{
_data: '==hidden==',
projection: 32618,
height: ...
- 20.2k
13
votes
2
answers
5k
views
Lots of layers in one or multiple services? (and why)
Scenario:
Client has a web mapping application. Does not want to split into multiple smaller applications. Although this goes against what the modern day approach is for maps on the web, (i.e. ...
- 65.3k
1
vote
0
answers
84
views
Improving performance when loading data in current Map using ArcGIS Pro SDK and C#
I want to load data into a current map in ArcGIS Pro 2.9. For this purpose, I want to use a .lyrx file. Currently, I have implemented it with:
Uri uriLayerPath = new Uri(strLayerPath);
LayerFactory....
- 65.3k
4
votes
0
answers
2k
views
Clustering table on geohash index and geohash ordered table [closed]
Could there be a performance gain by reorganizing the rows in a table by Geohash and then CLUSTERing this table on the Geohash index of the geometries?
CREATE TEMPORARY TABLE tmptbl AS
SELECT *, (...
- 65.3k
1
vote
0
answers
62
views
Speeding up QGIS Processing - Plugin [closed]
I am trying to calculate the angular integration and angular choice of a nationwide road network (Line SHP).
PST does the job, however it uses ALL (and by that I mean 100%) of my CPU's cores, but does ...
- 65.3k
0
votes
0
answers
56
views
Time to list all schemas of PostgreSQL with QGIS 3
QGIS 3.18 PostgreSQL 14.
We use QGIS to access data from a PostGIS server. The data are stored in 33 schemas.
one contains all the data in 17 tables;
the others schemas contain views and materialized ...
- 20.2k
13
votes
3
answers
9k
views
Poor Performance when using Spatial Indexes in MySQL
Re-post of a question asked on Stack Overflow when it was suggested this would be a better forum.
I'm trying a little experiment at pushing a data set which is not geo-spatial but fits it quite well ...
CommunityBot
- 1
1
vote
0
answers
103
views
QgsGeometry.fromWkt() seems weirdly slow compared to OSGeo/Shapely
wkt = 'LineString (64763.65372879413189366 5532941.864832429215312, 64811.82167246036988217 5533046.30194362625479698, 64827.02791490274103126 5533082.7359510725364089, 64833.97425112823839299 5533108....
- 3,919
3
votes
2
answers
161
views
Improving ST_Intersects performance in PostGIS
I have the following query, but it takes a long time to run. I tried creating an index, but there hasn't been much improvement. Any suggestions?
CREATE INDEX temp_clusters_geom_idx
ON temp_clusters
...
- 65.3k