Tag Info

Hot answers tagged

42

Although this question was already answered, I thought I could chime in an give my two cents. DISCLAIMER: I worked for ESRI at the GeoDatabase team for some years and was in charge of maintaining various parts of GeoDatabase code (Versioning, Cursors, EditSessions, History, Relationship Classes, etc etc). I think the biggest source of performance problems ...


24

I am using an example with 1 million randomly generated points inside of a filegeodatabase. Attached here. Here is some code to get us started: import time import arcpy arcpy.env.workspace = "C:\CountTest.gdb" time.sleep(5) # Let the cpu/ram calm before proceeding! """Method 1""" StartTime = time.clock() with arcpy.da.SearchCursor("RandomPoints", ...


18

My thoughts are: Export your shapefile to a file geodatabase feature class - I think its drawing performance will be better but am not sure by how much If you are using ArcGIS Desktop 10.0 or later move it into a Basemap Layer - this will improve drawing performance dramatically If you like the sound of pyramids for vector data, be sure to vote for this ...


17

A couple of general tricks I have found useful in the past in this situation: Run your Python script as stand-alone (e.g. from IDLE, PyWin, Eclipse or preferably CMD) to remove the overhead of ArcMap. Spawning subprocesses is an old trick to solving ArcGIS memory leaks even if you don't want to parallize a process. It works because the memory is released ...


16

One of the developers of arcpy.da here. We got the performance where it is because performance was our primary concern: the main gripe with the old cursors were that they were slow, not that they lacked any particular functionality. The code uses the same underlying ArcObjects available in ArcGIS since 8.x (the CPython implementation of the search cursor, ...


15

Since Postgis is a component of Postgres I would recommend this great book (I own it and I found it extremely valuable) on Postgres performance tuning: http://www.packtpub.com/postgresql-90-high-performance/book It starts from the basics (planning the hardware, os, etc) and then grows into explaining all those misterious configuration params that I never ...


14

PostGIS in Action, which has been released last month is a good book for worth reading. PART 1 - LEARNING POSTGIS 1 - What is a spatial database? Completed download chapter code and data 2 - Geometry Types Completed download chapter code and data 3 - Data Modeling Completed download chapter code and data 4 - Geometry Functions Completed download chapter ...


14

I want to take the chance of promoting OGR's virtual file system that writes geometries to a in-memory dataset. Using it is simple as @Luke demonstrated in this post drv = ogr.GetDriverByName( 'ESRI Shapefile' ) ds = drv.CreateDataSource(r'/vsimem/virtual.shp') This works just great. Creating a point shape file with ~300.000 geometries and two attribute ...


11

I'm not that familiar with QGIS, but I wonder how it compares with ArcGIS in terms of extensibility. Unfortunately there seems to be at least some tradeoffs between extensibility and performance. The best way I've found to get a feel for ArcGIS extensibility is to take a look at Esri's COM component categories found in the registry. Each category ...


11

The most efficient index for the query expressed in your question is the one on gid as it is the only column that appears in a where expression: CREATE INDEX table_gid ON table (gid); You can safely drop the gist index as it will only consume space and slow inserts/updates/deletes down. Long explanation As I said the most effective index in your case ...


10

ArcGIS is utterly woeful for dissolving/merging. We had to do a buffer/merge for 3 million points recently and soon gave up on using ArcGIS -- their help desk didn't have much clue either. Postgres did it in less than an hour using the st_union function. see http://blog.cleverelephant.ca/2009/01/must-faster-unions-in-postgis-14.html


10

According to System Design Strategies - Software Performance, an Esri technical article, file geodatabases are faster than shapefiles in their specific tests. They go into a lot of additional detail regarding what bottlenecks may exist and other performance considerations for various ArcGIS software use cases.


10

After contacting the appropriate IT admin, I recieved the following instructions, resulting in load times closer to my expectation: ArcMap now takes ~4s, ArcCatalog ~3s to load. Install ArcGIS service pack 3 (this did not change boot time on its own) run "esriremove.reg" to remove changes made by ArcGIS to the registry, the contents of esriremove.reg are: ...


10

As unicoletti said, the gist index in the geometry column would only work if you use ST_Contains() in the WHERE expression. For instance, if you would like to know all polygons that contain one another, you could use something like this: SELECT a.gid, b.gid FROM table AS a, table as b WHERE a.gid != b.gid and ST_Contains(a.way, b.way) In this case, ...


10

PostGIS. Geoserver documentation has the following comment: "Shapefiles are a very common format for geospatial data. But if you are running GeoServer in a production environment, it is better to use a spatial database such as PostGIS. This is essential if doing transactions (WFS-T). Most spatial databases provide shapefile conversion tools. Although there ...


8

I have just the same problem and was about to ask on mailing lists, so thanks to everybody! I am using Shooting Star with a million and a half rows on the routing table. It takes almost ten seconds to calculate it. With 20k rows it takes almost three seconds. I need Shooting Star because I need the turn restrictions. Here are some ideas I'm trying to ...


8

I can confirm that OL Canvas renderer redraws the whole layer when even a single feature needs to be redrawn. The exact point of the sources where this is implemented is this one in drawFeature. This has probably to do with the fact that the canvas is simply a bitmap so redrawing on top of it can produce artifacts if the new image is not perfectly placed on ...


8

This is an exciting question! How big is the raster you want to query? WKTRaster is stored in the database as a BLOB. In order to find the value at a specific point, from a known (x_0, y_0) corner coordinate row/column indices (i, j) are computed using (dx, dy) steps and rotation. With (i, j) known, the ST_Value() function can access the actual data at the ...


8

See this help document for creating multiple connections. This simple code will create multiple connections: EXEC SQL CONNECT TO testdb1 AS con1 USER testuser; EXEC SQL CONNECT TO testdb2 AS con2 USER testuser; EXEC SQL CONNECT TO testdb3 AS con3 USER testuser; You can then choose a connection to use: EXEC SQL AT connection-name SELECT ...


7

Ensure you are storing valid geometries, otherwise unexpected errors can happen (server crashes, erroneous distance and area calculations, etc., see What are the implications of invalid geometries). I highly recommend adding check constraints to production geometry columns to make sure the geometries are valid. It is simple to add (just replace ...


7

OPENGEO has a simple tutorial that will serve as a good starting point: Tuning PostgreSQL for Spatial Although not directly related to PostGIS, I've found these two posts informative when it comes to more 'general' PostgreSQL setups: Tuning PostgreSQL on your Dedicated Server Performance Tuning PostgreSQL This presentation by PGexperts is also very ...


7

Another approach to this would be to use the Spatial Join tool. Use the point as your input feature layer as above and the polygon layer as your identity features.Unlike SelectLayerByLocation, SpatialJoin does honor the extent environment. targetlayer = layername joinlayer=arcpy.PointGeometry(arcpy.Point(x, y)) fieldmappings = arcpy.FieldMappings() ...


7

When faced with tasks like this your primary objective is to be rational. Don't change params based on 'gut feeling'. While the gut seems to works for Hollywood it does not for us who live in the real world. Well, at least not my gut ;-). You should: establish a usable and repeatable metric (like the time required by a pgrouting query) save metric results ...


7

ArcGIS seems very bloated. I remember a huge performance hit when migrating from Arcview 3.2 to ArcGIS 8.0, and in a lot of places it still exists. At that time I thought it had a lot to do with ESRI migrating earlier Arc/Info code to Windows and having to cut some corners in performance, but I'm not sure if that's true. I recall seeing some examples on this ...


7

Are you connecting to a license server on a network? I've found that when I connect to our license server here at work from my home laptop, AM and AC do take longer to launch - but still not 1 minute, more like 15 seconds as opposed to about 3-5 seconds at the office. Also, as I mention in a comment in Jakub's answer, if you are referencing data (in a ...


7

The most important thing 9.x users need to know is that ArcSDE no longer exists. There had been no fundamental changes to the ArcSDE API since 9.0, when XML, INT64, UUID, CLOB, NSTRING and NCLOB types were added. Since then, the only changes have been Projection Engine mods to support newer projections and vertical datums. The API itself is now deprecated ...


6

In short, it's not optimal, but it might not be too bad either... On (3) - as long as you always specify the specific fields you want, the only extra "hit" should be on the initial request for the service metadata (which will be a bit larger due to the many fields). If you know the "set" or "theme" of grouped fields your app will use (out of the 500), you ...


6

Always keep your data in a different schema than public. Keeping your data separated from the public schema will make it easier for your to updagre PostGIS and will make it easier for you to restore schema dumps. Imagine that you dump your data inside the public schema. Since dump is so nice, it will also dump table definitions and functions, so to restore ...


6

I do not think that Arc is written in .NET. Arcobjects are written in C++. Arc may be slower due use of many advanced GUI's, help tools, add-ons etc. QGIS is great software but it lacks some useful features that may be good for beginners. Also I do not think that basic lavel tools in ESRI (Arcobjects) are slow. It usually comes down to user skills, if user ...


6

This relates to ArcGIS performance: ArcMap, ArcCatalog very slow to open on a new laptop with ample resources which may in part account for some of the performance issues. That thread shows how hardware, network, and licensing configuration can have a substantial effect on ArcGIS performance. Possibly, some of the reported differences in speed could be ...



Only top voted, non community-wiki answers of a minimum length are eligible