Tag Info

Hot answers tagged

24

I am using an example with 1 million randomly generated points inside of a filegeodatabase. Attached here. Here is some code to get us started: import time import arcpy arcpy.env.workspace = "C:\CountTest.gdb" time.sleep(5) # Let the cpu/ram calm before proceeding! """Method 1""" StartTime = time.clock() with arcpy.da.SearchCursor("RandomPoints", ...


18

My thoughts are: Export your shapefile to a file geodatabase feature class - I think its drawing performance will be better but am not sure by how much If you are using ArcGIS Desktop 10.0 or later move it into a Basemap Layer - this will improve drawing performance dramatically If you like the sound of pyramids for vector data, be sure to vote for this ...


16

One of the developers of arcpy.da here. We got the performance where it is because performance was our primary concern: the main gripe with the old cursors were that they were slow, not that they lacked any particular functionality. The code uses the same underlying ArcObjects available in ArcGIS since 8.x (the CPython implementation of the search cursor, ...


14

I want to take the chance of promoting OGR's virtual file system that writes geometries to a in-memory dataset. Using it is simple as @Luke demonstrated in this post drv = ogr.GetDriverByName( 'ESRI Shapefile' ) ds = drv.CreateDataSource(r'/vsimem/virtual.shp') This works just great. Creating a point shape file with ~300.000 geometries and two attribute ...


11

The most efficient index for the query expressed in your question is the one on gid as it is the only column that appears in a where expression: CREATE INDEX table_gid ON table (gid); You can safely drop the gist index as it will only consume space and slow inserts/updates/deletes down. Long explanation As I said the most effective index in your case ...


10

I'm not that familiar with QGIS, but I wonder how it compares with ArcGIS in terms of extensibility. Unfortunately there seems to be at least some tradeoffs between extensibility and performance. The best way I've found to get a feel for ArcGIS extensibility is to take a look at Esri's COM component categories found in the registry. Each category ...


10

After contacting the appropriate IT admin, I recieved the following instructions, resulting in load times closer to my expectation: ArcMap now takes ~4s, ArcCatalog ~3s to load. Install ArcGIS service pack 3 (this did not change boot time on its own) run "esriremove.reg" to remove changes made by ArcGIS to the registry, the contents of esriremove.reg are: ...


10

ArcGIS is utterly woeful for dissolving/merging. We had to do a buffer/merge for 3 million points recently and soon gave up on using ArcGIS -- their help desk didn't have much clue either. Postgres did it in less than an hour using the st_union function. see http://blog.cleverelephant.ca/2009/01/must-faster-unions-in-postgis-14.html


10

According to System Design Strategies - Software Performance, an Esri technical article, file geodatabases are faster than shapefiles in their specific tests. They go into a lot of additional detail regarding what bottlenecks may exist and other performance considerations for various ArcGIS software use cases.


10

As unicoletti said, the gist index in the geometry column would only work if you use ST_Contains() in the WHERE expression. For instance, if you would like to know all polygons that contain one another, you could use something like this: SELECT a.gid, b.gid FROM table AS a, table as b WHERE a.gid != b.gid and ST_Contains(a.way, b.way) In this case, ...


9

PostGIS. Geoserver documentation has the following comment: "Shapefiles are a very common format for geospatial data. But if you are running GeoServer in a production environment, it is better to use a spatial database such as PostGIS. This is essential if doing transactions (WFS-T). Most spatial databases provide shapefile conversion tools. Although there ...


8

ArcGIS seems very bloated. I remember a huge performance hit when migrating from Arcview 3.2 to ArcGIS 8.0, and in a lot of places it still exists. At that time I thought it had a lot to do with ESRI migrating earlier Arc/Info code to Windows and having to cut some corners in performance, but I'm not sure if that's true. I recall seeing some examples on this ...


7

Are you connecting to a license server on a network? I've found that when I connect to our license server here at work from my home laptop, AM and AC do take longer to launch - but still not 1 minute, more like 15 seconds as opposed to about 3-5 seconds at the office. Also, as I mention in a comment in Jakub's answer, if you are referencing data (in a ...


7

The most important thing 9.x users need to know is that ArcSDE no longer exists. There had been no fundamental changes to the ArcSDE API since 9.0, when XML, INT64, UUID, CLOB, NSTRING and NCLOB types were added. Since then, the only changes have been Projection Engine mods to support newer projections and vertical datums. The API itself is now deprecated ...


6

I do not think that Arc is written in .NET. Arcobjects are written in C++. Arc may be slower due use of many advanced GUI's, help tools, add-ons etc. QGIS is great software but it lacks some useful features that may be good for beginners. Also I do not think that basic lavel tools in ESRI (Arcobjects) are slow. It usually comes down to user skills, if user ...


6

ArcMap and ArcCatalog are generally slow to open and many complain about their performance in general. I have more less the same specs on my desktop (except I have 12GB RAM, not that it matters much since ArcGIS is a 32 bit app, and a 15,000 RPM hard drive) On this machine ArcCatalog opnes in 28 seconds the first time and in 15 seconds the second time I ...


6

FGDB_BULK_LOAD is not a compilation setting, it is a configuration option for the command line tools (can also be done programmatically). ogr2ogr --config FGDB_BULK_LOAD YES -f "FileGDB" MyFileGDB.gdb myKML.kml Would create a filegdb and load the KML vector data to it. Let me know if your performance still sucks. By the way, what platform are you on? ...


6

It depends, naturally. If your rendering only has to spatially scan a "few" (thousands) records to draw, then physical clustering might help, a bit. However, if your rendering has to scan 100s of thousands of records (classic scenario: find me all the highways in this square, but ignore the local roads) then clustering won't make much difference, whereas ...


5

This relates to ArcGIS performance: ArcMap, ArcCatalog very slow to open on a new laptop with ample resources which may in part account for some of the performance issues. That thread shows how hardware, network, and licensing configuration can have a substantial effect on ArcGIS performance. Possibly, some of the reported differences in speed could be ...


5

One of my former workmates set up a long-running GIS computation process such that it saved intermediate results on a RAM disk. Both he and his boss claimed that there was a very significant speed-up of the computation process, and computation time was quite critical in their project. I personally couldn't quite figure out why the RAM disk was needed at ...


5

There are many tips for improving ArcMap performance which might help, but here's three suggestions I've used. I'd make sure the Data Frame coordinate system matches my shapefile's and any other layers I have on in the TOC; but the fewer layers to be displayed the better. I'd only use my symbology based on simple lines and fills without transparency or ...


5

No. There is no performance, or reason otherwise, to always, apart from consistency of course, have the geometry column last. I suspect the reason it is last most of the time is just historical due to PostGIS being an add-in rather then a built in shipped feature of PostgreSQL. Most of the tables I manage have something like: id integer the_geom geometry ...


5

Yes. Label expressions slow drawing performance. In particular, avoid using label expression scripts to conditionally parse or format label strings or to concatenate the information in multiple fields into one label string when drawn. Instead, calculate a new field with exactly the information that is needed. The field can include formatting tags. From ...


5

Load the data into SpatiaLite in QGIS... best way is to create a new SpatiaLite database via the right-click GUI in the browser window, then simply drag and drop your shapefile onto the SpatiaLite database you just created. From there you have all the power of SpatiaLite at your disposal. Some more documentation can be found here, and there's the SpatiaLite ...


5

Turn off snapping (Snapping toolbar, uncheck "Use Snapping"). I've had this problem before when there are many vector layers in a project, the cursor is getting bogged down looking for a vertex (or edge, or whatever) to snap to. You could also try copying the image to be georeferenced to a new, empty ArcGIS project along with the bare minimum of vector ...


5

A lineage length of 289 indicates 289 states participate in that lineage. If this is after a compress operation, this indicates there are 289 states that cannot be compressed together to collapse that lineage without affecting the versioned table structure. A likely cause of this is that there are versions that directly reference those states. You can see ...


4

I've played a bit with RAM disks for a GIS, with mixed results. Its good to think about what you're trying to achieve, and how RAM might help: often, GIS analysis do sequential I/O operations, which aren't significantly faster in memory. Plus, the OS is already caching frequently accessed data from disk, which means that only the initial seek is costly. As ...


4

Instead of creating a RAM disk, how about using a "disk" that's made out of RAM? Also known as a Solid State Drive. We bought some OCZ Revo X2 Drives a couple of months ago and are very happy with them so far. I don't have specific statistics to share but the interactive performance when working with local data is definitely noticeable. The best site by ...


4

Interesting question. Unfortunately, I cannot fully answer it, but still would like to mention some related tools for measuring performance: mxdperfstat Tools on the map service publishing toolbar (Both of these are also mentioned in this blog article.)


4

We use a stopwatch and a spreadsheet, and measure (a) time from initial load to spinny-globe-refresh stops spinning, (b) time to refresh (press refresh button), (c) zoom to scale N, (d) zoom to scale NN, (f) pan. Repeat at least 3 times for each datastore. Repeat again at different times of day to account for network usage patterns by others. The results ...



Only top voted, non community-wiki answers of a minimum length are eligible