how to improve rendering performance of very large shapefile?

Question

I have a polygon shapefile that has a 100MB .dbf and 500MB .shp file component. The reason it is so large is it is a landbase classification for a whole district.

Everytime I am viewing the file in ArcCat or ArcMap and move the view window however slightly, the entire file needs to redraw from scratch. This is a problem the more zoomed-out I am. I have tried spatial indexing and importing into a geodatabase - neither approach provides any noticable performance improvement with respect to rendering.

ESRI's help page suggests that that to improve shapefile performance, the user can generalize the file. While this would obviously work, I do not want to lose any information. Splitting the file up is not ideal since I am doing a lot of geoprocessing/queries with its entire area. I guess I could avoid viewing the entire area at once - but sometimes, for example, it is good to see what parts of the file a query has selected.

Is there any other approach I could take to improve rendering performance?

(In theory, building shapefile "pyramids" would be ideal - I'm unsure why ArcGIS has never supported such an approach - at least that I am aware of...)

Answer 1

Two thoughts are:

1. Export your shapefile to a file geodatabase feature class - I think its drawing performance will be better but am not sure by how much
2. If you are using ArcGIS 10.0 or 10.1 move it into a Basemap Layer - this will improve drawing performance dramatically

PS If you like the sound of pyramids for vector data, be sure to vote for this ArcGIS Idea.

Answer 2

There are many tips for improving ArcMap performance which might help, but here's three suggestions I've used.

1. I'd make sure the Data Frame coordinate system matches my shapefile's and any other layers I have on in the TOC; but the fewer layers to be displayed the better.
2. I'd only use my symbology based on simple lines and fills without transparency or other complexities.
3. If you need both an overview and the ability to pan around up close, consider using the magnifier or viewer.

Answer 3

You can improve rendering by setting the layer display parameters so that the layer will not display at large scales (e.g. >1:10,000). You can find this option in the layer properties: Layer Properties > General Tab > "Don't show layer when zoomed out beyond..."

Also, storage location matters--for example, if it is stored on an old server with poor bandwidth, you are guaranteed to have poor performance. I routinely handle 1GB+ vector data over a server, which makes me question whether your system specs need to be updated (for reference, I'm running 12GB RAM, 2nd gen i7, average graphics card).

Answer 4

As a bit of a followup to Aaron's answer, you could also use a definition query to limit the number of results returned for visualization (and it includes analysis - I believe it functions a lot like a selection). If not all features are needed for viewing at any given moment and you're not switching regions a ton, a definition query could be a workable solution, though not an exact answer to your question or needs.

Answer 5

I hear your frustration. I routinely work with large shapefiles like this and don't have display problems, in general. I agree with all the comments above, especially making sure everything is in the same projection, including the data frame. I assume you have copied the file locally and are not trying to access it over the network? One thing that will cause display issues with shapefiles this size is if there are an extreme amount of vertices, as in a stream network. The only solution I have found to this is creating a python script to do layer definitions on the fly so I am only drawing a few at a time. Another thing would be to update your computers graphic memory and graphic card. Good luck!

Answer 6

Do you need to render the map to make your queries? What if you ran python scripts and accessed the data without drawing the map? I'm not sure your exact process, needs, etc, but a thought.