Take the 2-minute tour ×
Geographic Information Systems Stack Exchange is a question and answer site for cartographers, geographers and GIS professionals. It's 100% free, no registration required.

I have a large amount of GIS data, roughly 30 layers, each with 50,000 to 100,000 (Geo)records. Some of this data is GPS collected points, others are POI, some is government provided data (roads, rivers, water sheds, etc),

So far I'll either be using mapfish, geoserver, or feautureserver with jquery/ajax backend.

How should I organize my GIS Data to allow for fast and dynamic webside client searching? as well as feel free to provide your experience from projects especially working on large data sets with heavy user quering

share|improve this question
1  
Do you have a preference to send the data to the client as vector or images? –  Matthew Snape Feb 14 '11 at 14:32
    
@Matthew Snape - either or, what are the pros and cons? what do you recommend? –  dassouki Feb 14 '11 at 14:37
1  
Are these spatial searches or attribute searches? –  geographika Feb 14 '11 at 14:49
    
@geographika - a mix of both, but mostly attribute searches to produce a spatial result –  dassouki Feb 14 '11 at 15:16

3 Answers 3

Hallo

100 000 rows is quite small tables. If the geometries is just points it is very small :-)

Just index the columns that users will search on and it will be very fast.

I guess the spatial searches will be mostly from rectangles? Then the spatial index will pick out your points in no time without the need of a recheck. In that special case (finding points in a box) you can use the && operator to get them from only bounding box comparing. That's very fast if you have a spatial index on the points.

HTH Nicklas

share|improve this answer
    
the geometries consist of multi/polygons, lines, polylines, points, rivers, gps points, pretty much everythign –  dassouki Feb 14 '11 at 15:26
1  
ok, to make the spatial index more effective you could consider using st_dump to get all multi-geometries to single, if the attribute and overall design can live with that. Then the next step I think is looking at the actual queries you are going to run. What can be done to optimize beyond keeping the size of geometries small and indexing all geometries with GIST and all attributes with b-tree, is probably mostly design of the actual query you are going to run. –  Nicklas Avén Feb 14 '11 at 19:04

Think about how to structure your UI so that a user gets what they need, and only what they need. This can be done in many different ways including the use of "zones of interest", multi-select boxes, partial text matches etc. so you can return the 2 or 3 relevant features rather than a list of 50 the user has to wait to download and then scroll through. If there are many relevant results then dynamic paging should be considered.

A calculated field containing generalised versions of features for "preview" may also reduce bandwidth / load times.

Apart from that it is all about the attribute indexes. There is a free online book that goes through these in depth at http://use-the-index-luke.com/

Spatial indexes are also critical (and using tile caches is by far the biggest performance boost you can have), but these are more related to displaying layers on a map, or selecting by geometry.

share|improve this answer

B-Tree faster but GiST is more flexible...

"B-Tree indexes perform better, but GiST indexes are more flexible."

http://stackoverflow.com/questions/766488/whats-the-difference-between-b-tree-and-gist-index-methods-in-postgresql

share|improve this answer
    
can b-tree work as a multi-dimensional index? I though that was the reason for using gist in PostGIS that the multi-dimension ability is necessary to deal with spatial data (x, y and maybe even z dimensions) –  Nicklas Avén Feb 14 '11 at 15:04

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.