Take the 2-minute tour ×
Geographic Information Systems Stack Exchange is a question and answer site for cartographers, geographers and GIS professionals. It's 100% free, no registration required.

Is there any simple way for me to run millions of parcels through some type of script to see if a shape file parcel is on a golf course? For example if I have a layer that has all the golf courses and parcel layers and let's say the max distance would be 100 feet then how would i be able to extract just those parcels?

share|improve this question
1  
'buffer' the golf course [assuming is a polygon] (by 100 feet) and then 'intersect' the buffer zone with the parcel layer will give all the parcels - it might take a while but will work. –  Mapperz May 12 '11 at 2:43
1  
Hi NetConstructor, could you specify what software you're trying to accomplish this in? –  scw May 12 '11 at 3:22
add comment

3 Answers 3

up vote 9 down vote accepted

Since you have quite a lot of parcels that you want to test you should look for a software that uses spatial indexes for this type of calculations.

And also be sure to use a method in that software that really uses the indexes.

If you import your data into PostGIS you should use the ST_Dwithin function. Many people use buffer in combination with ST_Intersects or just ST_Distance but that will be very expensive in building buffers or not using indexes in the ST_Distance alternative.

If you have one table with your golf courses and one with your parcels your query could be something like:

SELECT parcels.id, parcels.geom 
from parcels INNER JOIN golf_courses ON
ST_Dwithin(parcels.geom, golf_courses.geom, 100);

This is if your coordinate system is feet-based.

edit: By default the result will be rolled back if the query doesn't end. But that might be configurable in PostgreSQL, I don't know. But I wouldn't be too worried about the query times. The hard work will be to build the index.

What you do when you are going to run a large query is that you first add a limit on how many rows you want returned. If you start with 10000 and it takes half a second then you know quite well how long the whole query will take.

The query time in this case will depend on two things 1) how complex the geometries is. That is because after the bounding box based index has found a lot of aspirants PostGIS will have to recheck which ones really is in range.

2) how much of the work will the index do. If you for example want to find how many lakes is in Norway and how many is in Sweden from a lakes dataset, then the index will not help almost anything because the bounding box of Norway will cover almost all of Sweden. But if you ask like you do, how relative small geometries relate to other small geometries, then the index will do most of the work by just comparing bounding boxes.

But, what s your parcels? Is it some sort of grid?

Have you looked at rester approaches to your problem. That might be the most efficient if the parcels is just a raster.

/Nicklas

share|improve this answer
1  
+1 Nice illustration of @Hairy's point. –  whuber May 12 '11 at 13:19
    
thanks... let me ask you... can postgis stream output data to a csv file with all the parcels that match? I am running over 120 million parcels through and I am concerned about the time this will take, loosing the results and having to possibly restart efforts again while missing some –  NetConstructor.com May 13 '11 at 1:14
    
@NetConstructor.com I updated the answer. –  Nicklas Avén May 13 '11 at 11:45
    
@Nicklas- parcels are another term for US based taxlots. –  RyanDalton May 13 '11 at 15:31
add comment

I think the best strategy depends on how many golf courses you have - and how much memory. You might want to buffer all golf courses by 100' and put them into memory.

In ArcGIS parlance this could be done by creating a geometry bag, looping through each golf course calling ITopologicalOperator.Buffer, and adding it to the geometry bag. Since Buffer creates circular arcs - and I've seen performance issues in the past with circular arcs - you may want to call IPolyCurve.Generalize on it before adding it to the geometry bag.

Once your geometry bag is populated, set ISpatialIndex to set AllowIndexing = true, and then call Invalidate.

Then create a spatialfilter, and use the geometry bag as the geometry.

Then create a featurecursor on your parcels using the spatialfilter, by passing it to IFeatureClass.Search, and loop through each parcel feature.

I've heard 10.1 is going to be 64 bit, and memory is cheap. Disk i/o is not going to improve that much (unless you buy an SSD). So this approach should be more attractive.

It would be useful if ArcGIS had some notion of an execution plan so we could tell it to behave this way when appropriate.

share|improve this answer
add comment

I'm not sure what GIS software you might be using, but in ArcGIS you could run either the Near or Spatial Join tool. Both tools will give you a distance value. You can then select or export out all the records that have a distance less than or equal to 100ft. ET Geowizard also has the Point Distance tool which is a free download.

share|improve this answer
3  
The most efficient and convenient method in ArcGIS is probably "Select by Location" available from the "Selection" menu in Arc 10. Many GISes have similar capabilities. –  whuber May 12 '11 at 2:40
1  
I agree with whuber, The Select By Location in ArcGIS gives you several different methods to run the analysis (intersects, within, bounds, line within, centroid, etc) In MapInfo Professional, you have a SQL function which allows you to do spatial querying (select a.id, b.name, b.id from a, b where a.object intersects b.object... etc...). –  Hairy May 12 '11 at 7:21
3  
select by location will be very slow (uses physical ram and pagefiles) with millions of parcels - task must be broken up to do this efficiently. –  Mapperz May 12 '11 at 14:22
    
If you're using ArcGIS10, I believe a Spatial Index on the layers being queried will help. –  Nathanus May 12 '11 at 18:48
    
any difference in your suggestion if I am running 120 mil+ parcels through this method? –  NetConstructor.com May 13 '11 at 1:16
show 1 more comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.