Is there any simple way for me to run millions of parcels through some type of script to see if a shape file parcel is on a golf course? For example if I have a layer that has all the golf courses and parcel layers and let's say the max distance would be 100 feet then how would i be able to extract just those parcels?
Since you have quite a lot of parcels that you want to test you should look for a software that uses spatial indexes for this type of calculations.
And also be sure to use a method in that software that really uses the indexes.
If you import your data into PostGIS you should use the ST_Dwithin function. Many people use buffer in combination with ST_Intersects or just ST_Distance but that will be very expensive in building buffers or not using indexes in the ST_Distance alternative.
If you have one table with your golf courses and one with your parcels your query could be something like:
This is if your coordinate system is feet-based.
edit: By default the result will be rolled back if the query doesn't end. But that might be configurable in PostgreSQL, I don't know. But I wouldn't be too worried about the query times. The hard work will be to build the index.
What you do when you are going to run a large query is that you first add a limit on how many rows you want returned. If you start with 10000 and it takes half a second then you know quite well how long the whole query will take.
The query time in this case will depend on two things 1) how complex the geometries is. That is because after the bounding box based index has found a lot of aspirants PostGIS will have to recheck which ones really is in range.
2) how much of the work will the index do. If you for example want to find how many lakes is in Norway and how many is in Sweden from a lakes dataset, then the index will not help almost anything because the bounding box of Norway will cover almost all of Sweden. But if you ask like you do, how relative small geometries relate to other small geometries, then the index will do most of the work by just comparing bounding boxes.
But, what s your parcels? Is it some sort of grid?
Have you looked at rester approaches to your problem. That might be the most efficient if the parcels is just a raster.
I'm not sure what GIS software you might be using, but in ArcGIS you could run either the Near or Spatial Join tool. Both tools will give you a distance value. You can then select or export out all the records that have a distance less than or equal to 100ft. ET Geowizard also has the Point Distance tool which is a free download.
|show 1 more comment|
I think the best strategy depends on how many golf courses you have - and how much memory. You might want to buffer all golf courses by 100' and put them into memory.
In ArcGIS parlance this could be done by creating a geometry bag, looping through each golf course calling ITopologicalOperator.Buffer, and adding it to the geometry bag. Since Buffer creates circular arcs - and I've seen performance issues in the past with circular arcs - you may want to call IPolyCurve.Generalize on it before adding it to the geometry bag.
Once your geometry bag is populated, set ISpatialIndex to set AllowIndexing = true, and then call Invalidate.
Then create a spatialfilter, and use the geometry bag as the geometry.
Then create a featurecursor on your parcels using the spatialfilter, by passing it to IFeatureClass.Search, and loop through each parcel feature.
I've heard 10.1 is going to be 64 bit, and memory is cheap. Disk i/o is not going to improve that much (unless you buy an SSD). So this approach should be more attractive.
It would be useful if ArcGIS had some notion of an execution plan so we could tell it to behave this way when appropriate.