4

In an effort to reduce processing time, I am looking for ways to pre-process data within my script. I have ~18,000 featureclasses with ~500 features each (all simple circle polygons) from which I want to eliminate coastal land area (another simplified polygon) using the Erase function.

My idea for reducing overall processing time is to check for overlap between each feature class and the coastline polygon. If there is overlap, the script should erase the land from the features. If there is no overlap, the script should move on, and look at the next feature class.

Specific questions:

  1. What is the most efficient, Pythonic way to say "if there is overlap, do XX?" I cannot find documentation to do something similar to "if Overlap == True" besides the method:

    MakeFeatureLayer-->SelectFeatureByLocation-->CopyFeature-->GetRowCount--> if RowCount>0

  2. Would this actually reduce processing time or am I crazy?

I'm working locally on my machine, running the script in IDLE, with ArcGIS 10.1 (with all licenses) on a 32-bit Windows 7, 3.33GHz Intel DuoCore, 4GB RAM computer.

Improve this question
16
  • You could use selectByLocation with Intersect, iteratively on every file, making a new file when you find intersecting geometry. Then run your script on those new files. Honestly though, you are not getting memory errors, general function failure 999999s or topology errors - I would count myself lucky and let the sucker run for a week :) Not a great answer I know...
    – Tom
    Apr 24, 2013 at 19:07
  • @Tom - Thanks for the suggestion - it is a good idea, though at this point the time trade-ff isn't as good. As it is I'm already working on features that have been through several selections/pre-processing, which is why I'm looking for an in-script option while letting this functional (yay it's working!) script run while I continue to tweak it to try speeding it up. Thanks again! Also the conversation continues on the first question if your curious about other possible solutions (see link in question above).
    – SharonB
    Apr 24, 2013 at 19:24
  • 2
    I liked the suggestion about the min bounding rectangle as a quick 'overlap' check. You could always implement that OR selectByLoc w/ Intersect in your script. Before you fill your fileList by walking your directory, perhaps you could throw in code that runs one of the suggested methods, outputting to a new dir, and walk that one? However I am unsure how long an erase would actually take if there is no erase to perform, so you may not see any gain. I'll keep thinking! How about multiple computers crunching away on subsets? :)
    – Tom
    Apr 24, 2013 at 19:28
  • 3
    You should consider doing these calculations with raster operations: buffering the points would be unnecessary (saving loads of computational time and disk I/O).
    – whuber
    Apr 24, 2013 at 20:30
  • 1
    @SharonB The erase function is smart enough for your needs (anecdotal evidence from my tries). If you try to erase and there is no overlap, the algorithm will simply quickly return the unchanged data. I believe you need to make sure your shapefiles have a spatial index and should all be in the same coordinate system (meaning there is an index that saves bounding boxes for each feature). On the other hand, as I have no access to Erase, I wrote my own erase that uses intersect, and it seems to be quite fast too (I run it with about 600,000 polygons, cutting off intersections).
    – Michalis Avraam
    Apr 25, 2013 at 23:50

2 Answers 2

Reset to default
3

Coastline data is a case where bounding box spatial indexing can be of limited benefit. Consider the example below.

enter image description here

Both polygons A and B are within the bounding box of the land polygon. The software may need to work through each of the thousands of vertices to identify if the polygons actually overlap. This work could be reduced by simplifying the land polygon and reducing the vertex count.

Alternatively you can divide the land polygon into a grid. This will make the spatial indexing much more useful and reduce the number of vertices to process. Based on the grid shown below, only the tiles in red would be identified as intersecting using bounding box spatial indexing. Polygon B would show as not intersecting without needing to process the land vertices at all.

enter image description here

Improve this answer
2

Don't know if this is still an open question, but an idea that occurred to me was that if you had the points/centroids from which your circle polygons were generated, you might be able to more quickly assess their euclidean distance from the coast, select those below a threshold, and then relate these points to the polygon feature class and use that for a single erase operation.

Improve this answer
0

Not the answer you're looking for? Browse other questions tagged or ask your own question.