5

I have selected a single polygon, which is large but not massively complex, from an ArcSDE 9.3 for Oracle feature class with 3.5 million polygons using ArcGIS Desktop 10.0 SP3.

I would simply like to copy it out via ArcPy into a new shapefile (or file geodatabase feature class) and to do so in less than 10 seconds.

So far I have not been able to get it to happen in less than about 5-6 mins.

I think I need to try and take advantage of the processing environment (arcpy.env.extent) during the CopyFeatures, but if I set that to the values returned by getSelectedExtent and do the copy the time taken still seems to be 5-6 mins.

If instead I could get at a centroid x,y (and one that is guaranteed to fall inside the polygon) then I could use that to buffer a tiny amount and use that to set arcpy.env.extent and I would expect the time to come down to around 10 seconds.

However, to get at the centroid I think I still need to use CopyFeatures to get it out into a geometry first which seems Catch 22.

The 10 second target comes from Quickest way to select ArcSDE polygon by point in ArcGIS Desktop using ArcPy? which is another question based on the same dataset.

Improve this question
18
  • Can the polygon be generalized first?
    – Mapperz ♦
    Commented Nov 3, 2011 at 13:47
  • Have you tried timing how long each step in your script takes? For example, try timing the opening of a SearchCursor on a feature layer with a selection. Only the selected rows will be returned. If you have an empty template feature class you could simply insert the read row into it using an InsertCursor. This may be faster than CopyFeatures.
    – blah238
    Commented Nov 3, 2011 at 22:01
  • @Mapperz Generalization is not an option in this case because I'm dealing with cadastral parcels.
    – PolyGeo ♦
    Commented Nov 3, 2011 at 22:49
  • @blah238 I think opening the SearchCursor on 3.5 million features in an ArcSDE for Oracle geodatabase is likely to be very slow (at least at 10.0) but I'll try to test that today in case I am wrong.
    – PolyGeo ♦
    Commented Nov 3, 2011 at 22:55
  • 1
    @PolyGeo, I would suggest adding the solution noted in your question as an actual "answer" below. It will be a lot easier for others to find, and it is perfectly OK to answer your own question if you have found a suiteable answer on your own.
    – RyanKDalton
    Commented Nov 9, 2011 at 23:41

2 Answers 2

Reset to default
4

The solution to this question can be found as a comment to an ArcGIS Idea I posted when I concluded from discussion here that I really had hit a performance limitation that could only be addressed through an ArcGIS/ArcPy enhancement.

Fortunately, the Copy Features I was hoping to do in 10 seconds or less can be done in 4-5 seconds by downloading a script called Make Definition Query from a Selection to run prior to the Copy Features.

I am indebted to Chris Fox for pointing me at it.

Improve this answer
2
  • 1
    Good to hear that selection set performance is improved in arcpy in 10.1. One possible explanation as to why the linked script runs much faster than opening a SearchCursor is that the default option ("OID") looks at the fidset of the layer -- this is probably where the performance improvement is achieved, assuming that is the option you are using.
    – blah238
    Commented Nov 10, 2011 at 1:35
  • @blah238 You are right that OID & fidset are the key to the solution because from Bruce Harold's Python script he says as a comment "Build a list of key field values. In the default case of using the OID field, just grab the FIDSet." Until your comment I'd not twigged to that - thanks!
    – PolyGeo ♦
    Commented Nov 10, 2011 at 22:53
0

Try creating the indexes for the SDE Featureclass. Create spatial and attribute indexes that will definately help you in querying when working in the SDE Environment. For the attribute indexes build index for the column in which you want to perform search. Also analyzing your fc will definately help. Let me know if it works out for you.

Improve this answer
1
  • I'm confident that the issue I'm trying to describe is not one of how the ArcSDE layer is configured. As you will be able to see in an ArcGIS Idea I just posted (ideas.arcgis.com/ideaView?id=08730000000bscKAAQ) the Copy Features can be done sub-second anytime you know an expression to isolate it but if you picked it interactively then you are back to several minutes of wait time to copy it out.
    – PolyGeo ♦
    Commented Nov 9, 2011 at 11:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.