Take the 2-minute tour ×
Geographic Information Systems Stack Exchange is a question and answer site for cartographers, geographers and GIS professionals. It's 100% free, no registration required.

I have selected a single polygon, which is large but not massively complex, from an ArcSDE 9.3 for Oracle feature class with 3.5 million polygons using ArcGIS Desktop 10.0 SP3.

I would simply like to copy it out via arcpy into a new shapefile (or file geodatabase feature class) and to do so in less than 10 seconds.

So far I have not been able to get it to happen in less than about 5-6 mins.

I think I need to try and take advantage of the processing environment (arcpy.env.extent) during the CopyFeatures but if I set that to the values returned by getSelectedExtent and do the copy the time taken still seems to be 5-6 mins.

If instead I could get at a centroid x,y (and one that is guaranteed to fall inside the polygon) then I could use that to buffer a tiny amount and use that to set arcpy.env.extent and I would expect the time to come down to around 10 seconds.

However, to get at the centroid I think I still need to use CopyFeatures to get it out into a geometry first which seems Catch 22.

(FYI The 10 second target comes from a different question based on the same dataset)

share|improve this question
    
Can the polygon be generalized first? –  Mapperz Nov 3 '11 at 13:47
    
Have you tried timing how long each step in your script takes? For example, try timing the opening of a SearchCursor on a feature layer with a selection. Only the selected rows will be returned. If you have an empty template feature class you could simply insert the read row into it using an InsertCursor. This may be faster than CopyFeatures. –  blah238 Nov 3 '11 at 22:01
    
@Mapperz Generalization is not an option in this case because I'm dealing with cadastral parcels. –  PolyGeo Nov 3 '11 at 22:49
    
@blah238 I think opening the SearchCursor on 3.5 million features in an ArcSDE for Oracle geodatabase is likely to be very slow (at least at 10.0) but I'll try to test that today in case I am wrong. –  PolyGeo Nov 3 '11 at 22:55
1  
@PolyGeo, I would suggest adding the solution noted in your question as an actual "answer" below. It will be a lot easier for others to find, and it is perfectly OK to answer your own question if you have found a suiteable answer on your own. –  RyanDalton Nov 9 '11 at 23:41
show 13 more comments

3 Answers

The writer ArcSDE FME can make many selections before translation ! You can use the Where clause for the Layer Selection as input. Good luck.

Selection with FME

share|improve this answer
    
The Layer Selection is not necessarily being made using a Where Clause. If it were this would be very easy to do using ArcPy and the time would be around 10-12 seconds. Instead the selection is being made interactively in an ArcMap session. It is copying out that polygon that seems to kick in something like a SearchCursor being run through 3.5 million records, presumably in order to determine which one to copy out, that leads to the 5-6 minute time delay. –  PolyGeo Nov 7 '11 at 5:41
    
Good luck with ArcPy –  twisig Nov 7 '11 at 12:27
add comment

Try creating the indexes for the SDE Featureclass. Create spatial and attribute indexes that will definately help you in querying when working in the SDE Environment. For the attribute indexes build index for the column in which you want to perform search. Also analyzing your fc will definately help. Let me know if it works out for you.

share|improve this answer
    
I'm confident that the issue I'm trying to describe is not one of how the ArcSDE layer is configured. As you will be able to see in an ArcGIS Idea I just posted (ideas.arcgis.com/ideaView?id=08730000000bscKAAQ) the Copy Features can be done sub-second anytime you know an expression to isolate it but if you picked it interactively then you are back to several minutes of wait time to copy it out. –  PolyGeo Nov 9 '11 at 11:16
add comment
up vote 4 down vote accepted

The solution to this question can be found as a comment to an ArcGIS Idea I posted when I concluded from discussion here that I really had hit a performance limitation that could only be addressed through an ArcGIS/ArcPy enhancement.

Fortunately, the Copy Features I was hoping to do in 10 seconds or less can be done in 4-5 seconds by downloading a script called Make Definition Query from a Selection to run prior to the Copy Features.

I am indebted to Chris Fox for pointing me at it.

share|improve this answer
1  
Good to hear that selection set performance is improved in arcpy in 10.1. One possible explanation as to why the linked script runs much faster than opening a SearchCursor is that the default option ("OID") looks at the fidset of the layer -- this is probably where the performance improvement is achieved, assuming that is the option you are using. –  blah238 Nov 10 '11 at 1:35
    
@blah238 You are right that OID & fidset are the key to the solution because from Bruce Harold's Python script he says as a comment "Build a list of key field values. In the default case of using the OID field, just grab the FIDSet." Until your comment I'd not twigged to that - thanks! –  PolyGeo Nov 10 '11 at 22:53
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.