How not to Copy Features when too many would be output?

Question

This question has already been posed to the ArcGIS Geoprocessing forum but I have had no responses there.

I have a 3.5 million polygon layer (cadastre) stored in Oracle via ArcSDE. I use a polygon feature chosen by an end user from a second layer to copy out the cadastral polygons that intersect it (via Copy Features). I can get this to work fine for most polygons, and get the few seconds to few minutes response I need, by using the polygon the user chooses to set the geoprocessing extent (arcpy.env.extent) prior to executing Copy Features.

However, for a few polygon features that the user might choose (think petroleum or gas pipeline corridor), the number of polygons that would be copied out can number up to about 200,000. These can be copied in about 15 mins, but rather than get all of these back so that I can write them into a very bloated PDF report, I would like to save time and pages by having the option to set a maximum number of features that can be copied out i.e. be able to set a MaxRecordCount = 500 that would stop the Copy Features once 500 had been copied.

Is there a way to tell Copy Features to only copy up to a maximum number of features?

Alternatively, is there a fast way to get a count of how many features in an ArcSDE layer are within the current geoprocessing extent?

UPDATE: No real progress made but the results and code below may illustrate better how long it takes to use SelectLayerByLocation and GetCount (at least at ArcGIS 10.0) against the 3.5 million polygon ArcSDE layer even when the tenement chosen only overlaps 9,000 polygons (original example takes 10x as long).

This is the output:

Copying one tenement to in_memory takes 2.87037144971 seconds
Setting arcpy.env.extent to that one tenement (in_memory)
SelectLayerByLocation takes 64.4129756363 seconds to intersect cadastre with this one tenement (in_memory)
8961 cadastral parcels found in Search Area
GetCount takes 206.410587153 seconds to come up with that number

This is the code:

import arcpy
import time

mxd = arcpy.mapping.MapDocument("C:\\MXDS\\CustRep\\Maps\\Search_layers_V1.mxd")
df = arcpy.mapping.ListDataFrames(mxd,"Layers")[0]
searchAreaLayer = arcpy.mapping.ListLayers(mxd,"Exploration Permits Petroleum (EPP)",df)[0]
searchAreaLayer.definitionQuery = "TENURENO = 644"
start = time.clock()
arcpy.CopyFeatures_management(searchAreaLayer,"in_memory/SearchArea")
elapsed = (time.clock() - start)
print "Copying one tenement to in_memory takes " + str(elapsed) + " seconds"

print "Setting arcpy.env.extent to that one tenement (in_memory)"
arcpy.env.extent = "in_memory/SearchArea"

cadastreLayer = arcpy.mapping.ListLayers(mxd,"Lot and Plan",df)[0]

arcpy.MakeFeatureLayer_management("in_memory/SearchArea","SearchArea")

start = time.clock()
arcpy.SelectLayerByLocation_management(cadastreLayer,"INTERSECT","SearchArea")
elapsed = (time.clock() - start)
print "SelectLayerByLocation takes " + str(elapsed) + " seconds to intersect cadastre with this one tenement (in_memory)"

start = time.clock()
result = arcpy.GetCount_management(cadastreLayer)
numRowsInSearchArea = int(result.getOutput(0))
print str(numRowsInSearchArea) + " cadastral parcels found in Search Area"
elapsed = (time.clock() - start)
print "GetCount takes " + str(elapsed) + " seconds to come up with that number"

Answer 1

Why not just do a Select Layer by Location instead of setting the geoprocessing extent? Alternatively, you could use a SearchCursor and stop after the 500th row.

Answer 2

This has now been requested as an enhancement at ArcGIS Ideas

Kim Ollivier pointed me in the direction of what will be a viable workaround for me, inspired by ArcStorm, in the ArcGIS Geoprocessing forum.

Answer 3

When using arcpy, you are not going for performance, you are going for convenience. This is explained further in a previous answer about arcgis scripting