Using arcpy what is the most efficient way of projecting a selection from a large data set?

Question

I have a large data set X(x1-xn) from which I must select consecutively the attributes A(a1-an) from X.fieldsSRC or X.fieldDST and project the selections.

I have a functioning arcpy script that does the following:

<inside loop incrementing n>
#proj_sr is defined here but omitted
where_clause = fieldSRC = n OR fieldDST = n #simplified for webparser

#create a new feature class based on selection
arcpy.Select_analysis('input', 'temp1', where_clause)
#project selection
arcpy.Project_management('temp1', 'temp2', proj_sr)

#PERFORM OPERATION ON PROJECTED SELECTION

#append selection, and project to destination SR
arcpy.Append_management('temp2', 'final_output')

#we must delete rows from source files otherwise we will treat them each twice
#create layer file in order to select and delete specific rows
arcpy.MakeFeatureLayer_management('input', 'layerfile', where_clause)
#delete rows selected above in layer file
arcpy.DeleteFeatures_management('layerfile')
<end of loop>

This works. In my mind there should be a manner to do the following which would be more efficient.

<inside loop incrementing n>
#proj_sr is defined here but omitted
where_clause = fieldSRC = n OR fieldDST = n #simplified for webparser

#create layer file in order to select and delete specific rows
arcpy.MakeFeatureLayer_management('input', 'layerfile', where_clause)
#project ACTUAL selection rather than a complete shapefile as was done earlier
arcpy.Project_management('layerfile', 'temp2', proj_sr)

#PERFORM OPERATION ON PROJECTED SELECTION

#delete rows selected above in layer file
arcpy.DeleteFeatures_management('layerfile')
<end of loop>

Obviously the arcpy.Project_management() function does not operate in the second code example. It's a rather large data set ('input') and recreating the feature class based on the selection each time does not seem efficient/necessary. It works but it is not fast.

Does anyone have a suggestion?

Answer 1

UPDATED ANSWER

The code below should work using the file geodatabase that Cyrille posted to the Esri Discussion Forum.

import arcpy

prjString = "PROJCS['World_Mercator',GEOGCS['GCS_WGS_1984',DATUM['D_WGS_1984',SPHEROID['WGS_1984',6378137.0,298.257223563]],PRIMEM['Greenwich',0.0],UNIT['Degree',0.0174532925199433]],PROJECTION['Mercator'],PARAMETER['False_Easting',0.0],PARAMETER['False_Northing',0.0],PARAMETER['Central_Meridian',0.0],PARAMETER['Standard_Parallel_1',0.0],UNIT['Meter',1.0]];-20037700 -30198300 10000;-100000 10000;-100000 10000;0.001;0.001;0.001;IsHighPrecision"

arcpy.CopyFeatures_management(r"C:/Support/Cyrille/MFL_Test.gdb/FlightPaths",r"C:/Support/Cyrille/FlightPaths.shp","#","0","0","0")
arcpy.MakeFeatureLayer_management(r"C:/Support/Cyrille/FlightPaths.shp","YVR_layer",'"src" = ' + "'YVR'")

arcpy.Project_management("YVR_layer", r"C:/Support/Cyrille/MFL_Test.gdb/YVR_Proj", prjString)

What I found is that my original code made the layer on a shapefile while Cyrille was making it on a file geodatabase feature class. To workaround what I think is either a MakeFeatureLayer bug or a Project bug what I did was to copy the feature class to a shapefile before performing those two steps on it. That way the where clause is respected.

I suspect the workaround will not be desirable for Cyrille so the next step will be to see if this bug has been reported previously and to report it if not.

ORIGINAL ANSWER

I just ran the test below which uses MakeFeatureLayer_management to apply an SQL expression to the underlying dataset (of 31 polylines), and then uses Project_management to project that layer (of one polyline). The results is a feature class of one projected polyline. So it looks like what you are trying to do in your second code example should work, unless your where_clause is not evaluating correctly. Consequently, my recommendation is to double-check that your where_clause is working.

import arcpy

arcpy.MakeFeatureLayer_management("C:/Temp/test.shp","test_Layer",'"ID" = 555')

arcpy.Project_management("test_Layer","C:/Temp/test.gdb/projectedFC","PROJCS['GDA_1994_Australia_Albers',GEOGCS['GCS_GDA_1994',DATUM['D_GDA_1994',SPHEROID['GRS_1980',6378137.0,298.257222101]],PRIMEM['Greenwich',0.0],UNIT['Degree',0.0174532925199433]],PROJECTION['Albers'],PARAMETER['False_Easting',0.0],PARAMETER['False_Northing',0.0],PARAMETER['Central_Meridian',132.0],PARAMETER['Standard_Parallel_1',-18.0],PARAMETER['Standard_Parallel_2',-36.0],PARAMETER['Latitude_Of_Origin',0.0],UNIT['Meter',1.0]]","#","GEOGCS['GCS_GDA_1994',DATUM['D_GDA_1994',SPHEROID['GRS_1980',6378137.0,298.257222101]],PRIMEM['Greenwich',0.0],UNIT['Degree',0.0174532925199433]]")

Answer 2

Have you tried using a SearchCursor and setting the spatial reference on it so it reprojects the results on the fly? Code below straight outta the help:

import arcpy

desc = arcpy.Describe("d:/base/data.gdb/latlongbnd")

# Create search cursor. Use the spatial reference object from the
#   described feature class so geometries are returned in decimal degrees.
rows = arcpy.SearchCursor("d:/base/data.gdb/buildings", "", desc.spatialReference)

Answer 3

SearchCursor takes a "where" clause - it's the best way to deal with a subset of records.