I have a feature class with 1 million records. Which is the faster way to copy data to another feature class: Feature Class to Feature Class or Insert Cursor?
-
I dont know the answer but using in_memory workspace (desktop.arcgis.com/en/arcmap/10.3/analyze/modelbuilder/…) will speed it up if you only need a temporary fc for further analysis– BERAMay 27, 2016 at 13:45
-
3I think you should test before asking a question like this, because that is so easy to do. I would expect FC2FC to win.– PolyGeo ♦May 27, 2016 at 19:34
-
I've done a number of performance calculations on one million row ASCII files, but not from an FC source. You really need to edit the question to specify the type of feature class (source, topology class, and mean attribute width) before any answer would be meaningful (and then it would only be meaningful on the platform where the operation was performed). Just code the comparison with your dataset, and then you'll know. You should certainly use a DA InsertCursor.– VinceMay 28, 2016 at 3:17
-
Thanks all. It is a Topology feature class. I am now comparing both the processes on 100,000 records because I believe 1 million is a tad bit difficult to test. yes I am using DA Insert Cursor.– Astha DevraMay 31, 2016 at 8:46
1 Answer
I created three sample random point datasets - 10 features, 100k features, 1m features - and tested the following options
- Copy (
arcpy.Copy_management()
) - Insert Cursor (
arcpy.da.InsertCursor()
) - Feature Class to Feature Class (
arcpy.FeatureClassToFeatureClass_conversion()
) - Copy Features (
arcpy.CopyFeatures_management()
)
Here are the results:
Copying 10 features
Copy
Copy: 0:00:01.360000
DA Insert Cursor:
DA Insert Cursor: 0:00:01.004000
Feature Class to Feature Class
Feature Class to Feature Class: 0:00:00.408000
Copy Features
Copy Features: 0:00:00.514000
Feature class to feature class fastest, Copy slowest
Copying 100k features
Copy
Copy: 0:00:06.154000
DA Insert Cursor:
DA Insert Cursor: 0:00:09.613000
Feature Class to Feature Class
Feature Class to Feature Class: 0:00:07.026000
Copy Features
Copy Features: 0:00:07.770000
Copy fastest, Insert Cursor slowest
Copying 1m features
Copy
Copy: 0:00:48.664000
DA Insert Cursor:
DA Insert Cursor: 0:01:25.717000
Feature Class to Feature Class
Feature Class to Feature Class: 0:01:05.846000
Copy Features
Copy Features: 0:01:13.412000
Copy fastest, Insert Cursor slowest
Here is the code I used to test. I had to add an extra copy right at the start as I found the very first function I fired each time I ran the script took several seconds slower to run - not sure if that was due to first access of gdbs or arcpy or something else. Subsequent tools didn't take so long to run.
import arcpy
from datetime import datetime
numbers = '1m' # 10, 100k, 1m
inputFC = r"N:\GISSE\SpeedTestInputs.gdb\Pt_{}_Random".format(numbers)
outputFC_copy = r"N:\GISSE\SpeedTest.gdb\Pt_{}_copy".format(numbers)
outputGDB_dainsert = r"N:\GISSE\SpeedTest.gdb"
outputFCName_dainsert = r"Pt_{}_dainsert".format(numbers)
outputFC_dainsert = r"N:\GISSE\SpeedTest.gdb\Pt_{}_dainsert".format(numbers)
outputGDB_fc2fc = r"N:\GISSE\SpeedTest.gdb"
outputFC_fc2fc = r"Pt_{}_fc2fc".format(numbers)
outputFC_copyFeatures = r"N:\GISSE\SpeedTest.gdb\Pt_{}_copyFeatures".format(numbers)
# Copy Starter - to eliminate any delays for first task
# due to having to access tools, gdb, etc.
arcpy.Copy_management(r"N:\GISSE\SpeedTestInputs.gdb\Pt_10_Random", r"N:\GISSE\SpeedTest.gdb\Pt_10_copyStarter")
#################
#################
# Start Testing #
#################
#################
print ""
print "Copying {} features".format(numbers)
print ""
########
# Copy #
########
print "Copy"
copyStart = datetime.now()
arcpy.Copy_management(inputFC, outputFC_copy)
copyEnd = datetime.now() - copyStart
print "\tCopy: {}".format(copyEnd)
####################
# DA Insert Cursor #
####################
print "DA Insert Cursor:"
daInsertStart = datetime.now()
arcpy.CreateFeatureclass_management(outputGDB_dainsert, outputFCName_dainsert, "POINT", inputFC, "DISABLED", "DISABLED", inputFC)
fields = ['SHAPE@', 'SomeNumbers']
with arcpy.da.InsertCursor(outputFC_dainsert, fields) as iCursor:
with arcpy.da.SearchCursor(inputFC, fields) as sCursor:
for row in sCursor:
iCursor.insertRow(row)
daInsertEnd = datetime.now() - daInsertStart
print "\tDA Insert Cursor: {}".format(daInsertEnd)
##################################
# Feature Class to Feature Class #
##################################
print "Feature Class to Feature Class"
fc2fcStart = datetime.now()
arcpy.FeatureClassToFeatureClass_conversion(inputFC, outputGDB_fc2fc, outputFC_fc2fc, where_clause="", field_mapping="", config_keyword="")
fc2fcEnd = datetime.now() - fc2fcStart
print "\tFeature Class to Feature Class: {}".format(fc2fcEnd)
#################
# Copy Features #
#################
print "Copy Features"
copyFeaturesStart = datetime.now()
arcpy.CopyFeatures_management(inputFC, outputFC_copyFeatures, config_keyword="", spatial_grid_1="0", spatial_grid_2="0", spatial_grid_3="0")
copyFeaturesEnd = datetime.now() - copyFeaturesStart
print "\tCopy Features: {}".format(copyFeaturesEnd)
####################
####################
# Testing Complete #
####################
####################
Just-in-case, to eliminate any possibly slowdown in the geodatabase I also ran each step individually, clearing out the geodatabase between each run. The results here were the same, so there didn't appear to be any slow-down affecting the later tools in the test.
-
Impressive work put into the answer by #Midavalo. The operations are not equivalent: I found that using CopyFeatures to copy a 3D source into an empty 2D destination will set it to 3D. Copying with cursors will leave the destination 2D. Jul 27, 2019 at 14:37