# Speeding up Arcpy script with lots of for loops and cursors?

I'm running this chunk of code (below) inside a larger script. This section is supposed to find the time for each boat (mmsi) that is closest before and closest after a single detection time. It first finds all the boat times before the detection time and puts them into a fc, then it does the same with all the boat times after the detection time. It then finds the max and min (respectively) of those two fcs and put them into yet another fc. However, this chunk of code is a) messy and b) taking forever to run.

I know from reading about speeding up python scripts that for-loops and cursors can slow things down. However, because of the way they are all nested in code, I can't figure out how to rewrite this in a simpler, faster way. Any thoughts?

arcpy.env.workspace = r'L:\gathr\indonesia\Sara\Date20160104\Detections_UTM.gdb'
workspace = r'L:\gathr\indonesia\Sara\Date20160104\Detections_UTM.gdb'
Geodatabase = arcpy.ListFeatureClasses()
detection_datalist = []
print Geodatabase
for fc in Geodatabase:
fc_name = str(fc)
print fc_name
detection_name = fc_name
fc_path = os.path.join(workspace, fc_name)
cursor = arcpy.SearchCursor(fc_path)
for row in cursor:
time1 = row.getValue("Date_Mscan")
time2 = time1.strftime("%Y-%m-%d %H:%M:%S")
print time2
time = datetime.strptime(time2,"%Y-%m-%d %H:%M:%S")
print time
del cursor
arcpy.env.workspace = r'L:\gathr\indonesia\Sara\Date20160104\fc_processed.gdb'
Geodatabase = arcpy.ListFeatureClasses()
for fc in Geodatabase:
fc_str = str(fc)
suffix = detection_name
if fc_str.endswith(suffix):
where_clause = '"ts_pos_utc_Converted" <= date' + "'%s'" %time
before = os.path.join(r'L:\gathr\indonesia\Sara\Date20160104\fc_processed.gdb', fc + "_before")
arcpy.Select_analysis(fc, before, where_clause)

listdates = []
rows = arcpy.SearchCursor(before)
row = rows.next()
while row:
value = row.ts_pos_utc_Converted
print value
listdates.append(value)
row = rows.next()
print listdates
try:
maxdate = max(listdates)
except:
maxdate = None
print maxdate

after = os.path.join(r'L:\gathr\indonesia\Sara\Date20160104\fc_processed.gdb', fc + "_after")
where_clause = '"ts_pos_utc_Converted" > date' + "'%s'" %time
arcpy.Select_analysis(fc, after, where_clause)

listdates = []
rows = arcpy.SearchCursor(after)
row = rows.next()
while row:
value = row.ts_pos_utc_Converted
print value
listdates.append(value)
row = rows.next()

print listdates
try:
mindate = min(listdates)
except:
mindate = None
print mindate
new_fc = os.path.join(r'L:\gathr\indonesia\Sara\Date20160104\fc_processed.gdb', fc + "_final")
layer = os.path.join(r'L:\gathr\indonesia\Sara\Date20160104\fc_processed.gdb', fc)
arcpy.CopyFeatures_management(layer, new_fc)
with arcpy.da.UpdateCursor(new_fc,"ts_pos_utc_Converted") as cursor:
for row in cursor:
if row[0] not in [mindate, maxdate]:
cursor.deleteRow()

del row, cursor

• use list comprehension and the da cursors Oct 25 '16 at 16:37
• Maybe implied by using da.SearchCursor, but you can skip the entire 'Select_analysis/write new feature class' steps by using your where clause in the da.SearchCursor. That should keep you from writing intermediate data to disk, but if you find you need to do so, whenever possible write intermediate data to the in_memory workspace rather than disk. Oct 26 '16 at 20:50
• All the advice here already is good. Bear in mind that the actual arcpy tools you're calling will run at their own speed - I have a script about 4x as long as yours that would take ~7 minutes tops if there wasn't a step with a Dissolve loop in it. That loop takes about 45 minutes, even using \in_memory. Arcpy is amazing for making scripting easy, but the tools aren't always fast. Oct 26 '16 at 22:02

Use arcpy.da cursors instead of old ones:

The arcpy.da cursors (arcpy.da.SearchCursor, arcpy.da.UpdateCursor, and arcpy.da.InsertCursor) were introduced with ArcGIS 10.1 to provide significantly faster performance over the previously existing set of cursor functions (arcpy.SearchCursor, arcpy.UpdateCursor, and arcpy.InsertCursor). The original cursors are provided only for continuing backward compatibility.

And as mentioned by @Ziggy you can use list comprehensions:

List comprehensions provide a concise way to create lists. Common applications are to make new lists where each element is the result of some operations applied to each member of another sequence or iterable, or to create a subsequence of those elements that satisfy a certain condition.

For example in combination with a da.SearchCursor:

datelist = [date for date in arcpy.da.SearchCursor(infc, ['DATEFIELD'])]

• I've been trying to figure out how to use list comprehensions. So could I use something like this: listdates = [row[0] for row in arcpy.da.SearchCursor(fc, fields)] to accomplish the same thing as: 'listdates = [] fields = "ts_pos_utc_Converted" rows = arcpy.da.SearchCursor(before, fields) listdates = [row[0] for row in rows] for row in rows: value = row[0] print value listdates.append(value)' Oct 28 '16 at 14:59