I have a very large dataset with line features that represent a utility, and lines that come off of those utility lines showing connections to buildings. (example shown below - utility line in black and offshoots in purple.)
I developed some iterative code that takes my selected black lines, selects the offshoots attached to it, and checks some of the attributes. If the attribute checks fail, it takes the id code, appends it to a list, then later uses that list to select the purple lines and export them to their own feature. The code works fine, this is not a troubleshooting post. My questions is about efficiency. Has anyone tried something like this in the past, and is there a better way to do it? I ask because this code took about 10 minutes to run on a given study area, and we have >100,000 study areas. If you multiply that out, it would take ~19 years to run on the whole data set, which essentially renders this useless. The study area had ~130 black lines, and ~450 purple lines in it. The tool took 604 seconds to run, which is a little over a second per feature.
My thought is that running nested for loops that each call upon a Arcpy SearchCursor is very slow, and I was wondering if there way any way to get a similar effect with better performance. Are there any links to documentation or other posts? I assume this is a rather niche use case.
with arcpy.da.SearchCursor("Feature1", ['SHAPE@','ID','Attribute1','Attribute2']) as cursor1: for row1 in cursor1: f_shape = row1 ID1 = row1 a1 = row1 a2 = row1 arcpy.SelectLayerByLocation_management("Feature2","BOUNDARY_TOUCHES",f_shape) with arcpy.da.SearchCursor("Feature2", ['SUBTYPE','ID_1','Attribute3','Attribute4','ID_2']) as cursor2: for row2 in cursor2: sub_type = row2 if sub_type <> 1: if a1 == 'criteria': if (a2 == 'something1' or a2 == 'something2'): if row2 <> 1: found_list.append(row2) elif a2 == 'something 3': if row2 <> 1: found_list.append(row2) if row2 <> ID1: if row2 not in found_list: found_list.append(row2)