# Finding duplicate attributes and selecting one to delete based on another column

I have a very large polygon attribute table with multiple pairs of duplicate superimposed polygons, whose areas are identical. There is also another column which indicates the years these polygons were made, with some being made later in time than others. See screenshot:

I would like to be able to sort through the table, and label each paired duplicate as a "pair", then delete the one that is older.

Here is the code I have come up with so far. Field calculator code block for a "Pairs" column that labels the pairs:

``````d = {}
def sort_by_pairs(inValue, year):
if inValue in d.keys:
d[inValue].append[year]
else:
d[inValue] = [year, ]

def label_pairs(inValue, year):
sort_by_pairs(inValue, year)
if len(d[inValue]) > 1:
return "Pair"
else:
return "No pair"
``````

Code block for a second column which would then sort by year (adapted from here):

``````uniqueList = []
d = {}
def isDuplicate(inValue, year):
if inValue not in uniqueList:
uniqueList.append(inValue)
d[inValue] = year
return 'No'
else:
if year < d[inValue]:
return "Delete this, keep other."
elif year > d[inValue]:
return "Keep this"
else:
return "Years equal --> check!"
``````

This solution is not perfect, but would be workable for my situation - however, I can't get the code for the first column to run. Is there a way to do multiple functions in the code block window, i.e. run the sort into a dictionary on the entire column first, then compare each value to the dictionary? If I understand correctly, that is what the issue is. Currently it's producing the ERROR 000539:Error running expression label_pairs(1.775, 2005) and TypeError: argument of type 'builtin_function_or_method' is not iterable.

I am running ArcGIS Desktop 10.5 and Python 2.7.

EDIT:

Tried adapting code from here and here so I could get the OID's after first manually selecting the duplicate pairs (getting the OID list worked), and then iterating through as an updatecursor (this did not work), to fill the "Pairs" column. This seems to be a slightly more efficient way to do it, but can't get it to work either.

Python window code (not FieldCalc, but the general one):

``````#this worked
lyr = arcpy.mapping.Layer('path')
set = lyr.getSelectionSet()

#this didn't work
if set:
with arcpy.da.UpdateCursor(r'Full data - no identical\Fields_noRepeat_manDelete', 'Pairs') as curs:
for row in curs:
row = ("Pair exists", )
else:
print("No selection")
``````

Not sure where the mistake is. How would I iterate through a set of ObjectIDs with an UpdateCursor?

• @BERA - Just added a screenshot. Hope that helps clarify the question a bit! Commented Feb 17, 2022 at 14:14
• It would be easier to sort then Delete Identical, but maybe you want to code it instead
– Bera
Commented Feb 17, 2022 at 14:30
• That is an option, but I think Delete Identical would just delete one of each randomly? I wanted to be able to choose the older one to delete by year, or datestamp. Also, it would be helpful to sort the data by pairs, so we can look at it later and compare the points, to find the reason for duplication. Commented Feb 17, 2022 at 14:51
• Some of the other corresponding data is different for the point pairs, so we would need to look at both to decide which ones to keep, if that makes sense. Commented Feb 17, 2022 at 15:06
• No it will delete either first or last, cant remember. So it should work
– Bera
Commented Feb 17, 2022 at 15:14

It might not be efficient with all the SearchCursors, but this worked for me.

``````#find identical polygons
fc = 'Polygon'
arcpy.management.FindIdentical("Polygon", "Polygon_FindIdentical", "Shape", output_record_option= "ONLY_DUPLICATES")

#for each series of identical polys in the table (feat_seq)
for seq in set(row[0] for row in arcpy.da.SearchCursor('Polygon_FindIdentical', "FEAT_SEQ")):
#get list of duplicate polygon OIDs
dupOID = [row[0] for row in arcpy.da.SearchCursor('Polygon_FindIdentical', "IN_FID",where_clause = "FEAT_SEQ="+str(seq))]
#define query to pull out just those OIDs
query_dup = "OBJECTID IN ({:s})".format(','.join(f"{x}" for x in dupOID))
#get max year for those polygons
maxyear = max([row[0] for row in arcpy.da.SearchCursor('Polygon', "YEAR",where_clause = query_dup)])
#check for multiple max years?
multimax = len([row[0] for row in arcpy.da.SearchCursor('Polygon', "YEAR",where_clause = query_dup+" AND YEAR="+str(maxyear))])
#loop through and flag based on year
with arcpy.da.UpdateCursor('Polygon',['YEAR','Duplicate_comment'],where_clause = query_dup) as cursor:
for row in cursor:
if row[0] < maxyear:
row[1] = "delete"
elif multimax > 1:
row[1] = "check"
else:
row[1] = "keep"
cursor.updateRow(row)
``````