I have rows in a table that have duplicate ID values in one field. I would like to identify these duplicates, keep the ID value of the earliest entry, and null out the ID field for the other duplicates. Since there is no date of entry field, I am relying on the ObjectID to identify the earliest entry (the lowest number). I am using a file geodatabase table, and operating in Arc 10.3.
For example:
999999 repeats twice, therefore I want to keep the ID for ObjectID 1, but null out the fm_uniqid for ObjectID 2. The same goes for 999998, which is repeated 3 times. I did the following manually.
from this:
to this:
The following code (source), helps me identify the duplicates, but now I'm wondering about my next steps.
import arcpy
fc = r"C:\Users\YCZ\testgdb.gdb/test_table"
checkfield = "fm_uniqid"
def checkDuplicate(fc, checkfield):
vals = [row[0] for row in arcpy.da.SearchCursor(fc, checkfield)]
dct = {x:vals.count(x) for x in vals}
for k, v in dct.iteritems():
if v > 1:
print k, v
if __name__ == '__main__':
checkDuplicate(fc, checkfield)
I am looking for advice on how to continue before I attempt any code. I believe the best way may be to put these identified duplicates into a list, then iterate through each value individually: Select by attribute for duplicated fm_uniqueIDs, identify the lowest objectID value, and set the fm_uniqid of all rows with an objectID greater than the lowest value, to null.
This seems quite tedious for a large dataset (40,000+ records)--are there recommendations of other approaches?