I am using python script and ArcGIS Desktop 10.8.1 to synchronize two datasets. There are many (thousands) of duplicate features that I would like to drop in an output. How can I specify that, in the case of identical features across datasets, I want to keep data from dmanfile and delete duplicates from cadfile. I am totally new to python, but here are the relevant parts of the code I have so far:

#input files from user console
dmanfile = input("./DSchemaFix1/D_Man_OG/D_Man_Fields_Complete.shp")
cadfile = input("./DSchemaFix1/CurrentCAD/CurrentCADFiles.shp")

gdf = gpd.read_file(dmanfile)
cad = gpd.read_file(cadfile)

gdf_appended = cad.append(gdf)

gdf_dupdropped = gdf_appended.drop_duplicates(keep='first', subset=['StreetName','Address', 'Apartment','ZipCode'])
  • 1
    Oh, you're right, I am using Geopandas. No, they don't share geometry. Its a dataset of address points. Many points in the updated set are displaced, so I am trying to match them by several fields that make up a full address.
    – Olive
    Commented Dec 15, 2020 at 19:42

1 Answer 1


Add a source column, sort by it and drop duplicates:

import geopandas as gpd

dman = gpd.read_file('/home/bera/Desktop/tempgis/dman.shp')
dman['source'] = 'dman'
cad = gpd.read_file('/home/bera/Desktop/tempgis/cadfile.shp')
cad['source'] = 'cad'

both = dman.append(cad)
no_dups = both.sort_values(by='source', ascending=False).drop_duplicates(subset=['StreetName','Address', 'Apartment','ZipCode'], keep='first') #dman come before cad and are kept

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.