5

I have a question regarding removal of duplicate points, which I did not find answered here yet. In my point shapefile there are some points at the same coordinates. Some of them have the exact same attributes and some have different attributes.

Is there a way to identify and delete the "real" duplicates (those with the same coordinates AND attributes) leaving out those which have the same coordinates but differ in the other attributes?

I only found tools yet, which only go by the coordinates in identifying duplicates but ignore the attributes.

4

As an alternative outside QGIS and working for any similar problem :

  • For point features, you can create a X and Y column with the point geometry (should you need one day to do the same for lines or polygons, you will need to get the WKT data instead)
  • Export all the attributes to Excel
  • Use the "Remove Duplicates" function located in the 'Data Tools' group, within the Data tab of the Excel ribbon (source)
  • After removal, import back from Excel to QGIS the points via the X and Y column (or use the WKT column for more complex entities).

The nice thing about this method is that you can use either one or multiple fields to de-duplicate, allowing you to choose what you prefer.

Remove duplicates

9

You may run this code (just copy-and-paste it in the Python Console):

layer = iface.activeLayer() # This command will load the layer currently selected

# Create the output layer
crs = layer.crs().toWkt()
outLayer = QgsVectorLayer('Point?crs='+ crs, 'cleaned' , 'memory')
prov = outLayer.dataProvider()
fields = layer.pendingFields() # Fields from the input layer
prov.addAttributes(fields) # Add input layer fields to the outLayer
outLayer.updateFields()

all_points = {}
index = QgsSpatialIndex() # Create a Spatial Index
for ft in layer.getFeatures():
    index.insertFeature(ft)
    all_points[ft.id()] = ft

ids_already_processed = [] # It will store all the feature ids already processed

for feat in layer.getFeatures():
    attrs_already_added = [] # It will store all the attributes already processed
    if feat.id() not in ids_already_processed:
        attrs_already_processed = []
        idsList = index.intersects(feat.geometry().boundingBox())
        attributes = [all_points[id].attributes() for id in idsList]
        geometries = [all_points[id].geometry() for id in idsList]
        for attr in attributes:
            if attr not in attrs_already_added:
                outGeom = QgsFeature()
                outGeom.setAttributes(attr)
                geom_index = attributes.index(attr)
                outGeom.setGeometry(geometries[geom_index])
                prov.addFeatures([outGeom])
                attrs_already_added.append(attr)
        ids_already_processed += [all_points[id].id() for id in idsList]

# Add the layer to the Layers Panel
QgsMapLayerRegistry.instance().addMapLayer(outLayer)

It will return a new point memory layer that stores only the features that have the same coordinates, but not the same attributes, as desired.

  • 1
    It's very nice the way you comment your code....very useful for learners. Thanks :) – gisnside Jul 3 '17 at 11:59
  • @gisnside, thanks, Maybe it was not the best way for doing the job, but at least it was readable and 100% working too. :) – mgri Jul 3 '17 at 12:22
  • It answer the question within qgis, which is what was asked. It's good to have all different ways. Yours is as valuable as any other one :) – gisnside Jul 3 '17 at 12:28
  • @gisnside maybe my code scared the OP, even if it performs all the calculations in a single step. However, good tip also from you. – mgri Jul 3 '17 at 12:30
  • I don't see it as a problem, having the code and a non code solution is a great set of answers for many users... team work ! – gisnside Jul 3 '17 at 13:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.