I am new to a company that makes use of a rather large database of point data. For whatever reason, the woman who was here before me made a huge amount of duplicate points - the attributes may be a little different but the coordinates are the same.

I've found that I can remove the points either in QGIS using the mmqgis plugin, or in ArcMap using the "delete identical" tool. However, when I use either method, I have noticed that there are some points that are removed completely. For most, it leaves one point but I have found a few that are completely gone.

Do any of you know of a method to add unique features to an attribute table?

I'm thinking that I can compare the original table to the "cleaned" one and add the missing points back. Loo


Since you are dealing with coordinates that are the same, Near (Analysis) will do the trick. Use a very small search radius such as 1 foot or one meter. Points that have a value greater than -1 in the newly added Near_FID column are duplicates.


@Paul had it:

You should try with the Shape field [instead of the Lat/Long fields]. It should be more exact and is guaranteed to be up-to-date.

Turns out that the geocoding software we use (MapMarker) threw a lot of the points into zip code centroids, giving them the same coordinates. I had to go in and look at the scare that MapMarker gives the points and run "delete identical" only on exact matches. Now I get to manually geocode ~250 points.


You might try this: If your attribute table already has x,y fields (if not, then create two new fields to hold the x,y values and calculate the geometry to get the coordinates) then create a new string field and calculate the x,y's into the new field (field calculator > expression: [X] & " , " & [Y]). Once this is done you can use the summary statistics tool against the new field and use "count" for the statistic type. This will return a table showing which x,y combinations have exact duplicates (the "count" field will show them with a number greater than 1). Since I don't know how many records you have in your file this next part may or may not work for you: You could add a field called "duplicates" to you summary table and then select all of the records with a value > 1 and calculate the records to "yes". Then switch the selection and calculate the rest of the records to "no". Then set up a one-to-many relate between your points and the summary table, select the records with "duplicates = 'yes'" and deselect the records you want to keep and then delete the rest. This may be too cumbersome for your purpose but maybe it will point you in the right direction.


http://www.gpsbabel.org/ GPS babel has many options to help you manage point data based on attributes, distance, etc

Not the answer you're looking for? Browse other questions tagged or ask your own question.