# How to find and merge duplicate points?

I have two point shapefiles: Red and Blue. Each point shapefile has a z value which represents a radius (i.e. represented as buffers in the example). A duplicate point is defined as a point that lies within Z distance of another point, which can range from 2 - 10 m. In this example there are ten pairs of duplicate points--seven pairs overlap perfectly, while three pairs overlap by falling within another point's Z value.

I would like to keep only one of each duplicate within Z and merge the two shapefiles together. Solutions in ArcGIS, Python, and R would be especially helpful. How can I find and merge duplicate points?

-
I have a solution, but first, how are you going to choose what point to keep? – nagytech Jul 25 '12 at 19:48
Since both of the points can be considered correct by themselves, though not together, a random selection of the two would be ideal. – Aaron Jul 25 '12 at 20:01
Would an average of the two points be acceptable? – nagytech Jul 25 '12 at 22:06
@Geoist: Yes, an average could work. – Aaron Jul 26 '12 at 12:03
If you have two blue [or red] points within their distance measures do you want two points in the output? Or what about more than two points [of any colour]? What about if A overlaps B overlaps C overlaps D but A does not overlap D? Output one point, or two [one from A & B and one from C&D]? – Spacedman Jul 29 '12 at 7:34

I was going to write my solution on the premise that you only had two layers, but I realized that a single layer solution would be both easier and more extensible. Please have a read through the following points, and let me know if you need clarification on any points.

# 1. Merge Data Into One Class / Shapefile

Just execute the Merge tool on each existing class / shapefile until you only have one to work with.

# 2. Calculate Z Geometry

If the geometry is Z-Enabled, you need to calculate the Z value into the attribute table in order to apply it to the buffer command.

Note: You will need to reference this process in a later step as well.

2.2. Set Field Type

2.3. Calculate Geometry

# 3. Geoprocessing

3.1. Buffer your points using the Z_Value field as a reference. Make sure to check the appropriate dissolve function.

3.2. Execute the Multipart to Singlepart tool to explode the data.

3.3. With the exploded data, add two new fields called 'Centroid_X' and 'Centroid_Y'.

3.4. Use the process defined in step 2 to extract the X and Y coordinates to the Centroid columns you just created.

3.5. Export the Attribute table to a DBF file. (Be sure to set the file type to DBF.)

3.6. Import the DBF file into the map, and add the X, Y data to the map frame.

# 4. Done

-
Thanks for the really good ideas! – Aaron Aug 1 '12 at 12:28
+1 for being a well presented answer! – Jeffrey Evans Dec 28 '15 at 20:47

The Integrate (Data Management) and Delete Identical (Data Management) tools in ArcGIS solves this problem. Simply add the point shapefile and choose the XY Tolerance. All points within the XY tolerance distance will be assumed to be the same. The resulting point is the mean distance between the two original points.

Integrate is used to maintain the integrity of shared feature boundaries by making features coincident if they fall within the specified x,y tolerance. Features that fall within the specified x,y tolerance are considered identical or coincident.

-
How would you assign the tolerance per point given that the radius is variable? – nagytech Feb 18 '14 at 0:22
In this case the tolerance is the max radius distance 10m. You can see from the screenshot that duplicates have the same radius and only two points are ever within eachother's radius. – Aaron Feb 18 '14 at 13:27
Perhaps you should re-phrase your question then.. – nagytech Feb 18 '14 at 22:50
@Geoist Thanks for your interest. What exactly are you unclear about? I would be happy to update the post with your suggestions. – Aaron Feb 18 '14 at 22:55

Hope I understood your question correctly..

Have you tried "Near" tool in proximity with Radius as 10m. Use RED one as input feature and Blue as Near features.. It will give the Id of Blue which are falling with RED point buffer of 10m. Based on that id (in output of near tool), you can delete those in Blue and then use the Merge tool, to combine RED and remaining Blue.

-
+1 Thanks vadivelan--some great suggestions here. – Aaron Jul 25 '12 at 20:10
@vadivelan That won't help given that the radius is not static. – nagytech Jul 25 '12 at 22:01
Yes Geoist, i missed that radius is variable. May be he can try to draw buffer based on radius value (there is an option to select field which has the radius value, but the field should be populated with z value).. then intersection/spatial join tool will help.... unfortunately this is leading to lot of manual work initially.. – vadivelan Jul 25 '12 at 23:18