Remove Duplicate Polygons Using Open Source Tools

Question

I have a shapefile containing features with duplicate geometries, but different attributes (e.g. a polygon with attribute XX, and an identical polygon with attribute XY). I want to identify the duplicates, then pick the one with the correct attributes, deleting the other.

I've tried topology tools in OpenJump/Kosmo, but without success. They have a 'delete duplicates' tool that does what I want, except seems to arbitrarily delete one of the features rather than letting me pick which one to keep.

I have access to QGIS, OpenJUMP, Kosmo, etc. and am able to install other open source/free tools if required.

Thoughts?

Answer 1

Besides PostGIS, you could also use a topological open source GIS (GRASS):

1. Download and install
2. Start and select the Location manager, use the tool to generate a new project database from your SHAPE file (called "GRASS Location"), see here for a step-by-step guide
3. Import the SHAPE file
4. Use the "v.clean" tool which offers a series of options
5. Export map back to SHAPE file format with "v.out.ogr"

Sidenote: we are planning to offer a WPS for this so that you can polish your data via our Web portal.

Answer 2

This is what I use to do in those cases. As I understand it you will have to do a manual check to decide which one has the right attribute. Then I use to do it in two steps. First I make a table or a view in PostGIS with the duplicated polygons, then I open this table in a desktop GIS together with the original layer where I want to delete the duplicates. That will lead me to the duplicates through my duplicate table and I can delete in the original table or shapefile or whatever it is.

The query to find the duplicates will be something like this if you have a unique id called gid and a geometry column called the_geom:

create table duplicates as
select a.the_geom, a.gid, b.gid from mytable a, mytable b 
where ST_Equals(a.the_geom, b.the_geom) and a.gid!=b.gid;

There is some variants on ST_Equals, but I think ST_Equals is the one you want. That will give you a table with the duplicate geometries and the gid (id) of the first and the second occasion of the polygon. You will get the same pair of duplicates twice though, but that shouldn't matter because you just want them to show you where they are.

HTH Nicklas

Answer 3

If you are using PostGIS 1.5.x, check out the ST_HausdorffDistance() function.
Duplicate geometries will yield a value of 0. Use it like the mentiond self joins. Pretty neat for comparing polygon shapes.

Answer 4

A more simple way to do. Calculate area in your desktop GIS and order by area ascending or descending. You will now have to check every line to see the duplicated objects in your attributes table. It's based on a supposition you never have exactly the same area for objects not duplicated althought it can be sometimes wrong.