I have an idea what may work for you. It is going to be based off some assumptions, but it would help narrow down your list of possible identical features. This would not be an automated process, but it would require manually looking at the duplicates. Based off the comments, it seems like the automated tools don't compare attributes so this would help you not accidentially delete features.
(1) Make a copy of your shapefile in case things go wrong.
(2) Add a column to your shapefile as a double.
(3) Calculate area for each feature using the most descriptive (most precise) format you can. Something where rounding may not be an issue.
(4) Run a summary (summarize) on that column. Make sure you select a unique identifier in the summarize and mark both first and last.
(5) In your output table, look for those records where the count field is higher than 1.
(6a) Manually check the features and repeat the process until there are no more duplicates.
(6b) You could just create a list of those unique ids and delete the features through arcpy, but you run the chance of possibly having two unidentical features with the same area.
Another Technique Using ArcPy
As I was constructing the above answer, I thought of the possibility that somehow the multiple authors of this data may have actually used the same unique identifiers for duplicated features. IF that is the case, you may be able to find duplicates through looping in arcpy.
The way I would think about doing this using ArcPy could be taxing on your system and take a little bit.
(1) Make a copy of your shapefile (in case again)
(2) Add a new column to denote duplicates. Something that takes like a 'y' or 'n' or 0 or 1 or whatever would work.
(3) Create a list in python to store the unique identifier.
(4) Run an Update Cursor (
arcpy.UpdateCursor('LAYERNAME')). For each record, check you list to see if it contains that identifier and mark your column for duplicates if it is there.
myList = 
rows = arcpy.UpdateCursor("layername")
for row in rows:
if str(row.UniqueIdentifier) in myList:
row.DuplicateColumnName = "y"
#not there, add it
(5) Then you can compare or do whatever you want with those marked columns.
There are probably better ways to do these comparisons, but those are two that I believe should work or at least get you started.
Based on the comment by elrobis, you could utilize the minimum bounding rectangle to further decrease the chance of removing incorrect features.
Using ArcMap, you can run the Minimum Bounding Geometry tool in Data Management. After checking over the options, I think using the CONVEX_HULL option would probably be best.
If you compare the MBG_APodX/Y1, MBG_APod_X/Y2 fields along with MBG_Orientation for duplicates, you should be able to get a good idea of duplicated features. I would suggest using the Summarize method I described above to compare. Pick one of the vertices (coordinates) from the bounding rectangle to find duplicates. You may get a few incidental 'matches', but once you add in the other vertices plus orientation, it would be a fairly safe bet that the results features are duplicates.
Although I haven't used it and am not quite sure of the results from this tool, you might find examining the resulting shapefile easier if you used the Summary Statistics tool in ArcMap. It looks like you can summarize multiple columns that way instead of my single column option.
I don't think there would be a completely automated way of doing this without having the worry of possibility deleting a not duplicate feature. These methods should help limit the number of features you would need to manually review though.