Finding differences between shapefiles using ArcGIS for Desktop?

Question

Every month I have to QA/QC some streets data for an entire county (over 47,000 records). The first step is to compare the two files. Currently, I'm using a tool that was created in house via VBA approx 5 years ago. It works compares the two files and reports back the differences based on a few factors (added records, deleted records, address changes, city changes, segment length/vertices changes). I then export these to individual shapefiles. Additionally, the tool works in ArcGIS 9.3 but will no longer work in ArcGIS 10.1

The tool works great, but it takes approximately an hour to complete.

I've started to work on hopefully improving this tool to speed up the performance. I've looked into a variety of methods including the Compare Feature Tool and Select by Location.

Both of these tools work significantly faster, however, they don't seem to capture everything that I need. The big issue I have with the Compare Feature Tool is that I cannot choose which field to compare on. It defaults to ObjectID when I need to compare by Segment_ID. There isn't an option to select that.

Does anybody have a good solution or ideas on how to achieve a full compare like I've described?

---

For some reason, ArcGIS isn't recognizing the join that I'm attempting per RyanDalton's answer.

But

Taking the advice of a join I have figured out how to find the added records and deleted records differences in the two files by:

1. Joining the Previous Month shapefile to the Current Month shapefile via the Segment_ID field
2. Performing a definition query where Segment_ID IS NULL
3. Exporting that to a new shapefile (Added Features)

I then reverse the joins and find the Deleted Features

Still working on a way to find the other differences listed and then string them all together in a model.

---

I can find all records that have changed by performing a select by location records from one of the datasets that are identical to the other dataset. I then switch the selection and it gives me all the records that are not identical.

I'll put it all in a model and hopefully it works just as well.

Answer 1

With the Compare Feature Tool, you should just choose your Segment_ID field as the sort field in the dialog.

The [sort] field or fields [are] used to sort records in the Input Base Table and the Input Test Table. The records are sorted in ascending order. Sorting by a common field in both the Input Base Features and the Input Test Features ensures that you are comparing the same row from each input dataset.

If you are wanting to get the Segment_ID field as part of your output table, try joining the output Compare Feature table to your source Test Table on cf.Object_ID=tt.Object_ID, then export out the joined table with your Segment_ID.

You could easily put this into a model for easier repeatability.

Answer 2

In my experiences Feature Compare always uses FID or ObjectID as the basis of the comparison even if segment_id is selected as Sort Field. Shapefile FIDs are recycled (reordered) so after a few edits many if not all FIDs are different for their corresponding geographies. They will thus have a "true" feature compare error. If you use an ObjectID from GeoDatabase or GISquirrel then the IDs are not recycled and you can rely on this tool. You have to keep the databases in tacked over the period of time so the ObjectIds are never reordered.

Regardless the Sort Field description in the GP tool help is VERY misleading. This tool would be tremendous if the basis of comparison could be the segment_ID.