Often I find myself in a situation where I don't care whether or not my tool produces a new feature class, but I do care how long it takes to combine all of my large datasets. Does it take longer to produce a new feature class using the Merge tool instead of the Append tool, or are Merge and Append essentially the same in terms of performance?
Take this answers based on the tools themselves and not an actual benchmark:
The merge tool creates a new feature class, which takes time in itself, before it crams together the two datasets.
The append tool with the TEST option assumes that both datasets have the same fields (field names) and crams them together without having to create a new feature class (sounds faster).
The append tool with the NO TEST option allows for field mapping to combine like feature classes that may have different field names. This requires some behind the scenes conditional testing, which would take more time.
As the size of the dataset grows, the amount of time it takes to create a new fc seems insignificant. The only way to know for sure would be to do some benchmarks with your large datasets and post the answers here!
I suspect the difference isn't much it's more about what you want out of the tool in the end (field mapping vs. new feature class vs. no new feature class)
Merge takes both geometry and attributes and combines (merges) the entire dataset into a new feature dataset.
Append is good way to join extra data to an existing dataset - it can have options to control subtypes of features being appended.
The key difference is
If the Schema Type TEST is specified, the schema (field definitions) of the input datasets must match that of the target dataset in order for the features to be appended. If the Schema Type NO_TEST is specified, input dataset schema (field definitions) do not have to match the target dataset. However, any fields from the input datasets that do not match the fields of the target dataset will not be mapped to the target dataset unless the mapping is explicitly set in the Field Map control.
subtype (Optional) A subtype description to assign that subtype to all new data that is appended to the target dataset.
You might be interested in the 'Tiled processing of large datasets'
I just ran all three variants. I always combined the same 63 shapefiles. Each shapefile contains around 63000 points created from a RasterToPoint operation. All shapefiles have the same attribute table.
The different variants needed the following times to compile:
- Merge: 13 mins 57 secs
- Append with TEST: 8 mins 34 secs
- Append with NO_TEST: 9 mins 12 secs
Seems like Append with TEST as an input parameter is the fastest one. Obviously the choice of which Append to use depends not on speed but rather on your input files as explained above.
Hope it is useful.