12

Often I find myself in a situation where I don't care whether or not my tool produces a new feature class, but I do care how long it takes to combine all of my large datasets. Does it take longer to produce a new feature class using the Merge tool instead of the Append tool, or are Merge and Append essentially the same in terms of performance?

  • 2
    Some Anecdotal Evidence: I have always found Merge to be faster, especially when dealing with large amounts of features. – Devdatta Tengshe Jun 26 '12 at 12:45
  • but Append works best when you have domains and subtypes predefined. – Mapperz Jun 26 '12 at 13:26
6

Merge takes both geometry and attributes and combines (merges) the entire dataset into a new feature dataset. enter image description here

Append is good way to join extra data to an existing dataset - it can have options to control subtypes of features being appended.

http://resources.arcgis.com/en/help/main/10.1/index.html#//001700000055000000

The key difference is

If the Schema Type TEST is specified, the schema (field definitions) of the input datasets must match that of the target dataset in order for the features to be appended. If the Schema Type NO_TEST is specified, input dataset schema (field definitions) do not have to match the target dataset. However, any fields from the input datasets that do not match the fields of the target dataset will not be mapped to the target dataset unless the mapping is explicitly set in the Field Map control.

subtype (Optional) A subtype description to assign that subtype to all new data that is appended to the target dataset.

http://resources.arcgis.com/en/help/main/10.1/index.html#//001700000050000000

enter image description here

You might be interested in the 'Tiled processing of large datasets'

http://resources.arcgis.com/en/help/main/10.1/index.html#//01m10000000r000000

  • 1
    I am aware of the differences between the tools, but I am wondering whether these differences lead to a significant difference in processing time. I typically use NO_TEST when using the append tool since I make sure that my field definitions match before appending/merging. – MTerry Jun 26 '12 at 15:38
5

Take this answers based on the tools themselves and not an actual benchmark:

The merge tool creates a new feature class, which takes time in itself, before it crams together the two datasets.

The append tool with the TEST option assumes that both datasets have the same fields (field names) and crams them together without having to create a new feature class (sounds faster).

The append tool with the NO TEST option allows for field mapping to combine like feature classes that may have different field names. This requires some behind the scenes conditional testing, which would take more time.

As the size of the dataset grows, the amount of time it takes to create a new fc seems insignificant. The only way to know for sure would be to do some benchmarks with your large datasets and post the answers here!

I suspect the difference isn't much it's more about what you want out of the tool in the end (field mapping vs. new feature class vs. no new feature class)

0

I just ran all three variants. I always combined the same 63 shapefiles. Each shapefile contains around 63000 points created from a RasterToPoint operation. All shapefiles have the same attribute table.

The different variants needed the following times to compile:

  • Merge: 13 mins 57 secs
  • Append with TEST: 8 mins 34 secs
  • Append with NO_TEST: 9 mins 12 secs

Seems like Append with TEST as an input parameter is the fastest one. Obviously the choice of which Append to use depends not on speed but rather on your input files as explained above.

Hope it is useful.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.