Merging multiple shapefiles using ArcPy

Question

I have a list of paths of all the shapefiles (around 50) to be merged. All of them are at the same projection and none of them is sized more than 110MB (shp+dbf). The combined size of merge (shp+dbf) is around 1.5GB. The arcpy code used is

arcpy.Merge_management(list_of_paths, output)

Problem: Merging all these files somehow creates an empty file (after around 15 minutes) with an error message:

arcgisscripting.ExecuteError: ERROR 999998: Unexpected Error.
Failed to execute (Merge). 

Answer 1

You could try iterating the list and appending each shapefile to a target. It may be that one of the shapefiles is corrupt and this could help you isolate the problem child.

# assign first item as target
target = list_of_paths[0]
print("target - {}".format(target))

# iterate over remaining files, appending each one to target
for shp in list_of_paths[1:]:
    print("\tappending - {}".format(shp))
    arcpy.Append_management(shp, target)

Answer 2

Ahh - the dreaded 'unexpected' error. My first try would be to save this into a file GDB. Based on what you say you shouldn't exceed the shapefile size limits but maybe some temp file is. For other ideas see: ESRI 999999 error ideas