The dataset have shapefiles saved in different folders, as shown below. enter image description here

The numbers of shapefiles in each folder are not equal. The content of one folder is shown below. enter image description here

I need to merge all shapefiles in each folder and save the merge results in their corrsponding folder based on the FacIDxx, as shown below. enter image description here enter image description here

The following code works fine for the requirements mentioned above, but it takes lots of time when the dataset folder has many shapefiles to be merged.

import arcpy

#resetting the workspace
def find_all_fcs(workspace):
    prev_workspace = arcpy.env.workspace

    arcpy.env.workspace = workspace
    fcs = arcpy.ListFeatureClasses()

    arcpy.env.workspace = prev_workspace
    return fcs

arcpy.CreateFolder_management("C:/Users/TL/Desktop/812test", "Merge")
fileinIntersect = r"C:\Users\TL\Desktop\812test\Intersect"
fcs = find_all_fcs(filein0)
for fc in fcs:
    tempname_Intersect = fileinIntersect+'\\'+fc[:-4]
    fcs_Intersect = find_all_fcs(tempname_Intersect)
    arcpy.CreateFolder_management(r"C:\Users\TL\Desktop\812test\Merge", fc[:-4])
    input_Merge = []
    for fc_Intersect in fcs_Intersect:
        tempname_toMerge = tempname_Intersect+'\\'+fc_Intersect
    output_Merge = r"C:\Users\TL\Desktop\812test\Merge"+'\\'+fc[:-4]+'\\'+fc[:-4]+'.shp'
    arcpy.Merge_management(input_Merge, output_Merge)

I wonder if someone can help me improve the efficiency of the code.

  • 1
    What does your ArcGIS Runtime tag have to do with your question? Also, you seem to be asking for a code review when such are at best problematic and more likely specifically off-topic: meta.gis.stackexchange.com/questions/4304/… At minimum I think you should identify what the most time consuming step is in your code. Then you can perhaps ask more specifically about that.
    – PolyGeo
    Commented Aug 16, 2016 at 2:02
  • Thanks for your information. Actually, I don't know what is ArcGIS Runtime tag, but I will take a look at it. When I mentioned the runtime is long, I noticed that the merge step in my code for the folder with a lot of shapefiles (2500+) needed to be merged.
    – Tianxin Li
    Commented Aug 16, 2016 at 2:49
  • 1
    I think the time taken for the rest of your code will be insignificant in comparison to merging 100+ shapefiles. I think your question should focus on the performance of the Merge tool which would make it a duplicate of gis.stackexchange.com/questions/28244/…
    – PolyGeo
    Commented Aug 16, 2016 at 2:52

1 Answer 1


os.path is faster than arcpy for filesystem manipulation - use that to search your folders and set up the new folders. To speed up the merge, use append_management. Create the new folder, copy the first shapefile, then append the rest of the shapefiles to that one. Append will also allow you to keep master files that can be added to over time (if needed). Merge has to create a new feature class before merging the data. Append does not do this, so may be slightly faster. The processing time is going to depend on the size of the data.

  • Thanks for your information. Based on your description, I believe you totally understand my questions and I think you have done something similar before. I will take a look at the grammar of os.path and append_management. I will post further questions as needed for this question. Good luck with your study and work.
    – Tianxin Li
    Commented Aug 16, 2016 at 2:52
  • gis.stackexchange.com/questions/28244/… suggests that the jury may still be out on Merge/Append performance comparison. I would have guessed Append NO_TEST would be quickest but I cannot recall ever measuring.
    – PolyGeo
    Commented Aug 16, 2016 at 3:00
  • @Tianxin Li You're welcome. If you find the info useful, could you please upvote my answer. If you post more comments I will improve my answer until it solves the problem. Good luck to you as well.
    – jbalk
    Commented Aug 16, 2016 at 3:32
  • Absolutely. Thanks for your help. It took me about 10 hours to run the code I put on above. I haven't got a chance to improve the code based on your suggestions, but I will find a time to do that.
    – Tianxin Li
    Commented Aug 16, 2016 at 21:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.