Iterators in model builder - merging nested files with same filename ending

Question

I am trying to merge files that are nested in different folders by their file names. For example folder 1 contains up to 100 subfolders that each contain a set of shapefiles: subfolder1_A.shp, subfolder1_B.shp, subfolder1_C.shp, etc.

I want to merge shapefiles from each subfolder with the same file name ending (e.g. merge subfolder1_A, subfolder2_A, subfolder3_A, ..., subfolder100_A), so the final output is a series of merged shapefiles with the same file name ending (A, B, C, etc.). So for example, A_merged.shp, B_merged.shp, C_merged.shp, etc.

I am not familiar with python, and would prefer to do this using model builder. I have tried using iterate feature class and merge tools, but not sure how to make this work like I want. Possibly using inline substitution or the iterator wildcard somehow?

Any idea what I should do?

Answer 1

Do an OS walk to iterate all folders and sub folders. For each folder list all the feature class. Split the name of the feature class at your underscore and compose a list, then process that list so the names are unique. Use those unique names as a wild card while relisting all the feature class in the same folder. Lastly run it through the merge tool. I wired the below py into a toolbox script tool and it works OK for a single like geometry.

import arcpy
import os
import math
import sys

FolderOrWorkspace = arcpy.GetParameterAsText(0)
arcpy.AddMessage("======= START")

for dirname, dirnames, filenames in os.walk(FolderOrWorkspace): #Iterate through folder
    for subdirname in dirnames:
        arcpy.env.workspace = os.path.join(dirname, subdirname) # Workspace set to folder
        UList =[]
        arcpy.AddMessage(" ")
        arcpy.AddMessage("      - Folder: "+ subdirname)
        FeatureClassList = arcpy.ListFeatureClasses()
        arcpy.AddMessage("      - FEATURE CLASS LIST: ")
        arcpy.AddMessage(" ")
        arcpy.AddMessage(FeatureClassList)
        arcpy.AddMessage(" ")
        for ShortName in FeatureClassList: # Make a list of all the feature names after the the first "_"
            LastName = ShortName.split("_")[1]
            UList.append(LastName)
        arcpy.AddMessage("      - Short Name List: "+ str(UList))
        UniqueShortName = set(UList) # Process the list to be unique short names only
        arcpy.AddMessage("      - Unique Short Name List: "+ str(UniqueShortName))

        for mergetest in UList: # create a list of feature class ending in the short name
            NameWildCard = ("*_"+mergetest)
            MergeFeatureClassList = arcpy.ListFeatureClasses(NameWildCard,'',FeatureClassList)
            MFCLCount = len(MergeFeatureClassList)
            arcpy.AddMessage("      - Unique Short Name Count: "+ str(MFCLCount))

            if MFCLCount <= 1:
                arcpy.AddMessage("      - Only 1 Feature Class: Not Merged")
            else:
                MergeName = ("Merged"+mergetest) # the name of the merge shapefile

                try:
                    arcpy.AddMessage("      - Merging: " + str(MergeFeatureClassList))
                    arcpy.AddMessage(" ")
                    arcpy.Merge_management (MergeFeatureClassList, MergeName,"")    # Saves it in the same folder                    # NB Merge is an Advanced tool
                except Exception as e:
                    arcpy.AddError("       "+e.message)

arcpy.AddMessage("======= END")

You might want to add a geom type check..

Screen Grab: Shapefiles

Script Tools: Adding Script Tools