2

The script I have put together should copy all shapefiles, dwg's, and feature classes (GDB) in an existing network directory to an existing geodatabase with a feature dataset regardless of the file. The script only needs to throw an exception or tell me that the file exists because it only has a copy function (arcpy.copy_management(indata,outdata)).

Here's the code:

import arcpy, os, shutil, time

arcpy.env.workspace = os.path.normpath('//svcfs01/datavolume2/Standards/Water Resources/GIS/_Water Resources GIS Database/New Folder')
targetGDB = os.path.normpath('//svcfs01/datavolume2/Standards/Water Resources/GIS/_Water Resources GIS Database/_Unprocessed_Raw_Data.gdb/WRSpatial/')
featureClasses = [arcpy.Describe(a).name for a in arcpy.ListFeatureClasses()]
feature_classes = []
targetPath = []
for dirpath, dirnames, filenames in arcpy.da.Walk(os.path.normpath('//svcfs01/datavolume2/Standards/Water Resources/GIS/_Water Resources GIS Database/New Folder'),datatype="Any",type="Any"):
    for filename in filenames:
        feature_classes.append(os.path.join(dirpath,filename))
        file, filext = os.path.splitext(filename)
        targetPath.append(os.path.join(targetGDB,file))
        if arcpy.Exists(targetPath)==False:         
            try:    
                arcpy.Copy_management(feature_classes, targetPath)      
            except Exception as e:  
                print "Unable to copy %s to %s" %(feature_classes, targetPath)      
        else:   
            print "%s already exists....skipping....." %(feature_classes)

The error I get is:

Unable to copy [u'\\\\svcfs01\\datavolume2\\Standards\\Water Resources\\GIS\\_Water Resources GIS Database\\New Folder\\Drainage_Area.shp'] to [u'\\\\svcfs01\\datavolume2\\Standards\\Water Resources\\GIS\\_Water Resources GIS Database\\_Unprocessed_Raw_Data.gdb\\WRSpatial\\Drainage_Area']
Unable to copy [u'\\\\svcfs01\\datavolume2\\Standards\\Water Resources\\GIS\\_Water Resources GIS Database\\New Folder\\Drainage_Area.shp', u'\\\\svcfs01\\datavolume2\\Standards\\Water Resources\\GIS\\_Water Resources GIS Database\\New Folder\\TC.shp'] to [u'\\\\svcfs01\\datavolume2\\Standards\\Water Resources\\GIS\\_Water Resources GIS Database\\_Unprocessed_Raw_Data.gdb\\WRSpatial\\Drainage_Area', u'\\\\svcfs01\\datavolume2\\Standards\\Water Resources\\GIS\\_Water Resources GIS Database\\_Unprocessed_Raw_Data.gdb\\WRSpatial\\TC']

print e
Object: Error in executing tool
  • u'//svcfs01/datavolume2/Standards/Water Resources/GIS/_Water Resources GIS Database/New Folder\\Drainage_Area.shp' << look at the \\ before the shapefile, they are going the incorrect way and not matching the other path formatting. I think if you fix that path you will be good. – ed.hank Nov 17 '17 at 17:31
  • 3
    You appear to want to have a shapefile within a file geodatabase (.../_Unprocessed_Raw_Data.gdb/WRSpatial/Drainage_Area.shp) This is seriously wrong and will not work. At a minimum, you must remove the ".shp" before concatenating it to the FGDB path. – Vince Nov 17 '17 at 17:55
  • @ed.hank, Okay, I've got the paths correct by adding this: arcpy.env.workspace = os.path.normpath('//svcfs01/datavolume2/Standards/Water Resources/GIS/_Water Resources GIS Database/New Folder') @Vince I've tried this bu it still leaves the extension – Kevin G Nov 17 '17 at 18:27
  • @Vince I've tried this but it still leaves the extension for dirpath, dirnames, basenames in arcpy.da.Walk(os.path.normpath('//svcfs01/datavolume2/Standards/Water Resources/GIS/_Water Resources GIS Database/New Folder'),datatype="Any",type="Any"): for basename in basenames: feature_classes.append(os.path.join(dirpath, basename)) – Kevin G Nov 17 '17 at 18:33
  • 1
    Comments is not the proper place for legible code. Instead, Edit the question. At this point, comment out the entire if block and just use a print statement to generate something that looks correct (your strings should not be one-element arrays). Using spaces in file paths is not best practice, nor is leading underscores. – Vince Nov 17 '17 at 22:15
1

Okay, here is the working code. The Copy_management wasn't the function to use. There is another post that may explain this reason, and the need for array's when using Copy_management. I've used that code before, but my knowledge is limited in the differences and why it works for that case and not this one: Copying ArcSDE geodatabase to file geodatabase using ArcPy?

The syntax of the unix path versus the windows path is important; especially when using network or local drives that have spaces or come from many different directories (see my powershell code on how this data is collected: https://stackoverflow.com/questions/45717027/copy-item-and-rename-to-a-specific-parent-directory-in-powershell)

So, here I use the arcpy.CopyFeatures_management function and I am able to copy any data that can be read in the arc environment from one location to another whether it be folder to folder, folder to GDB feature dataset etc. Again, not sure why this doesn't need an array, but the other script does.

import arcpy, os, shutil, time  
arcpy.env.workspace = os.path.normpath('//svcfs01/datavolume2/Standards/Water Resources/GIS/_Water Resources GIS Database/New Folder')
targetGDB = os.path.normpath('//svcfs01/datavolume2/Standards/Water Resources/GIS/_Water Resources GIS Database/_Unprocessed_Raw_Data.gdb/WRSpatial/')
for dirpath, dirnames, filenames in arcpy.da.Walk(os.path.normpath('//svcfs01/datavolume2/Standards/Water Resources/GIS/_Water Resources GIS Database/New Folder'),datatype="Any",type="Any"):
    for filename in filenames:
        feature_classes = os.path.join(dirpath,filename)
        file, filext = os.path.splitext(filename)
        targetPath = os.path.join(targetGDB,file)
        if arcpy.Exists(targetPath)==False:         
            try:    
                arcpy.CopyFeatures_management(feature_classes, targetPath)      
            except Exception as e:  
                print "Unable to copy %s to %s" %(feature_classes, targetPath)      
        else:   
            print "%s already exists....skipping....." %(feature_classes)

Note that data such as dwg's always spit out the same name under CopyFeature_management. I'm sure there's an innumerable amount of ways to do this, but I circumvented the issue with the following code to be added just after getting the file without the extension:

        file, filext = os.path.splitext(filename)
        ROOT_DIR = os.path.dirname(os.path.abspath(dirpath))
        CONFIG_PATH = os.path.join(ROOT_DIR,'config.conf')
        project = os.path.basename(os.path.dirname(CONFIG_PATH))
        copyName = ''.join(e for e in (project + file) if e.isalnum())
        targetPath = os.path.join(targetGDB,copyName)

The config.conf is a file that can be made as a text file with the extension changed. Then, it can be placed anywhere in the directory of source data that is being copied(or in whatever directory you find meaningful, keeping in mind the ROOT_DIR) to get the directory name of its location.

The thought behind this is that the organizational structure of the directory tree above the data could be used to more accurately describe what is being copied. At the same time, files named the same (but with different parents) will inherit a unique name that will avoid confusion if they are known to be different.

Lastly, and also optional, the ''.join(e for e in (project + file) if e.isalnum()) line leaves only letters and numbers without spaces since GDB's don't like these characters.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.