Take the 2-minute tour ×
Geographic Information Systems Stack Exchange is a question and answer site for cartographers, geographers and GIS professionals. It's 100% free, no registration required.

I'm trying to develop a tool using ArcGIS 10 that queries our company's data stores for any data that intersects a feature.

So that when we acquire a new property we can run this tool and it will find all of the data we have that intersects the new property, clip it, and save it in a new location.

share|improve this question
1  
Can you elaborate on your question at all? This is very vague-sounding. –  blah238 May 2 '12 at 6:46
    
First I want to search through Subdirectories. This is what I have so far, syntax error on if statement: import os, arcpy, sys basedir = sys.argv[1] #dirnames uses a recursive function to list subdirs: def dirsearch(basedir): [dirpath,dirnames,filenames]=os.walk(basedir) if dirnames.length == 0 intersectfeatures(dirpath) else: for dir in dirnames: dirsearch(dir) # a method that does the intersecting of all fcs in a directory def intersectfeatures(dirpath): arcpy.envworkspace=dirpath fclist=arcpy.ListFeatureClasses() for fc in fclist: –  Visceral May 2 '12 at 19:38
1  
Please edit your question to include your code sample and be sure to indent it by four spaces to get it to format correctly. Comments are not designed to contain anything more than short code expressions like this = "an example". –  blah238 May 2 '12 at 20:48

4 Answers 4

up vote 6 down vote accepted

A lot depends on how your data are stored. If your data store has grown organically and the data are jumbled in a load of directories and mix of shapefiles and geodatabases, you could use the Python os.walk function to list your directory structure and then subset your list to include only GDBs and SHPs. You would then need to recursively work through each dataset in the GDBs and each of the SHPs. This approach is a bit brute force but you could farm it out to a few CPU cores with a bit of threading and judicious use of the subprocess module.

If your data are neatly in a single database (whether that be a file-based geodatabase or SDE) your life should be a lot easier. You just then recursively walk through the database structure (using SQL and not the os.walk function obviously)

EDIT following Visceral's comments: Given that your directory tree includes mostly shapefiles you need to sift your files to only include the ones ending in '.shp'. Even if they ONLY contained shapefile, your are still only interested in the ones with the '.shp' extension. There's a few ways of doing this but here's an example:

def dirsearch(basedir):
    '''Gets all the shapefiles in the directory structure'''
    shpList = []
    for dirpath,dirnames,filenames in os.walk(basedir):
        shpList.extend([os.path.join(dirpath, fnm) for fnm in filenames if fnm.endswith('.shp')])
    return shpList
share|improve this answer
    
Thanks for the response Sylv. Unfortunately, the data is in a load of directories with mostly shapefiles. –  Visceral May 2 '12 at 19:30
    
Thanks The data is in a load of directories mostly with shapefiles. This is my code so far: import os, arcpy, sys basedir = sys.argv[1] #dirnames uses a recursive function to list subdirs: def dirsearch(basedir): [dirpath,dirnames,filenames]=os.walk(basedir) if dirnames.length == 0 intersectfeatures(dirpath) else: for dir in dirnames: dirsearch(dir) # a method that does the intersecting of all fcs in a directory def intersectfeatures(dirpath): arcpy.envworkspace=dirpath fclist=arcpy.ListFeatureClasses() for fc in fclist: –  Visceral May 2 '12 at 19:37
    
That is often the case :)! os.walk will be your friend here. I recommend you add a bit of code to sift out everything that does not end with '.shp' because your list will contain loads of extraneous file you don't want (not directly) like the '.shx', '.xml' and so on. –  MappaGnosis May 3 '12 at 8:48

Here's what I used to do a quick count of GIS data on a server. Should be easy to modify for the intersect.

import arcpy
from arcpy import env
import os
env.workspace = "G:/data"

fcList = []
count = 0
for dirname, dirnames, filenames in os.walk('G:/data'):
    for subdirname in dirnames:
        print os.path.join(dirname, subdirname)
        env.workspace = os.path.join(dirname, subdirname)
        fcList = arcpy.ListFeatureClasses()
        for fc in fcList:
            print fc
            count = count + 1
            print count
share|improve this answer

I am not much of a python scripter but a client sent us about three hundred shapefiles in about 100 folders and I didn't have the time to extract them separately.

I created the model below using model builder and it works just fine.

enter image description here

share|improve this answer

This is more for ArcGIS 10.1 Service Pack 1 to current versions. Arcpy has a method called Walk which allows you to look through directories and sub-directory. The walk method is useful as you can specify the data types that you are looking for. The below is the example from the documentation. The example shows the Walk function being used to catalog polygon feature classes.

import arcpy
import os
workspace = "c:/data"
feature_classes = []
for dirpath, dirnames, filenames in arcpy.da.Walk(workspace,
                                              datatype="FeatureClass",
                                              type="Polygon"):
    for filename in filenames:
    feature_classes.append(os.path.join(dirpath, filename))

You could modify the code to loop through the catalogue, add a check to see if the features intersect and if true, clip the data.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.