Take the 2-minute tour ×
Geographic Information Systems Stack Exchange is a question and answer site for cartographers, geographers and GIS professionals. It's 100% free, no registration required.

My office will be seeing a big change in its GIS section. This section has been operational since the 1980's and has a huge collection of GIS data (i.e., shapefiles, raster files, data, etc) but never been through any inventory. Now it will happen.

Is there any automated way to extract all the information about the GIS data (i.e., shapefile, arc-info coverage, layer file, *.mxd, gdb, raster file, and more) from a PC to an Excel file? The information may include date of creation, date last edited, folder or container name, etc.

share|improve this question
2  
What version of ArcGIS are you at? At 10.1 SP1 this is made much easier by arcpy.da.walk. –  blah238 Jan 20 '13 at 10:17
1  
It never hurts to start out by taking a visual inventory and sketching out a design before you get into attacking an old server with python. –  Roy Jan 22 '13 at 19:47
    
In response to @Roy - you might consider starting with this FREE download: voyagergis.com –  Czed Jan 30 '13 at 14:27

5 Answers 5

This works for me, using the arcpy.da.Walk function at ArcGIS 10.1 SP1:

import arcpy, csv, os

workspace = r"c:\GISData"
output = r"C:\temp\test.csv"

with open(output, 'wb') as csvfile:
    csvwriter = csv.writer(csvfile)
    for dirpath, dirnames, filenames in arcpy.da.Walk(workspace):
        for filename in filenames:
            desc = arcpy.Describe(os.path.join(dirpath, filename))
            csvwriter.writerow([desc.catalogPath, desc.name, desc.dataType])

The csv module is also used to simplify writing the output file. Excel can open CSV files so you can view them as spreadsheets.

See the arcpy.Describe function for additional properties you can include in the output.

If you are specifically looking to parse out information from the actual metadata, see the script in this answer: Creating a table containing all filenames (and possibly metadata) in a File Geodatabase

share|improve this answer
    
@blah239, excel can open text files too, just need to provide the deliminator. –  artwork21 Jan 20 '13 at 15:21
4  
True, but the Excel CSV dialect takes care of all the tricky issues such as embedded quotes, newlines and commas. It also doesn't require going through a wizard to just open the file. –  blah238 Jan 20 '13 at 15:34
    
thx for the clarification. –  artwork21 Jan 20 '13 at 15:41

When you use Python, you must use the correct modules to do what you want. To find all files in a directory with extension shp, for example, there are much simpler solutions that was presented without the break, which is awful...(as the solution presented by Nathan W, but there are many, many others, just search on Internet)

Some examples with relevant modules:

1) with the glob module:

shapefiles only:

import glob
import os
os.chdir("mydir")
for files in glob.glob("*.shp"):
    print files

shapefiles and geodatabases:

import glob
types = ('*.shp', '*.gbd') # the tuple of file types
files_grabbed = []
for files in types:
     files_grabbed.extend(glob.glob(files)) #files_grabbed = the list of shp and gbd files

if you want to search also in the subdirectories:

import glob
for f in glob.iglob("/mydir/*/*.shp"): #search immediate subdirectories 
    print f

2) with os.listdir and list comprehension (in two lines) -> list of results

path = 'mydir'
shape_files = [f for f in os.listdir(path) if f.endswith('.shp')]
gdb_files = [f for f in os.listdir(path) if f.endswith('.gdb')]

3) with fnmatch module:

import fnmatch
for file in os.listdir('path'):
    if fnmatch.fnmatch(file, '*.shp'):
        print file

and many others solutions, recursive etc.

share|improve this answer
    
how do you ignore .shp.xml files using '*.shp' method? –  artwork21 Jan 20 '13 at 15:18
1  
Did you try it? glob.glob("*.shp") does not return .shp.xml files on my end. –  blah238 Jan 20 '13 at 15:37
    
@blah238, no did not try, thx. –  artwork21 Jan 20 '13 at 15:40

If you have ArcGIS Desktop 10.0 (or any of its service packs), I think your best bet is writing a python script that uses os.walk to look through a defined GIS directory and searches for common GIS file extensions such as .shp, .gdb, .mdb, etc... and writes the result to a comma delimited text file. You can then bring the text file into excel, see code example below:

import os, arcpy

#create blank text file
txt = open("C:\\Temp\\GISlayers.txt", "w")

for root, dirs, files in os.walk("C:\\Temp\\temp"):
    for f in files:

        #look for shapefiles
        foundSHP = f.find(".shp")
        if foundSHP >0:
            checkEXT = f[-3:]
                if checkEXT <> "xml":
                    desc = arcpy.Describe(root + "\\" + f)
                    #write info to text file
                    txt.write(desc.name + "," + desc.catalogPath + "\n")

        #look for file geodatabases
        foundGDB = f.find(".gdb")
        if foundGDB >0:
            desc = arcpy.Describe(root)
            for child in desc.children:
            #write info to text file
            txt.write(child.name + "," + child.path + "\n")
        break
txt.close()

If you are using ArcGIS 10.1 (or later) for Desktop then there is another Answer here that uses arcpy.da.Walk which was not available at 10.0 or earlier.

share|improve this answer
    
You might want to check you code. It will only search for gdb if it finds a shape first. Seems the indention is all messed. –  Nathan W Jan 20 '13 at 8:20
    
I also don't f.find is the correct usage here. This would be better written like this: gist.github.com/4577289 Untested of course. –  Nathan W Jan 20 '13 at 8:29
    
don't think f.find** –  Nathan W Jan 20 '13 at 9:41
    
Other simplifications could include using the csv module to abstract the file writing a bit, and using arcpy.da.walk at 10.1 SP1 to let ArcGIS handle listing the GIS data types. –  blah238 Jan 20 '13 at 10:07
    
Thanks! I am working on extracting as many info as I can from that age old database. –  blu_sr Jan 20 '13 at 10:35

Thanks artwork21 and Nathan W for your response. And yes Nathen's code made the magic.

import os, arcpy

#create blank text file
with open("C:\\Temp\\GISlayers.txt", "w") as txt:
for root, dirs, files in os.walk("C:\\Temp\\temp"):
    for f in files:
        #look for shapefiles
        if f.endswith('.shp'):
            desc = arcpy.Describe(root + "\\" + f)
            #write info to text file
            txt.write(desc.name + "," + desc.catalogPath + "\n")

        #look for file geodatabases
        if f.endswith('.gdb'):
            desc = arcpy.Describe(root)
            for child in desc.children:
                #write info to text file
                txt.write(child.name + "," + child.path + "\n")

        #look for layer files
        if f.endswith('.lyr'):
            desc = arcpy.Describe(root + "\\" + f)
            #write info to text file
            txt.write(desc.name + "," + desc.catalogPath + "\n")

        #look for img file
        if f.endswith('.img'):
            desc = arcpy.Describe(root + "\\" + f)
            #write info to text file
            txt.write(desc.name + "," + desc.catalogPath + "\n")
txt.close()

Only file name, and location. The pc i'll be working with has lots of coverage (the arc-info file) file, will it work on them too?

share|improve this answer
    
As for the version of my software, I use AG 10.1 SP1, but other pc use various versions of ESRI software- Arc info mostly. –  blu_sr Jan 20 '13 at 10:23
    
I'm actually not sure if arcpy.da.walk will list coverages, but I would guess not since it is not listed in the dataType or type filters. –  blah238 Jan 20 '13 at 10:32
    
Here is a shorter version of the code: gist.github.com/4577289. As the logic for shp, lyr, and img is the same we just just do them in one if statement. –  Nathan W Jan 20 '13 at 11:58
1  
You also don't need the txt.close() if you are using with because it will do that for you when the block exits. –  Nathan W Jan 20 '13 at 11:58

If you want to avoid programming, this might be the easiest and fastest method.

There is an add-on for Excel called ASAP Utilities. There is a 90-day free trial but after that, it's $49 USD for business use. It's free for student or personal use. The add-on adds a lot of useful functions. One of which is creating a list of files in a folder structure. It also provides file properties. You can limit the results by file type if you wish.

Here's a video of how to do this.

I've used this add-on before and the results are very quick.

Note, I am not affiliated with this software company.

share|improve this answer
1  
Thnx Fezter, but i don't think it'll fetch the GIS file types as before. The .shp is not just .shp, has many other files with it. –  blu_sr Jan 21 '13 at 3:57
    
It can get any and all file types in a folder. –  Fezter Jan 21 '13 at 4:26
1  
@Fetzer unless it knows how to read GIS datasets out of File and Personal Geodatabases, I'd be surprised if it would work here since there isn't a true correlation between each file and each dataset –  nicksan Jan 22 '13 at 20:55
    
Oh yeah, you're right. I missed that you had geodatabases. This wouldn't work for you. Sorry about that. But, it's a good plug in anyway. –  Fezter Jan 22 '13 at 22:40

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.