Identify corrupt shapefiles or rasters via Python or ArcObjects

Question

I'm interested in a way to identify rasters (e.g., ESRI GRID format) or shapefiles that are corrupt that would prevent them from being used in an analysis tool. I have a large number in structured folder tree and would like to be able to script something that will walk the file tree and be able to ID rasters that, for example if you tried to view it in ArcCatalog it would not allow it to be displayed and accordingly would make any geoprocessing tool fail. Same deal for shapefiles or feature classes.

My preference would be somehow to do this in Python (and I wouldn't be opposed to a non-ESRI python module or other) but I'm open for anything. I guess you could hack something together by trying to validate geometry or capitalizing on some aspect of the file design, but what would make the most sense? Or is there another way to do it that is specifically designed for such a purpose?

My goal is to just be able to run a simple Quality Control mechanism on some large week-long preprocessing files to make sure they're all sound before I start doing the actual batch processing.

thanks, Tom

Answer 1

For the rasters, why not just use a simple script to output the properties (cell size, extent etc.) of all the rasters you are going to use. If getting the properties fails then anything more advanced will also fail.

Plus you can write the raster properties to a log file and check that the extents, cell sizes, value types etc. are all what you expect.

import arcgisscripting, sys, string, os
gp = arcgisscripting.create()

try:
    # Set local variables
    InRaster = "D:/Data/elevation"
    InPropertyType = "MAXIMUM"

    # Process: GetRasterProperties
    zmax = gp.GetRasterProperties (InRaster, InPropertyType)
    # log these to a file or apply logic here to 
    # make sure values are in the expected ranges


except:
    # Print error message if an error occurs
    # likely to be an invalid raster
    print gp.GetMessages()

Also your mention of week long processing is somewhat worrying. It is very likely you will have to rerun the processes, so is there any way you can break it down into several smaller intermediate steps. This way you can check the results at each stage, and if something is incorrect you only need to resume from the previous step rather than the beginning.

Answer 2

For the vector data sources, use Check Geometry/Repair Geometry in tandem.

Answer 3

As far as vector shp files I would use something like the shape file checker on arcscripts.
shape file checker

For Raster The most common error during geoprocessing is invalid field type.
You might consider discovering the field types in your geoprocessing modules and then run the check on field types in your rasters.
I guess other types of erros or some other things to consider might be the pallette being indexed, or color depth being unsupported for particular processing.

Actually being corupt seems a bit more rare. But sure it can happen. Here might be some things to look for.
esri forums
corrupt dted
I think that most of the time these cases could be that using another software to open the file and save it back out to correct format/paramters fixes a lot of these issues.