Take the 2-minute tour ×
Geographic Information Systems Stack Exchange is a question and answer site for cartographers, geographers and GIS professionals. It's 100% free, no registration required.

I put together a script to take a TIGER shapefile of all census tracts in a state, select the census tracts by each county, and export them into individual county-level kml files.

The problem is that the script only sometimes runs to completion, but more often, crashes python.

The crash happens in the for county in countyList: loop, though not at the same spot each time, and does not seem to be dependent on the size of the exported kml file. I don't get any errors, so I'm not sure of the line on which the script crashes. A Windows message pops up and says that "pythonw.exe" has stopped working (I've been running the script from the python win in IDLE).

EDIT: I have tried running the script without a Try/Except block to see if any python errors occur. However, the same issue is still present. That is, the script stops without any type of Python error. Windows simply alerts that the program (pythonw.exe or python.exe) has stopped working. This is the case when running the script in both IDLE and through Powershell.

I searched for possible problems and came up with several possible solutions based on other examples, suggestions, etc., none of which have seemed to make any difference. I attempted deleting some of the intermediate items(i.e. the search cursor variables), using gc.collect() to collect garbage at certain points throughout the script, and setting a timer to wait at certain intervals (the gc and timer are commented out here since they don't seem to make much difference.

I also updated to ArcGIS service pack 1 with 64 bit python processing, which doesn't seem to make a difference either. I'm running a computer with 4gb of memory, but the usage stays well below 2gb during the process, so I'm not sure if it's a memory issue.

Is there something else in the script that I'm not seeing that might be causing Python to crash?

Here's the code:

# Batch Shapefile to kml in Arcmap

'''
This script takes a shapefile, splits the file by its sub-parts, then exports each sub-part as a kml file. 
'''

print "importing modules..."
import sys, traceback, arcpy, os, gc, time

try:

    # Setting Defaults
    workspace = r"C:\Illinois"
    stateFIPS = "17"

    # Setting the workspace
    arcpy.env.workspace = workspace

    # Get a list of the feature classes in the directory
    # ListFeatureClasses ({wild_card}, {feature_type}, {feature_dataset})
    fcList = arcpy.ListFeatureClasses()
    print("Feature classes found: " + str(fcList))

    for FC in fcList:
        # Make a feature layer of the feature class
        theOutFL = "temp_FL"
        theFeatureLayer = arcpy.MakeFeatureLayer_management(FC, theOutFL)

        print "starting search cursor"

        valuesList = []
        srows = arcpy.SearchCursor(FC)
        counter = 1
        for srow in srows:
##            if counter >= 11 and counter <= 20:
            if srow.COUNTYFP10 not in valuesList:
                valuesList.append(srow.COUNTYFP10)
##                    counter += 1
        valuesList.sort()
        for value in valuesList:
            print value
        del srow
        del srows
        print "Collecting garbage..."
        gc.collect()

        countyList = valuesList
  ##      theCount = 1
        print "\nCreating kml files for counties in 'countyList' ..."
        for county in countyList:
            theQuery = "\"COUNTYFP10\" = '" + str(county) + "'"
            # Select rows if the county matches the current query    
            # SelectLayerByAttribute_management (in_layer_or_view, {selection_type}, {where_clause})
            arcpy.SelectLayerByAttribute_management(theFeatureLayer, "NEW_SELECTION", theQuery)

            # Specify a name for the out kmz file
            theOutPath = r"C:\temp"
            # theOutFile = the state fips + the county fips .kmz
##            theOutFile = "kml" + str(theCount) + ".kmz"
            theOutFile = stateFIPS + county + ".kmz"
##            theCount = theCount + 1
            outKMZ = theOutPath + os.sep + theOutFile
            # Export the selected portions to a KML
            # Check if the file already exists. if it does, delete it
            if arcpy.Exists(outKMZ):
                print "the file" + theOutFile +  " already exists. Deleting the file..."
                arcpy.Delete_management(outKMZ)
            # LayerToKML_conversion (layer, out_kmz_file, layer_output_scale, {is_composite}, {boundary_box_extent}, {image_size}, {dpi_of_client})
            print "Exporting selections to new KML file..."
            arcpy.LayerToKML_conversion(theFeatureLayer, outKMZ, 500000)
            # Clear the selection before the next iteration
            print "Finished " + theOutFile + "\n"
##            print "Clearing selected features...\n"
##            arcpy.SelectLayerByAttribute_management(theFeatureLayer, "CLEAR_SELECTION")
##            theCount = theCount + 1
##            if theCount %10 == 0:
##                print "Collecting garbage..."
##                time.sleep(1)
##                gc.collect()


        print "Exports complete. Files can be found in the directory: " + theOutPath

except:

    tb = sys.exc_info()[2]
    tbinfo = traceback.format_tb(tb)[0]
    # LAST sys FUNCTIONS DEPRECTATED AFTER 1.5 --> pymsg = "PYTHON ERRORS:\nTraceback Info:\n" + tbinfo + "\nError Info:\n     " +        str(sys.exc_type) + ": " + str(sys.exc_value) + "\n"
    pymsg = "PYTHON ERRORS:\nTraceback Info:\n" + tbinfo + "\nError Info:\n     " + str(sys.exc_info()[1])

    msgs = "ARCPY ERRORS:\n" + arcpy.GetMessages(2) + "\n"

    arcpy.AddError(msgs)
    arcpy.AddError(pymsg)

    print msgs
    print pymsg

    arcpy.AddMessage(arcpy.GetMessages(1))
    print arcpy.GetMessages(1)
share|improve this question
1  
Is there a particular place in the process where your script fails consistently? –  Jay Guarneri Mar 28 '13 at 13:09
2  
Is there a reason for the ; in theCount = 1;? –  Jason Mar 28 '13 at 14:18
    
So no errors are being thrown? I see you have a try except, so just curious if you are seeing any errors. Do you know how far your script is getting? Can you see any of your print statements? –  Jamie Mar 28 '13 at 14:49
1  
I believe that LayerToKML requires a feature layer, rather than a feature class as the first argument. Layer To KML help –  Krausers Mar 28 '13 at 16:29
2  
Have you solved this Krausers? I have a few suggestions, see on what layer it is failing on and run Repair Geometry / Check G on it - often when I iterate this is the issue! Anything else? –  Tom Apr 23 '13 at 3:44

1 Answer 1

I actually had this exact same issue. I had a script that would search through sub folders in a root directory, check if they had shapefiles, convert the shapefiles to feature layers then to kmls. It would run through about 20-40 conversions and then just stop, no error codes, just the script would end. It would end at different places each time. My solution was to use a co-workers computer with a better memory and processor, and it went through all 600 folders and conversions without a snag. The code didn't change at all between attempts, just which computer I was using. Both computers I was on had ArcGIS 10.1 SP1 and were running windows 7

share|improve this answer
1  
Do you have any details about the memory differences of the two machines? It does sound like this could be a memory issue. Between revisiting my code and better understanding where the performance hang-up is occurring, we may be able to come up with an official answer to the question. –  Krausers Apr 21 at 14:25
1  
I am glad to hear that I am not the only one that has had this problem. I had a arcpy script that tried to process hundreds of shapefiles, but similarly it would make it through about 60 files and then stop. I was getting an error code that the next file in the for loop that the code was supposed to process didn't exist. I though it was related to Arc writing files to the computer's memory but I couldn't figure out how to delete these temporary files or the Arc memory cache. I had this problem using Arc 10.0 I thought that the computer I was using was plenty powerful. –  F_Kellner Apr 21 at 17:31

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.