Why might Python Script be Crashing Inconsistently on ArcPy Batch LayerToKML script?

Question

I put together a script to take a TIGER shapefile of all census tracts in a state, select the census tracts by each county, and export them into individual county-level kml files.

The problem is that the script only sometimes runs to completion, but more often, crashes python.

The crash happens in the for county in countyList: loop, though not at the same spot each time, and does not seem to be dependent on the size of the exported kml file. I don't get any errors, so I'm not sure of the line on which the script crashes. A Windows message pops up and says that "pythonw.exe" has stopped working (I've been running the script from the python win in IDLE).

EDIT: I have tried running the script without a Try/Except block to see if any python errors occur. However, the same issue is still present. That is, the script stops without any type of Python error. Windows simply alerts that the program (pythonw.exe or python.exe) has stopped working. This is the case when running the script in both IDLE and through Powershell.

I searched for possible problems and came up with several possible solutions based on other examples, suggestions, etc., none of which have seemed to make any difference. I attempted deleting some of the intermediate items(i.e. the search cursor variables), using gc.collect() to collect garbage at certain points throughout the script, and setting a timer to wait at certain intervals (the gc and timer are commented out here since they don't seem to make much difference.

I also updated to ArcGIS service pack 1 with 64 bit python processing, which doesn't seem to make a difference either. I'm running a computer with 4gb of memory, but the usage stays well below 2gb during the process, so I'm not sure if it's a memory issue.

Is there something else in the script that I'm not seeing that might be causing Python to crash?

Here's the code:

# Batch Shapefile to kml in Arcmap

'''
This script takes a shapefile, splits the file by its sub-parts, then exports each sub-part as a kml file. 
'''

print "importing modules..."
import sys, traceback, arcpy, os, gc, time

try:

    # Setting Defaults
    workspace = r"C:\Illinois"
    stateFIPS = "17"

    # Setting the workspace
    arcpy.env.workspace = workspace

    # Get a list of the feature classes in the directory
    # ListFeatureClasses ({wild_card}, {feature_type}, {feature_dataset})
    fcList = arcpy.ListFeatureClasses()
    print("Feature classes found: " + str(fcList))

    for FC in fcList:
        # Make a feature layer of the feature class
        theOutFL = "temp_FL"
        theFeatureLayer = arcpy.MakeFeatureLayer_management(FC, theOutFL)

        print "starting search cursor"

        valuesList = []
        srows = arcpy.SearchCursor(FC)
        counter = 1
        for srow in srows:
##            if counter >= 11 and counter <= 20:
            if srow.COUNTYFP10 not in valuesList:
                valuesList.append(srow.COUNTYFP10)
##                    counter += 1
        valuesList.sort()
        for value in valuesList:
            print value
        del srow
        del srows
        print "Collecting garbage..."
        gc.collect()

        countyList = valuesList
  ##      theCount = 1
        print "\nCreating kml files for counties in 'countyList' ..."
        for county in countyList:
            theQuery = "\"COUNTYFP10\" = '" + str(county) + "'"
            # Select rows if the county matches the current query    
            # SelectLayerByAttribute_management (in_layer_or_view, {selection_type}, {where_clause})
            arcpy.SelectLayerByAttribute_management(theFeatureLayer, "NEW_SELECTION", theQuery)

            # Specify a name for the out kmz file
            theOutPath = r"C:\temp"
            # theOutFile = the state fips + the county fips .kmz
##            theOutFile = "kml" + str(theCount) + ".kmz"
            theOutFile = stateFIPS + county + ".kmz"
##            theCount = theCount + 1
            outKMZ = theOutPath + os.sep + theOutFile
            # Export the selected portions to a KML
            # Check if the file already exists. if it does, delete it
            if arcpy.Exists(outKMZ):
                print "the file" + theOutFile +  " already exists. Deleting the file..."
                arcpy.Delete_management(outKMZ)
            # LayerToKML_conversion (layer, out_kmz_file, layer_output_scale, {is_composite}, {boundary_box_extent}, {image_size}, {dpi_of_client})
            print "Exporting selections to new KML file..."
            arcpy.LayerToKML_conversion(theFeatureLayer, outKMZ, 500000)
            # Clear the selection before the next iteration
            print "Finished " + theOutFile + "\n"
##            print "Clearing selected features...\n"
##            arcpy.SelectLayerByAttribute_management(theFeatureLayer, "CLEAR_SELECTION")
##            theCount = theCount + 1
##            if theCount %10 == 0:
##                print "Collecting garbage..."
##                time.sleep(1)
##                gc.collect()


        print "Exports complete. Files can be found in the directory: " + theOutPath

except:

    tb = sys.exc_info()[2]
    tbinfo = traceback.format_tb(tb)[0]
    # LAST sys FUNCTIONS DEPRECTATED AFTER 1.5 --> pymsg = "PYTHON ERRORS:\nTraceback Info:\n" + tbinfo + "\nError Info:\n     " +        str(sys.exc_type) + ": " + str(sys.exc_value) + "\n"
    pymsg = "PYTHON ERRORS:\nTraceback Info:\n" + tbinfo + "\nError Info:\n     " + str(sys.exc_info()[1])

    msgs = "ARCPY ERRORS:\n" + arcpy.GetMessages(2) + "\n"

    arcpy.AddError(msgs)
    arcpy.AddError(pymsg)

    print msgs
    print pymsg

    arcpy.AddMessage(arcpy.GetMessages(1))
    print arcpy.GetMessages(1)

Answer 1

I actually had this exact same issue. I had a script that would search through sub folders in a root directory, check if they had shapefiles, convert the shapefiles to feature layers then to kmls. It would run through about 20-40 conversions and then just stop, no error codes, just the script would end. It would end at different places each time. My solution was to use a co-workers computer with a better memory and processor, and it went through all 600 folders and conversions without a snag. The code didn't change at all between attempts, just which computer I was using. Both computers I was on had ArcGIS 10.1 SP1 and were running windows 7