I have a script written in Python that uses ArcPy to create maps of the kernel densities of thousands of datasets. Each dataset is relatively small, but there about 50,000 sets in total.
When I run my script, it stops after about 10-15 minutes and gives the "python.exe has stopped working" popup. The Windows 7 event viewer tells me it is an Exception Code 0x0000005, which appears to be an access violation. The datasets the code works on are not the same each time, so it is not a specific set that is causing the problem.
Watching the task manager when the script is running, I noticed that python.exe is increasing its memory usage every few seconds, which makes me believe there is a memory leak somewhere. I thought Python had garbage collection? Is there something that arcPy is doing?
EDIT: So I tracked down the part of my code causing the problem, but I don't know HOW it is leaking. I think I am cleaning everything up. Does anyone spot something I missed?
def makeMap(asno, isKdens): try: mxd = arcpy.mapping.MapDocument("F:\\testFiles\\testMap.mxd") ###Reference to basemap document df = arcpy.mapping.ListDataFrames(mxd, "Layers") ###Get data frame reference in var ##++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ if isKdens: ###Set up layer file lyrFile = arcpy.mapping.Layer(r"F:\\testFiles\\output\\AS_"+asno+"_KDens.lyr") ##++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ else: ###Set up layer file lyrFile = arcpy.mapping.Layer(r"F:\\testFiles\\output\\AS_"+asno+".lyr") lyrFile.visible = True ###Add layer file arcpy.mapping.AddLayer(df, lyrFile) ###ref added layer lyr = arcpy.mapping.ListLayers(mxd) lyr.visible = True #If this is a raw map, just select points. if isKdens==0: arcpy.SelectLayerByAttribute_management ("pointslyr", "NEW_SELECTION", "\"asno\" = "+asno+"") ##++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ #If KDens map, apply symbology else: arcpy.ApplySymbologyFromLayer_management(lyr, templateLayer) ##++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ###Zoom to selection df.zoomToSelectedFeatures() lyrExtent = lyr.getSelectedExtent() df.extent = lyrExtent ###Export to PDF!!! ##++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ if isKdens: arcpy.mapping.ExportToPDF(mxd, r"F:\\testFiles\\output\\AS_"+asno+"_KDens_Map.pdf") ##++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ else: arcpy.mapping.ExportToPDF(mxd, r"F:\\testFiles\\output\\AS_"+asno+"_Map.pdf") except: print arcpy.GetMessages() #####DELETION del mxd, df, lyrFile, lyrExtent, lyr print "DELETED MXD AND DFS"
So, I also tried turning off logging, since I have heard that can cause memory leaks, too. There is still something leaking. I also tried using
Delete_management instead of
del, but that just messes things up. It almost seems like my code is not running sequentially. A delete after a reference still gives a "references before defined" error...
Anyway, now I am trying to locate the memory leak using memory analyzer packages. Out of all of them that work with Python, I only got
objgraph to work. I placed a
objgraph.show_growth call before and after the function whose code is above. Here are the results:
tuple 11959 +11959 function 3643 +3643 dict 1632 +1632 wrapper_descriptor 1086 +1086 cell 927 +927 list 635 +635 builtin_function_or_method 590 +590 property 551 +551 method_descriptor 390 +390 weakref 373 +373
weakref 398 +25 wrapper_descriptor 1105 +19 tuple 11977 +18 dict 1644 +12 member_descriptor 172 +4 getset_descriptor 213 +3 method_descriptor 392 +2 instance 15 +2 list 636 +1 `_RLock 2 +1
It doesn't look like any of my objects are leaking, though I am not quite sure what
instance means in the second growth dump. Anyone have any insights?