2

I have written an ArcPy script to process a DTM into a point feature class to be used in Landform classification and ranking. It successfully produces what I need most of the time.

Randomly the buffer polygon step produces an empty feature class. If I run the same process "by hand" it works. If I run the script again, or from a value just before it failed, it also works. The issue is not consistent, it may do it on the 3rd loop, or the 300th loop.

I've attempted to mitigate this issue by checking my delete variable statements (memory leak management), and by writing a get count followed by an "if" statement that runs a "compact", "repair geometry", "add spatial index", "Delete the last buffer feature class", and "re buffer". But to no avail. The first part of the script is below.. Any ideas would be welcome..

import arcpy
import winsound

######## Check out any necessary licenses
arcpy.CheckOutExtension("spatial")
from arcpy.sa import *

######## Script arguments
FolderLocation = "C:\\Data\\Basin\\"
InRaster = "C:\\Data\\DEM\\Basin1"
GDB = "C:\\Data\\Basin1.gdb"

LandForm = GDB+"\\LandForm"
TempP = GDB+"\\TempP"

TopVal = 1037
ProcessingVal = 772
BotVal = 150
Range = TopVal-BotVal
Range2 = ProcessingVal - BotVal
Range3 = Range2
print "There are: "+ str(Range) +" values in this raster. With: " +str(Range2) + " to process"
print " "

#### Naming Conventions
SingleValue = "s_"
BandValue = "b_"
CountValue = "r_"
LandFormIValue = "u_"
PNeigbours = "p_"
Frequency = "f_"
BufferName = "buf_"
MultiName ="m_"
IntName = "i_"
DisName = "d_"

MegaBuffCount = 0
MegaMultiCount = 0
MegaCount = ProcessingVal +1

while (MegaCount) > (BotVal):
    MegaCount = (MegaCount-1)
    ## Reclass Single Values
    SingleValueName = SingleValue + str(MegaCount) +".tif"
    SingleValueNamePath = FolderLocation + SingleValueName
    SingleValuePolygonName = SingleValue + str(MegaCount)
    SingleValuePolygonNamePath = GDB+"\\"+SingleValuePolygonName
    SingleValueCountName = CountValue + str(MegaCount)+".tif"
    SingleValueCountNamePath = FolderLocation + SingleValueCountName
    LandFormIncrementName = LandFormIValue + str(MegaCount) +".tif"
    LandFormIncrementNamePath = FolderLocation + LandFormIncrementName

    RC1 = str(BotVal)
    RC2 = str(MegaCount -1)
    NODATA = "NODATA"
    RC3 = str(MegaCount)       
    RC4 = str(TopVal)
    Val = 1
    Equ = "{0} {1} {2}; {3} {4} {5}".format (RC1,RC2,NODATA,RC3,RC4,Val)
    print "Process Number: " + str(Range2) + "   Height Value: "+ str(MegaCount) + " Count Processed: " +str(int(Range3)- int(Range2))
    print "A: Raster S Single Band:" + str(Range2) + " Equ: " +str(Equ)
    Range = Range -1
    Range2 = Range2-1

    Raster = arcpy.sa.Reclassify(InRaster,"Value",Equ, "NODATA")
    Raster.save(SingleValueNamePath)

    ## To Poly
    arcpy.RasterToPolygon_conversion(SingleValueNamePath, SingleValuePolygonNamePath, "NO_SIMPLIFY", "Value")
    print "B: Conversion Polygon: "+ SingleValuePolygonNamePath

    ####### TEST DIAGONALS
    print "      B1: Processing Diagonals"
    BufferNearName = BufferName + str(MegaCount)
    BufferNearNamePath = GDB+"\\" + BufferNearName
    MultiPartName = MultiName + str(MegaCount)
    MultiPartNamePath = GDB+"\\" + MultiPartName
    IntersectName = IntName + str(MegaCount)
    IntersectNamePath = GDB+"\\" + IntersectName
    DissolveName = DisName + str(MegaCount)
    DissolveNamePath = GDB+"\\" + DissolveName

    #Spatial Ref: GDA 94 Z50
    sr = arcpy.SpatialReference(28350)

    arcpy.Buffer_analysis(SingleValuePolygonNamePath,BufferNearNamePath,"100 decimeters","FULL","ROUND","ALL","")
    bufcount = arcpy.GetCount_management(BufferNearNamePath)
    result_value = bufcount[0]
    buffercount = int(result_value)

    MegaBuffCount = MegaBuffCount + buffercount

    if buffercount == 0:
    print "DANGER!!!!!!!!"
        arcpy.Delete_management(BufferNearNamePath)
        arcpy.Compact_management(GDB)
        arcpy.RepairGeometry_management (SingleValuePolygonNamePath)
        arcpy.AddSpatialIndex_management(SingleValuePolygonNamePath,0)
        arcpy.Buffer_analysis(SingleValuePolygonNamePath,BufferNearNamePath,"100 decimeters","FULL","ROUND","ALL","")
        winsound.PlaySound("SystemHand", winsound.SND_ALIAS)

        print "      B2: Buffer Error: " +str(bufcount) +" polygons. This Processing run: " + str(MegaBuffCount)

        bufcount2 = arcpy.GetCount_management(BufferNearNamePath)
        result_value2 = bufcount2[0]
        buffercount2 = int(result_value2)

        if buffercount2 == 0:
            arcpy.Delete_management(BufferNearNamePath)
            arcpy.Delete_management(SingleValueNamePath)
            arcpy.Delete_management(SingleValuePolygonNamePath)
            winsound.PlaySound("SystemHand", winsound.SND_ALIAS)
            winsound.PlaySound("SystemHand", winsound.SND_ALIAS)
            STOP # Deliberate error

    else:
        print "      B2: Buffer: " +str(bufcount) +" polygons. This Processing run: " + str(MegaBuffCount)

    arcpy.MultipartToSinglepart_management(BufferNearNamePath,MultiPartNamePath)
    Multicount = arcpy.GetCount_management(MultiPartNamePath)
    Mulresult_value = Multicount[0]
    MulCount = int(Mulresult_value)
    MegaMultiCount = MegaMultiCount + MulCount

    print "      B3: Multi Part: " +str(Multicount) +" polygons. This Processing run: "+ str(MegaMultiCount)

2 Answers 2

1

ArcGIS 10.1 has a few bugs when it comes to working with buffers. Especially with Geodesic buffers and the dissolve type "all" method. (NIM083208, NIM082599, NIM086086,NIM087868,NIM087913). I swapped to a projected system, then separated the buffer component into it's own module and the random errors stopped.

0

I've seen this issue in one of my applications that has considerable Geoprocessing requirements.

In my case I saw very similar symptoms running it with 4Gb RAM on several machines.

I then tested on several machines with 12Gb RAM and it ran cleanly on them.

My advice is, try to test it on a machine with more RAM before spending more effort trying to debug.

If you need to add RAM to make it work at the cost of a few $100s it will be money well spent.

2
  • You could be right, it's a rather intensive process. I was thinking of pushing parts of it into sub processes to manage the memory better. However I will test it on a better computer first.
    – GISI
    Commented Jul 10, 2014 at 8:33
  • So as luck would have it, I got a big box to process this on. The issue did not go away.
    – GISI
    Commented Oct 27, 2014 at 7:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.