2

I want to sum the pixel values of large numbers of rasters of different extents (ArcMap v10.2). 16GB RAM.

I have the following code (with help from this site) which solves the different extent problem and works fine for up to about 1000 rasters (40-50 mb) and outputs a single raster of about 50mb correctly. But when the number gets much larger ArcGIS closes. It iterates through all the rasters but crashes at the save - I suspect it is accumulating data in memory throughout the loop and eventually runs out of memory.

At the moment the rasters are all in directory folders (5000 per folder - I need to find a better way to do this as well).

None of the file paths have spaces in them.

I have tried saving as tif rather than GRID but no difference. I have tried to save within the loop but couldn't get that to work. I've tried using a .gdb as the workspace and scratchWorkspace. Ive tried some of the things suggested here https://geonet.esri.com/thread/101053 . Next im going to try storing the rasters in a mosaic dataset and iterate through that but I expect that will take me awhile to figure out.

import arcpy
import os
from arcpy.sa import *


ws=arcpy.GetParametersAsText(0)
outputFolder=arcpy.GetParametersAsText(1)
outputName=arcpy.GetParametersAsText(2)
extentRaster=arcpy.GetParametersAsText(3)

arcpy.env.overwriteOutput=1
arcpy.env.scratchWorkspace = ws
arcpy.env.workspace = ws
arcpy.CheckOutExtension("Spatial")

eRaster=Raster(extentRaster)
arcpy.env.extent=eRaster.extent
arcpy.env.snapRaster=extentRaster

#create a list of rasters in the workspace
rasterList = arcpy.ListRasters("*", "GRID")

i = 0
#loop through rasters in list
for raster in rasterList:
    arcpy.AddMessage ("processing raster: %s" %i)

    #sum rasters together and convert no data to zero
    if i == 0:
        outSUM = Con(IsNull(raster),0,raster)
        i += 1
    else:
        outSUM = outSUM + Con(IsNull(raster),0,raster)
        i += 1

#save final output to the current workspace
arcpy.env.workspace = outputFolder
outSUM.save(outputName)
  • 1
    Maybe you can write a parent script that calls your script anew every 1000 or so rasters, and stores intermediary rasters from each group of 1000 that you can sum up at the end? – Martin Jan 9 '15 at 13:29
  • 1
    A few thoughts: 1) try running cell statistics 2) run the script as a 64-bit background geoprocessing task 3) tile the images based on a fishnet grid (i.e. make the problem smaller) and loop through the tiles (e.g. a_1 + b_1 + c_1; a_2 + b_2 + c_2;...). – Aaron Jan 9 '15 at 14:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.