I am working on a single tiff with over 20k bands, each representing daily temperature, and I'll be extracting and preforming simple raster math on roughly 3500 of the bands. The problem I'm running into is that each band is taking ~2 min to process which would equate to a 4 day running time. I am sure there must be a more efficient manner for processing this.
*edit: When running for 1 band in range (1, 2) I am still getting a ~2min process time.
*edit 2: Relevant information of file.
Columns, Rows: 179, 195
Number_of_Bands: 20454
Cell_Size__X.Y: 0.0625, 0.0625
Format: TIFF
Pixel_Type: floating point
Pixel_Depth: 32 Bit
I have run the same script on a subset with ~200 bands from the file above and the processing time per-band is nearly 2 seconds each.
>>import took 0:00:20.362000
>>set up environment took 0:00:08.220000
>>a full band loop took 0:01:43.342000
>>a full band loop took 0:01:42.302000
>>a full band loop took 0:01:41.980000
>>a full band loop took 0:01:41.359000
>>a full band loop took 0:01:41.072000
>>one season loop took 0:08:32.306000
import datetime
t1 = datetime.datetime.now()
import arcpy, os, arcinfo
from arcpy.sa import *
t2 = datetime.datetime.now()
print "import took %s" % ( t2-t1)
arcpy.env.overwriteOutput = True
# Set the current workspace
arcpy.env.workspace = "C:\\raster\\folder"
arcpy.CheckOutExtension('Spatial')
# Get and print a list from the workspace
rasters = arcpy.ListRasters('*', 'TIF')
cutoffTemp = float(277.2)
t3 = datetime.datetime.now()
print "set up environment took %s" % ( t3-t2)
for raster in rasters:
t4 = datetime.datetime.now()
print(raster)
d = arcpy.Describe(raster)
number_of_bands = d.bandCount #number of bands from Hist2005
print(number_of_bands)
arcpy.AddMessage("Processing raster {} for {} bands".format(raster, number_of_bands))
listcomp90 = [] #create a list of output tifs, empty for each input raster
for band_number in range(1, 6): #small range for test
t6 = datetime.datetime.now()
listcomp91.append(band_number)
Hist = arcpy.Raster("{}\\Band_{}".format(raster, band_number)) #gets inidiviual bands for input
arcpy.AddMessage('in the band number cycle and the current band number is {}'.format( band_number))
output = arcpy.sa.Con(Hist >= cutoffTemp, sapHist) #if band values >= cuttoffTemp then temp
Rasname, Rasext = os.path.splitext(raster)
output.save(os.path.join(arcpy.env.workspace, "New{}_91_comp_{}.tif".format(Rasname, band_number)))
listcomp91.append(os.path.join(arcpy.env.workspace, "New{}_91_comp_{}.tif".format(Rasname, band_number)))
t7 = datetime.datetime.now()
print "a full band loop took %s" % (t7-t6)
t5 = datetime.datetime.now()
print "one season loop took %s" % (t5-t4)