0

I am using the Earth Engine Python API in a script tool that I'm running from ArcMap. Essentially, the tool will allow the user to draw an area of interest, and from this area of interest, the total rainfall will be calculated using CHIRPS data from Google Earth Engine.

I am able to get this to work fine when I run each line individually from the IDLE development environment: rainfall is calculated within my area of interest and a bar graph is returned. However, when I run the script from the script tool in ArcMap, I get the following error:

an error occurred: ImageCollection.getRegion: Too many values: 3452924 points x 1 bands x 11 images > 1048576.

I tried drawing a very small area of interest (ten square meters), but I still I got this error. The problem is occurring when I use the getRegion command to bound the image collection by just the area of interest. But again, this didn't throw an error when I was using IDLE.

Below is the relevant snippet of my script (note: the getRegion command is in the third-to-last line of the code):

def earthEngine(aoi):
    ee.Initialize()
    arcpy.AddMessage("In GEE function")
    #Set the bounding box coordinates of the aoi
    desc = arcpy.Describe(aoi)
    xmin = desc.extent.XMin
    xmax = desc.extent.XMax
    ymin = desc.extent.YMin
    ymax = desc.extent.YMax
    boundingBox = ee.Geometry.Polygon([[[xmin, ymax], [xmin, ymin], [xmax, ymin], [xmax, ymax]]])

    #CHIRPS image collection
    chirps = ee.ImageCollection('UCSB-CHG/CHIRPS/DAILY')

    #Set the time period over which images will be captured.
    #Only rainfall from the past year will be reported.
    now = datetime.datetime.now()
    endYr = now.year
    endMo = now.month
    endDy = now.day
    stYr = endYr - 1
    stMo = endMo
    stDy = endDy
    #Create variables for years and months of interest
    years = range(stYr, endYr + 1)
    endMoRange = endMo + 13
    months = range(endMo, endMoRange)
    #Set date in EE format
    startdate = ee.Date.fromYMD(stYr, stMo, 1)
    enddate = ee.Date.fromYMD(endYr, endMo, 1)
    #Filter CHIRPS by date
    Pchirps = chirps.filterDate(startdate, enddate).sort('system:time_start').select('precipitation')
    #Function to calculate monthly precipitation total
    def calcMonthlySum(imageCollection):
        mylist = ee.List([])
        for m in months:
            if m <= 12:
                mo = m
                yr = years[0]
                w = imageCollection.filter(ee.Filter.calendarRange(yr, yr, 'year')).filter(ee.Filter.calendarRange(mo, mo, 'month')).sum()
                mylist = mylist.add(w.set('system:time_start', str(mo) + '/' + str(yr)))
            else:
                mo = m - 12
                yr = years[1]
                w = imageCollection.filter(ee.Filter.calendarRange(yr, yr, 'year')).filter(ee.Filter.calendarRange(mo, mo, 'month')).sum()
                mylist = mylist.add(w.set('system:time_start', str(mo) + '/' + str(yr)))
        return ee.ImageCollection.fromImages(mylist)

    #Call function to calculate monthly rainfall
    monthlyChirps = ee.ImageCollection(calcMonthlySum(Pchirps))
    #Limit output to the area of interest
    monthlyChirps2 = monthlyChirps.getRegion(boundingBox,24000,"epsg:4326").getInfo()

    #Create pandas dataframe to plot
    df = pd.DataFrame(monthlyChirps2, columns = monthlyChirps2[0])

aoi, which is passed to the earthEngine function, is a polygon that the user draws.

It seems that the problem is that the data being passed to getRegion is too large. I'm curious if there's a work-around to this, and/or if there's an explanation as to why this will work in an integrated development environment, but not when it's run from ArcMap.

  • I have no answer, but it is amazing to me that you were able to integrate these at all. I suppose you tried this question on the ee developers' list? Does it work with a defined aoi, not user-drawn? – J Kelly Aug 17 '18 at 2:39
  • 2
    How do you know it works from IDLE? Do you get a printout or something? What happens if you draw a point? I wonder if you're getting a universe polygon mixed in there somehow. – Nicholas Clinton Aug 17 '18 at 21:25
  • Thanks for the suggestion about the EE Developers' list. I have not yet posted it there. I will do that next. The script still has the same error when using a user defined AOI (ie, uploading a polygon shapefile). @NicholasClinton I was able to get it work with a user-drawn point shapefile (rather than a polygon). It appears that ArcMap fails when reading the large set of data within the polygon, but when it is a single point, it succeeds. – paul Aug 17 '18 at 21:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.