Counting pixels in masked raster in Google Earth Engine results in error 'Remote request too large'

Question

Following on from a previous question (Points to raster (image) in Google Earth Engine) I have created an Image of observations for a particular species, masked such that we are only considering pixels with a value of one (i.e. the species was observed in that pixel).

The extent of the image is necessarily large, because the observations were spread out, which means that the whole image contains many, many pixels but most are zeroes and should therefore be ignored in calculations.

All I want to do is count the number of pixels with a value of one. I tried the following code:

// Mask
var ebdMasked = ebdCountImg.mask(ebdCountImg.gt(0));

// Count pixels
var count = ebdMasked.reduceRegion({
  reducer: ee.Reducer.count(),
  geometry: roi,
  maxPixels: 1e13
  });
print(count);

Which produced the error:

Dictionary (Error)
Remote request too large (3355443200 > 83886080) for output: [<Object>].

I actually know in this case that the answer is 256, so in no way should this be too large for output. I assumed something was wrong with the mask meaning that all pixels were being considered, so I exported the masked Image to take a look. It seemed fine and, lo and behold, when I read the exported ebdMasked back in, and try again, it works as expected:

// Try again with exported layer
var ebdMasked2 = ee.Image("users/rasenior/ebdImg");

// Mask
var ebdMasked2 = ebdMasked2.mask(ebdMasked2.gt(0));

// Count pixels
var count2 = ebdMasked2.reduceRegion({
  reducer: ee.Reducer.count(),
  geometry: roi,
  maxPixels: 1e13
  });

print(count2);

Produces:

Object (1 property)
   first: 256

I'm sure I'm missing something very simple!

Full code is here: https://code.earthengine.google.com/6608ad2279f249324ed730a56605a025

Answer 1

The error “Remote request too large …” is an unfortunate limitation — unfortunate in that it really shouldn't exist, and it's also not worded very well.

What it actually means is that you're somehow trying to compute an image that's too big (number of pixels times number of bytes needed to store each pixel). Most likely, this is due to your reprojection and reduceResolution somehow requiring computing many more pixels intermediately than the reduceRegion asked for per tile of the reduction.

Ideally, Earth Engine would automatically try computing the reduction in smaller tiles. But for now, you can specify the tileScale parameter in reductions — try tileScale: 8.

---

As of October 24, 2019, this error message has been changed, hopefully for the better. It now reads, for example from the above script:

Output of image computation is too large (3200.0 MiB > 80.0 MiB).
If this is a reduction, try specifying a larger 'tileScale' parameter.