3

I am trying to export MOD13A2 data from Earth Engine. Export attempts have resulted in payload errors consistently, even after chipping away at the data volume. I tried removing clipping elements of the workflow based on this answer, but to no avail. It's weird because at ~24 million km^2, I don't understand why the 1km MOD product for only North America exceeds payload limits. I'm not performing complex image operations, and if it were an issue of raster size I'd expect Earth Engine to break the export into tiles. The only thing I can think of is that I flatten the imageCollection to a single image with (in this case) 20 bands - but I tried using ee.ImageCollection.toBands() and Tyler Erikson's approach here. These are both techniques I've used in other contexts with no problem.

The error,

Request payload size exceeds the limit:41943304 bytes

only occurs after the export has been initiated. That leads me to believe this isn't an image operation issue, since the first layer is drawn correctly in the plotting window. Surprisingly (to me) it seems like my Macbook goes into overdrive when I select "run" from the export dialogue box - not sure what my computer needs to be doing as this isn't a client-side operation as far as I know. But, the export dialogue freezes, the fan kicks on high and Google Chrome gets super slow; only after a few minutes does the error pop up and my computer returns to a normal operating situation.

Reproducible workflow below:

//// Data filters
// Create a region of interest
var ROI = ee.FeatureCollection('USDOS/LSIB_SIMPLE/2017')
  .filter(ee.Filter.eq('wld_rgn', "North America"));
Map.addLayer(ROI, {}, "ROI");
// Create a QA mask + clipping function
var masker = function(image){ 
  var mask = image.select('SummaryQA').eq(0);
  var maskedImage = image.updateMask(mask);
  return maskedImage;
};

//// Import data
// Import, filter by date, and apply QA mask
var MOD = ee.ImageCollection("MODIS/006/MOD13A2")
  .map(masker)
  .filterDate("2000-01-01", "2001-01-01")
  .select("NDVI");
// Look at the image collection
print("MOD",MOD);

// Plot the first image 
Map.addLayer(MOD.first(), {min:0,max:10000}, "NDVI");

// Convert to single flattened image
var ndvi_Img = MOD.toBands();

// Export the data
// Export a cloud-optimized GeoTIFF.
// See https://developers.google.com/earth-engine/exporting
Export.image.toDrive({
  image: ndvi_Img,
  description: 'ndvi_Img',
  folder: "NoAm_MOD13A2",
  scale: 1000,
  region: ROI,
  fileFormat: 'GeoTIFF',
  maxPixels: 1.0E13,
  formatOptions: {
    cloudOptimized: true
  }
});
3
+50

The Issue

Functions from the class Export are all client-side functions, that's why your browser begins to struggle.

The specific issue you are having, is that you are passing a very complex and big feature collection to the region parameter of Export.image.toDrive(). I don't know what exactly happens behind the scenes but most likely since exporting is a client-side function, this comparatively big feature collection is being sent to your computer. There it is packaged into the export request which is ridiculously huge due to containing the entire geometry.

This leads to the error you get, complaining about the request size being too large.

The Fix

To fix it you need to construct a simpler region of interest and pass it to the export function. For example by constructing a polygon yourself:

var roi_simple = 
    ee.Geometry.Polygon(
        [[[-179, 85],
          [-179, 11],
          [-48, 11],
          [-48, 85]]], null, false);

or by simplifying the featureCollection first, before passing it to the export function:

var roi_bounding = ROI.map(function(feature){
  return feature.bounds()
})

var roi_simple = roi_bounding.geometry()

(To be honest though: This could most likely be silently handled by Earth Engine, since the region will get converted to a bounding box anyway afaik. So instead of doing the conversion after the request was sent from client side, passing the client only the bounding box of the specified region should work just as well. This would also fix freezing browsers exporting dubious regions.)

1
  • 1
    Excellent answer, thanks for the thought and clarity. One thing to note is that your use of roi_bounding.geometry() to generate roi_simple caused "holes" to appear in the final ROI where multiple features in the FeatureCollection overlapped. I was able to get around this by using roi_bounding.union() with a non-zero error instead. – JepsonNomad Aug 6 '20 at 18:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.