5

My goal is to calculate mean EVI for a collection of polygons for each image in an image collection.

My collection of polygons is a GEE feature collection (called 'polys') and I'm using the Landsat 8 EVI image collection.

I've been able to create a script that executes this task for a single polygon in this feature collection, posted below and link here.

My question is how do I take this script and scale it up to execute the task for my whole collection of polygons? Script I'm working on is also below and here.

Everything runs smoothly up until step 4 (mapping a reduceRegion/(s) function across the image collection), where earth engine returns a 'computation timed out' or an 'earth engine memory capacity exceeded' error. I've tried breaking both the image collection and the feature collection into smaller chunks with no success. I wonder if exporting a task to my google drive could be a solution, but I'm unsure of how to write the correct code for this.

Scripts:

1) Mean EVI for a single polygon for each image in a collection (works):

// 1. Filter 'polys' feature collection by property 'Name'. 
var NU009 = polys.filterMetadata('Name', 'equals', 'NU009');

// 2. Get Landsat 8 EVI collection and filter to NU009.
var l8 = ee.ImageCollection('LANDSAT/LC08/C01/T1_8DAY_EVI')
                            .filterBounds(NU009)
                            .filterDate('2014-01-01','2018-12-30');

// 3. Clip each Landsat image to the area covered by NU009.
var l8_clip = l8.map(function(image) {
   return image.clip(NU009);
  });

// 4. Calculate mean EVI for NU009 in each image.
//Define a function that calculates mean EVI for NU009.
//Returns an image collection where each image has a new property, 
//'meanEVI'.
var reducer = function(image) {
   var meanEVI = image.reduceRegion({
     reducer: ee.Reducer.mean(),
     geometry: NU009.geometry(),
     scale: 30,
     bestEffort: true
    });
   return image.set('meanEVI', meanEVI);
 };

//Map the reducer function over the image collection.
var l8_reduce = l8_clip.map(reducer);
print(l8_reduce);

2) Mean EVI for all polygons for each image in a collection (computation times out):

// 1. Filter 'polys' by size.

// Define a function that calculates area of each polygon.
// New property 'GEE_Area' is in square meters. 
var feature_area = function(feature) {
  var area = feature.geometry().area();
  return feature.set('GEE_Area', area);
};

polys = polys.map(feature_area);

// Filter polys by their GEE_Area.
// Must be greater than 30 square meters (900m) to be sufficiently 
//represented by a Landsat pixel.
var polys_filtered = polys.filterMetadata('GEE_Area', 'not_less_than', 900);

// 2. Get Landsat EVI image collection. 
var l8 = ee.ImageCollection('LANDSAT/LC08/C01/T1_8DAY_EVI')
                                .filterBounds(polys_filtered)
                                .filterDate('2014-01-01','2018-12-31');

// 3. Clip each Landsat image to the area covered by all polygons.
var l8_clip = l8.map(function(image) {
  return image.clipToCollection(polys_filtered);
});

// 4. Calculate mean EVI for each polygon in each image. 

// Define a function that calculates mean EVI for all polygons using 
//'reduceRegion' method.
var reduceRegion = function(image) {
  var meanEVI = image.reduceRegion({
    reducer: ee.Reducer.mean(),
    geometry: polys_filtered.geometry(),
    scale: 30,
    bestEffort: true
  });
  return image.set('meanEVI', meanEVI);
};

var l8_reduceRegion = l8_clip.map(reduceRegion);
print(l8_reduceRegion);

// Alternative to above function:
// Define a function that calculates mean EVI for all polygons using 
//'reduceRegions' method.
var reduceRegions = function(image) {
  var meanEVI = image.reduceRegions({
    collection: polys_filtered,
    reducer: ee.Reducer.mean(),
    scale: 30});
  return meanEVI;
}; 

var l8_reduceRegions = l8_clip.map(reduceRegions);
print(l8_reduceRegions);
4
  • Please include scripts in the body of the question.
    – TomazicM
    Commented Dec 2, 2019 at 20:25
  • scripts have been added! Commented Dec 2, 2019 at 20:54
  • We cannot reproduce the error without access to your feature table polys... You'll have to create a script where the asset is loaded (also set accessibilityto public). Most likely you'll have to increase the scale, reduce the geometry size or decrease the feature collection in size.
    – Jobbo90
    Commented Dec 3, 2019 at 10:58
  • Thanks Jobbo90 for your comment, I've added links to both scripts, including assets, to the post above. Commented Dec 3, 2019 at 19:10

1 Answer 1

7

You should definitely try to export the result, which will likely solve the time out problem. Below is a toy example that replicates your two region reduction options and demonstrates exporting the tables as CSV files to your Google Drive account. A few things to note:

  1. There is no need to clip each image by NU009, the region reducing functions will do this for you based on the provided argument to the geometry parameter. Additionally, if you are clipping an image by a featureCollection, be sure to use: clipToCollection() instead of clip().

  2. The two options you are considering produce quite different results. Consider what kind of table output you want. The first calculates the mean of EVI of the aggregate pixels interesting the features of the featureCollection and the second calculates mean EVI per image per feature.

  3. It is worth trying to export the results using the native 30m resolution and all of the features you currently have in the featureCollection and run it with the profiler (Crtl+Alt+Enter) so you can track what functions are using a lot of memory.

  4. The toy example script uses only 5 months of imagery from one path/row and there are about 39 features. These attributes allow it to run quickly in the browser and export in about 1 minute. A larger time series with more features (and/or more complexity) may not process fast enough to print results to the Code Editor console (times out after 5 minutes) and may take much longer to export.

Code Editor script

/**
 * @license
 * Copyright 2019 Google LLC.
 * SPDX-License-Identifier: Apache-2.0
 * 
 * @description
 * An example of reducing all the elements of an imageCollection by all
 * the elements of a featureCollection and exporting the results to
 * Google Drive.
 */

// Import US counties and protected areas featureCollections. 
var counties = ee.FeatureCollection('TIGER/2018/Counties');
var protectedAreas = ee.FeatureCollection("WCMC/WDPA/current/polygons");

// Create a featureCollection that is all of the protected areas interesting
// Santa Clara County, CA, USA.
var scCounty = counties.filter(ee.Filter.eq('NAME', 'Santa Clara'));
var polys_filtered = protectedAreas.filterBounds(scCounty);

// Import a Landsat EVI collection and filter it by geographic extent and date
// range.
var l8 = ee.ImageCollection('LANDSAT/LC08/C01/T1_8DAY_EVI')
  .filterBounds(polys_filtered)
  .filterDate('2018-01-01','2018-06-01');



// Below are two options for summarizing a time series of EVI for the protected
// areas interesting Santa Clara County.

// #############################################################################
// ### REDUCE AGGREGATE REGIONS OPTION ###
// #############################################################################

// Merge the geometries of the 'polys_filtered' featureCollection into a single
// multiPolygon geometry.
var geom = polys_filtered.geometry();

// Define a function that will calculate mean EVI of all pixels within the
// multiPolygon geometry defined by the 'polys_filtered' featureCollection.
var reduceRegion = function(image) {
  // This regionReduction returns a dictionary; in this case a single key-value
  // pair for the mean summary of EVI.
  var meanEVI = image.reduceRegion({
    reducer: ee.Reducer.mean(),
    geometry: geom,
    scale: 30,
    bestEffort: true});
  // Along with mean EVI for the region (renamed), add some info about the image
  // that mean EVI is based on (the image ID and the image date). Define these
  // attributes in a dictionary.
  var props = {
    'meanEVI': meanEVI.get('EVI'),
    'imgID': image.id(),
    'date': image.date().format('YYYY-MM-dd')};
  // Return the information as a feature, so that the result of mapping this
  // function over a collection will be another collection that can be cast as
  // a featureCollection, which can be exported to Google Drive. Use the 'geom'
  // to defined the feature's geometry, and the 'props' dictionary to define the
  // properties of the feature.
  return ee.Feature(geom, props);
};

// Map the above function over the Landsat EVI imageCollection to calculate mean
// EVI per image for the region expressed by the merged geometries of all the
// features in the 'polys_filtered' featureCollection. Cast the result as a
// featureCollection.
var l8_reduceRegion = ee.FeatureCollection(l8.map(reduceRegion));

// Export the featureCollection (table) as a CSV file to your Google Drive
// account. The resulting table will have as many rows as there are images and
// three columns for the 'props' set above as well as some system properties
// added by default. The table will be written to the Google Drive account
// associated with your Earth Engine account and will be placed in a folder
// called 'reduce_region_test' and named 'l8_reduceRegion' - both provided as
// parameter arguments.
Export.table.toDrive({
  collection: l8_reduceRegion,
  description: 'l8_reduceRegion',
  folder: 'reduce_region_test',
  fileFormat: 'CSV'});

// Plot the table as a time series chart. X is the date, Y is mean EVI. There is
// only one series because all of the features' geometries are merged.
print(ui.Chart.feature.byFeature(l8_reduceRegion, 'date', 'meanEVI'));



// #############################################################################
// ### REDUCE BY INDIVIDUAL REGIONS OPTION ###
// #############################################################################

// Define a function to be mapped over the EVI image collection that calculates
// mean EVI per image for all polygons in the 'polys_filtered' featureCollection
// using the 'reduceRegions' method.
var reduceRegions = function(image) {
  var meanEVI = image.reduceRegions({
    collection: polys_filtered,
    reducer: ee.Reducer.mean(),
    scale: 30});
  // Return the featureCollection with the EVI mean summary per feature, but
  // first...
  return meanEVI
    // ...remove any features that have a null value for any property.
    .filter(ee.Filter.notNull(['mean']))
    // ...map over the featureCollection to edit properties of each feature.
    .map(function(feature) {
      // Return the feature, but first...
      return feature
        // ...select only two properties of interest; there are about 30
        // (all of the properties of the original feature plus mean EVI), not all
        // are necessary in this case. Also rename the 'mean' property returned
        // be default from the above reduceRegions function to 'meanEVI'.
        .select(['mean', 'WDPAID'], ['meanEVI', 'WDPAID'])
        // ..add an image ID and date property of the image being reduced.
        .set({
          'imgID': image.id(),
          'date': image.date().format('YYYY-MM-dd')
      });
  });
};

// Apply the above defined function to all images. The result will be a
// collection of featureCollections. Each collection contains all of the
// 'polys_filtered' features with a set of properties that describe the mean
// EVI for each feature along with an image ID from which EVI is based on
// and the date of the image.
var l8_reduceRegions = l8.map(reduceRegions)
  // Flatten the collection of featureCollections into a single featureCollection.
  .flatten();

// Export the featureCollection (table) as a CSV file to your Google Drive account.
// See the above note on 'Export.table.toDrive' for more information.
Export.table.toDrive({
  collection: l8_reduceRegions,
  description: 'l8_reduceRegions',
  folder: 'reduce_region_test',
  fileFormat: 'CSV'
});

// Plot the table as a time series chart. X is the date, Y is mean EVI. There
// are same many series as there are features in the 'polys_filtered' feature
// featureCollection.
print(ui.Chart.feature.groups(l8_reduceRegions, 'date', 'meanEVI', 'WDPAID'));
2
  • Thanks jdbcode for the comments and for the example script. Will definitely play around with this script using my data. I've added links to both of my scripts including my feature collection as a loaded asset in case you're interested in having a look. Commented Dec 3, 2019 at 19:17
  • thanks Justin. I had a similar problem so I used your code (option 2) and it works in providing a csv with averages for each polygon. The problem is that in the resulting csv, not all polygons are included (4 out of 5 are not included), and the averages are slightly different from the image downloaded and averaged manually. What could have possibly gone wrong? TIA
    – okapina
    Commented May 23, 2022 at 8:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.