Earth Engine: feature collection will not export even in chunks

Question

I am trying to export zonal statistics from the Worldpop dataset for about 450,000 0.01° tiles, but I can't get it to work. When I break the data into chunks of 50,000 observations, the computation still times out on each.

My tiles were imported into GEE (anyone can read):

var tiles = ee.FeatureCollection('users/salem043/deforSample');

I mosaicked the country-year data in Worldpop together to produce annual images:

var years = ee.List.sequence(2000, 2012)
// Make some band names from the years numbers.
var yearBandNames = years.map(function(n) { return ee.Number(n).format("pop_%d") })

var yearImages = ee.ImageCollection(
  years.map(function(year) {
    var oneYear = ee.ImageCollection("WorldPop/GP/100m/pop")
      .filter(ee.Filter.calendarRange(year, year, 'year'))
      .filterBounds(tiles)
    return oneYear.mosaic()
  }))
  .toBands()
  .rename(yearBandNames)

Then to produce the data frame, I select the year image and perform a reducer function to get the zonal statistics. I perform the same function as defined below for all years 2001-2012:

var pop_2001 = yearImages.select("pop_2001");

var sumPop_2001 = tiles.map(function(feature) {
  return feature.set(pop_2001.reduceRegion({
    reducer: ee.Reducer.sum(), geometry: feature.geometry(), scale: 30
  }));
});

I tried exporting this data in chunks of 50,000 tiles at a time, but each computation timed out. Can anyone offer advice as to how to make this program more efficient so that it exports? Full code here

Answer 1

First, since the scale of your raster is 100 meters, set the scale in reduceRegion to 100.

Then, if you do the same for every year and every tile, why not map over the Image Collection instead of calling .toBands() and use reduceRegions() instead of reduceRegion().

Something like this should work (haven't tested it):

var yearImages = ee.ImageCollection(
  years.map(function(year) {
    var oneYear = ee.ImageCollection("WorldPop/GP/100m/pop")
      .filter(ee.Filter.calendarRange(year, year, 'year'))
      .filterBounds(tiles)
    return oneYear.mosaic().set('system:index', ee.String(year))
  }))

var sumPop = yearImages.map(function(image) {
  return image.reduceRegions({
    collection: tiles, 
    reducer: ee.Reducer.sum(), 
    scale: 100
  })
});
var flat = sumPop.flatten()

It might still time out, since 450,000 is quite a lot, but like this you should have a better chance.