2

I am trying to export zonal statistics from the Worldpop dataset for about 450,000 0.01° tiles, but I can't get it to work. When I break the data into chunks of 50,000 observations, the computation still times out on each.

My tiles were imported into GEE (anyone can read):

var tiles = ee.FeatureCollection('users/salem043/deforSample');

I mosaicked the country-year data in Worldpop together to produce annual images:

var years = ee.List.sequence(2000, 2012)
// Make some band names from the years numbers.
var yearBandNames = years.map(function(n) { return ee.Number(n).format("pop_%d") })

var yearImages = ee.ImageCollection(
  years.map(function(year) {
    var oneYear = ee.ImageCollection("WorldPop/GP/100m/pop")
      .filter(ee.Filter.calendarRange(year, year, 'year'))
      .filterBounds(tiles)
    return oneYear.mosaic()
  }))
  .toBands()
  .rename(yearBandNames)

Then to produce the data frame, I select the year image and perform a reducer function to get the zonal statistics. I perform the same function as defined below for all years 2001-2012:

var pop_2001 = yearImages.select("pop_2001");

var sumPop_2001 = tiles.map(function(feature) {
  return feature.set(pop_2001.reduceRegion({
    reducer: ee.Reducer.sum(), geometry: feature.geometry(), scale: 30
  }));
});

I tried exporting this data in chunks of 50,000 tiles at a time, but each computation timed out. Can anyone offer advice as to how to make this program more efficient so that it exports? Full code here

1 Answer 1

0

First, since the scale of your raster is 100 meters, set the scale in reduceRegion to 100.

Then, if you do the same for every year and every tile, why not map over the Image Collection instead of calling .toBands() and use reduceRegions() instead of reduceRegion().

Something like this should work (haven't tested it):

var yearImages = ee.ImageCollection(
  years.map(function(year) {
    var oneYear = ee.ImageCollection("WorldPop/GP/100m/pop")
      .filter(ee.Filter.calendarRange(year, year, 'year'))
      .filterBounds(tiles)
    return oneYear.mosaic().set('system:index', ee.String(year))
  }))

var sumPop = yearImages.map(function(image) {
  return image.reduceRegions({
    collection: tiles, 
    reducer: ee.Reducer.sum(), 
    scale: 100
  })
});
var flat = sumPop.flatten()

It might still time out, since 450,000 is quite a lot, but like this you should have a better chance.

5
  • This is helpful - thank you! But is the mapping reducer still missing something? The function only produces one zonal statistic: It should produce multiple, for each year.
    – C. Ashley
    Mar 22, 2021 at 14:16
  • I don't know, unfortunately I can't try out the code quickly since you didn't give access to your script
    – JonasV
    Mar 22, 2021 at 14:30
  • Perhaps the link to my script was not working earlier? I just edited the original post, updating the link to the script. It should work now.
    – C. Ashley
    Mar 22, 2021 at 14:38
  • It's best to give access by using the Get Link button in the code editor, try it this way
    – JonasV
    Mar 22, 2021 at 14:39
  • I just tested it and everything is working. I only changed my answer a bit, to show from which year what is coming from. If it still isn't clear I would need more info about your tiles asset to help further.
    – JonasV
    Mar 22, 2021 at 14:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.