3

Description: The code below contains a function that calculates the least required days for Sentinel 2 image collection to cover at least 99% of a given area of interest (aoi). In other words, Sentinel 2 can fully cover small areas with image collection filtered with 1 day only. If an area is relatively large then image collections need to be gathered from more than 1 day and then mosaiced.

Problem: The function works as expected ,however, it freezes the web browser because it uses .getInfo() command inside it. Specifically at var c = data1.size().getInfo() line located near the end of the code.

Requirement: I would like to find an alternative to this command, so that the function won't freeze the web browser. I believe the solution is by converting the whole function to server-side, but I couldn't achieve that.

Failed solutions:

  • I've tried using .evaluate() instead of .getInfo() , and it didn't work.
  • I've tried to reconstruct the whole function by using .map instead of while loop. It also didn't work.

The Code:

var Iraq = ee.FeatureCollection("users/Yousif_Almamalachy/Iraq_Governorates");
var Gov_name = 'ANBAR'
var aoi = Iraq.filter(ee.Filter.eq("NAME_GOVER", Gov_name));

print(CdaysCheck(aoi))

function CdaysCheck(aoii){
var stoper = false;
var i = 1;
while (i) {
  var sentinel = ee.ImageCollection("COPERNICUS/S2_SR");
  var aoi = aoii;
  
  
  var sStart_Date = ee.Date(Date.now()).advance(-1,'month')
  var eEnd_Date =  ee.Date(Date.now())
  var aAllowed_Cloud = 100
  var Minimum_Areal_Coverage = 0.99
  var cCoverage_Days = i
  var data = sentinel.filterBounds(aoi).filterDate(sStart_Date,eEnd_Date)
    .filter(ee.Filter.lt('CLOUDY_PIXEL_PERCENTAGE', aAllowed_Cloud));
  var sscale=data.first().select('B1').projection().nominalScale().getInfo()
  var polygon_area;
  // if (definedmode.getValue() == 1)
      polygon_area = aoi.geometry().area().divide(1e6)
  // else{polygon_area = aoi.area().divide(1e6)}
  var minimum_area = ee.Number(polygon_area).multiply(Minimum_Areal_Coverage)
  
  // Get the dates of filtered images with less than specified cloud percentage
  var dates = data
      .map(function(image) {
        return ee.Feature(null, {'date': image.date().format('YYYY-MM-dd')})
      })
      //.distinct('date')
      .aggregate_array('date')

  // Filter Images that doesn't contain all bands
  var data = data.map(function(n) {
    return n.set('Num_of_bands',n.bandNames().length());
  })
  data = data.filterMetadata("Num_of_bands","equals",23);

  // Mosaic images of the same date and calculate its average cloud cover
  var dd = ee.ImageCollection(
    ee.List(dates.distinct()).map(function (n) {
    var date = ee.Date(n)
    var filtered = data.filterDate(date, date.advance(cCoverage_Days,'day')) 
    var image = ee.Image(filtered.mosaic())
    return image.set({'date':n})
  }));
  // Filter Images that doesn't cover polygon area (raster area calculation method)
  var data1 = dd.map(function(n) {
    var data = ee.Image(1).mask(n.select('B1'))
    var raster_area = data.multiply(ee.Image.pixelArea()).reduceRegion({
      reducer: ee.Reducer.sum(),
      geometry: aoi,
      scale: sscale,
      maxPixels: 1E13
    }).get('constant')
    raster_area=ee.Number(raster_area).divide(1e6)
    return n.set('Area_KM2',raster_area)
  })
  data1 = data1.filterMetadata("Area_KM2","greater_than",ee.Number(minimum_area));
  var c = data1.size().getInfo()
  if (c !== 0) 
    {break}
  i += 1
  }
  return ee.String(i.toString())
}

2 Answers 2

4

This kind of computation ('loop until condition') is never a good fit for Earth Engine and you should always figure out how to reframe the question such that you don't need it, rather than try to force it.

In this case, you can simply compute the coverage for N days (N= [1, 2, 3, 4, 5, ...]) and then find the first one that exceeds your coverage threshold with a filter. It's more computation, but much easier.

Also: as a simplifying hint, you can just get the mean() of the mask in the aoi to determine % coverage (masked pixels will be 0).

https://code.earthengine.google.com/1659bba88fcd562bb5faee3f28bfe51c

var Iraq = ee.FeatureCollection("users/Yousif_Almamalachy/Iraq_Governorates")
var Gov_name = 'ANBAR'
var aoi = Iraq.filter(ee.Filter.eq("NAME_GOVER", Gov_name))
  
var aAllowed_Cloud = 100
var Minimum_Areal_Coverage = 0.99
var start = ee.Date('2021-07-01')

var sentinel = ee.ImageCollection("COPERNICUS/S2_SR")
    .filterBounds(aoi)
    .filter(ee.Filter.lt('CLOUDY_PIXEL_PERCENTAGE', aAllowed_Cloud))
    
var proj = sentinel.first().select('B1').projection()

// Returns a feature containing the percentage of unmasked pixels.
function coverage(days) {
  var end = start.advance(days, 'day')
  var mosaic = sentinel.filterDate(start, end).mosaic()
  var cover = mosaic.select(0).mask().reduceRegion({
    reducer: ee.Reducer.mean(),
    geometry: aoi, 
    crs: proj,
    maxPixels: 1e10
  }).values().get(0)
  
  return ee.Feature(null, {
    days: days,
    cover: cover
  })
}

var result = ee.FeatureCollection(ee.List.sequence(1, 15).map(coverage))
print(ui.Chart.feature.byFeature(result, 'days', 'cover'))
var minDays = result.filter(ee.Filter.gte('cover', Minimum_Areal_Coverage)).first().get('days')
print('Required days: ', minDays)
3
  • Dear @Noel, Thanks for your insightful answer. However, the code needs to loop over increment days (1,2,3 and so on) and then loop over dates retrieved from ImageCollection. If you notice, the code you provided return an answer of 10 while the true answer is 3. The code must implement a loop inside loop as mentioned above, and if run without a stopping condition it will be exhaustive code and will not run. Let me rephrase my question here for clarification. I need a method to run a loop and stop it when a condition is met all on server-side. Commented Aug 8, 2021 at 8:14
  • 1
    As I said, that'd not something earth engine does well. The only way to do that is to use an iterate and a server-side ee.Algorithms.if function, however that's not going to make your code work better (in fact, it'll make it worse) because Earth Engine will always evaluate both sides of the if() and then return just one of them. If you want to map this function over a bunch of dates, then just add another map around it for the dates you want to consider. Commented Aug 9, 2021 at 13:33
  • I see your point, and I already did what you said about adding the dates loop. The code works, but its exhaustive as I mentioned and will not run if the aoi is relatively big. That being said, the stopping condition within a loop is a must. Can you please show me how to stop an ee.List.iterate function at specific condtion? if it's possible take the following code for showing the method: code.earthengine.google.com/a2efdfea06ae518d024256d2785311f9 Commented Aug 10, 2021 at 10:11
1

After a fruitful conversation with Noel Gorelick, I've managed to write a function parallel to the one in my question but completely done in server-side. This way I've avoided using .getInfo() as requested.

The basic idea here is to implement a loop that breaks at specific condition which can be easily done in client-side using break command. To do the same work on server-side, one must understand the fact that there is no code breaking in server-side. Instead, I used ee.List.iterate function with multiple ee.Algorithms.If functions to make the loop divert to a micro-loop (non-exhaustive one) when the condition is met as shown below:

var Iraq = ee.FeatureCollection("users/Yousif_Almamalachy/Iraq_Governorates");
var Gov_name = 'WASIT'
var aoi = Iraq.filter(ee.Filter.eq("NAME_GOVER", Gov_name));
///////////////////////////////////////////////
///////////////////////////////////////////////
///////////////////////////////////////////////
print(CdaysCheck(aoi))

function CdaysCheck(){
// This function calculates the minimal days required for sentinel 2 collection
// to cover at least 99% of a specific aoi (area of interest). 
// It generates a mosaic for every number of days (starting with 1 day)
// then it calculates it's aerial coverage with respect to aoi and test If it exceeds 99% or not.
// If not the number of days is incresed and the procedure is repeated. Otherwise the loop terminates.
  
var list = ee.List.sequence(1,5) //Iterate for 5 days.

//In each iteration if check1 function returns 0 then add 1 to the previous iteration and rerun check1
// Else if check1 returns 999 or any number other than 0 then return 999
/// 999 is used as a flag to terminate the iteration.
var addone = function(current,previous){
  previous = ee.List(previous)
  var pr = ee.Number(previous.get(-1))
  var cu = ee.Number(current)
  var cd = ee.Number(check1(pr,aoi))
  var n1 = ee.Algorithms.If(cd.eq(ee.Number(0)),pr.add(1),
           ee.Algorithms.If(cd.eq(ee.Number(999)),
           ee.Number(999),ee.Number(999)))
  return previous.add(ee.Number(n1))
}

var start = [1] //Start iteration at 1
var result = ee.List(list.iterate(addone,start))
result = ee.String(result.filter(ee.Filter.lt('item', 999)).get(-1)) 
return ee.String(result)
///////////////////////////////////////////////
///////////////////////////////////////////////
///////////////////////////////////////////////

function check1(i,aoii){
  
  // If i equals to 999 then don't go further... just return 999.
  // Else if i equals any number other than 999 or 0 then move forward with the continu function.
  return ee.Number(ee.Algorithms.If(ee.Number(i).neq(ee.Number(999)),
                   continu(ee.Number(i),aoii),ee.Number(999)))
                   

  function continu(i,aoii){
    var aa = i
    var sentinel = ee.ImageCollection("COPERNICUS/S2_SR");
    var aoi = aoii;
    
    var sStart_Date = ee.Date(Date.now()).advance(-30,'day')
    var eEnd_Date =  ee.Date(Date.now()).advance(-20,'day')
    var aAllowed_Cloud = 100
    var Minimum_Areal_Coverage = 0.99
    var cCoverage_Days = aa
    var data = sentinel.filterBounds(aoi).filterDate(sStart_Date,eEnd_Date)
      .filter(ee.Filter.lt('CLOUDY_PIXEL_PERCENTAGE', aAllowed_Cloud));
    // var sscale=data.first().select('B1').projection().nominalScale().getInfo()
    var sscale= 10
    var polygon_area;
    // if (definedmode.getValue() == 1)
        polygon_area = aoi.geometry().area().divide(1e6)
    // else{polygon_area = aoi.area().divide(1e6)}
    var minimum_area = ee.Number(polygon_area).multiply(Minimum_Areal_Coverage)
    
    // Get the dates of filtered images with less than specified cloud percentage
    var dates = data
        .map(function(image) {
          return ee.Feature(null, {'date': image.date().format('YYYY-MM-dd')})
        })
        .aggregate_array('date')
  
    // Filter Images that doesn't contain all bands
    var data = data.map(function(n) {
      return n.set('Num_of_bands',n.bandNames().length());
    })
    data = data.filterMetadata("Num_of_bands","equals",23);
    
    // Mosaic images of the same date/date range and calculate the area of the mosaic
    // return the result as image collection 
    var dd = ee.ImageCollection(
      ee.List(dates.distinct()).map(function (n) {
      var date = ee.Date(n)
      var filtered = data.filterDate(date, date.advance(cCoverage_Days,'day')) 
      var image = ee.Image(filtered.mosaic())
      var data1 = ee.Image(1).mask(image.select('B1'))
      var raster_area = data1.multiply(ee.Image.pixelArea()).reduceRegion({
        reducer: ee.Reducer.sum(),
        geometry: aoi,
        scale: sscale,
        maxPixels: 1E13
      }).get('constant')
      raster_area=ee.Number(raster_area).divide(1e6)
      return image.set({'date':n,'Area_KM2':raster_area})
    }));
    
    // Filter Images that doesn't cover 99% of polygon area
    var data2 = dd.filterMetadata("Area_KM2","greater_than",ee.Number(minimum_area));
    var ii = ee.Number(ee.Algorithms.If(ee.Number(data2.size()).gt(0),ee.Number(i),ee.Number(0)))
    return ii //return the number of image collections that covers 99% of specified aoi (area of interest)
}
}
}
4
  • Your example either shows a time out when running or throws an error: 'String (Error) List.get, argument 'list': Invalid type. Expected type: List<Object>. Actual type: Integer. Actual value: 999'
    – Jobbo90
    Commented Sep 22, 2021 at 9:00
  • True, thanks for notifying me about this error. I've fixed it and edited my code above. Kindly, check it again and feedback. However, I don't recommend using it on a large scale AOI, otherwise, a timeout error will occur. Commented Sep 23, 2021 at 11:23
  • I have a question about your interesting idea. If n1 = 999, then it is added to previous. In the next iteration, 999 is extracted from previous and is added to previous again. Does it mean that we have to complete iterations for every element in the List sequence, which does not really stop the loop? Commented Oct 27, 2021 at 7:24
  • True, Generally, on server side you cannot stop an iteration (or looping); but you can divert each iteration to a mini (non-exhaustive) loop which will make the parent loop move towards the end faster. Commented Oct 28, 2021 at 14:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.