2

I have a set of pre-calculated masks as image assets in a Google Earth Engine folder. I want to map over an image collection of Sentinel images and set each image's mask based on the matching pre-calculated asset. The precalculated mask assets are named after the Sentinel id to which they apply.

There is a similar question to this here, but my issue is that I wish to do this from inside a map function and that means that "getInfo()" will not work as you cannot use clientside operations inside a map function (see the docs).

My failing code is as follows:

var add_mask = function(image) {
  var path = ee.String("users/my_account/my_masks/");
  var mask = ee.Image(path.cat(image.id()));  //can't use getInfo() here
  return image.updateMask(mask.not());  // these masks have '1' for invalid
}

mySentinel2Collection = mySentinel2Collection.map(cloudMask);

The error I get is:

ImageCollection (Error)
Actual value for parameter 'id' must be a constant.

I have tried casting the image id as a string:

var mask = ee.Image(path.cat(ee.String(image.id())));

But this doesn't work and I get the same error.

I have also tried using combine() to join to image collections, but of course the ID don't match even though the masks use the Sentinel ID because the path to the stored image collection is part of the id.

EDIT: Actually combine() does seem to work after all - I don't know why it didn't look like it at first. Also I don't understand why it works because the ids are demonstrably different except for the final part (after the last "/"). The documentation is opaque as to what it is actually testing here so I don't have 100% confidence - however, see my solution below.

  • I'm curious what gets returned from image.id(). Try debugging outside of the mapped function. Can you run: print(mySentinelCol.first().id()) - does the result seem reasonable. Can you construct a valid path to a mask asset from it when concatenated with ee.String("users/my_account/my_masks/"). – Justin Braaten Nov 21 '19 at 22:06
  • also, does it help at all to cast the image as an ee.Image and use ee.Algorithms.String() to cast the id to string like: ee.Algorithms.String(ee.Image(image).id()) – Justin Braaten Nov 21 '19 at 22:07
  • @jdbcode - thanks for the comments. I can't do print(mySentinelCol.first().id()) inside the map function as this returns a variable not initialised error (because it mixes clientside and serverside methods). I can do it outisde and it returns a string as does .getInfo(). Unfortunately ee.Algorithms.String() also gives the same error as in my post. I think the problem is not casting to a string but the fact that the id for ee.Image() may not be a variable (at least not inside a map function). This seems like an unnecessary restriction in the API but I can't hink of a work around. – MappaGnosis Nov 22 '19 at 9:13
0

I found a solution, possibly two solutions:

  1. Use combine() - this appears to work and when the documentation says it matches on 'id' it apparently only matches on the final part of the 'id' after the last "/". The ids appear to be different when you do print(image) because they include the path. However, I am able to now confirm that the id's in my case do match using print(image.id().getInfo()) (outside of a map function obviously). This would have been so much simpler if the documention was a bit more explicit about what it means by id - and/or if the reporting of 'id' was consistent between print functions!

    var myMasks = ee.ImageCollection("users/my_account/myMasks")
    mySentinel2Collection = mySentinel2Collection.combine(myMasks);
    
    var applyMask = function(img) {
      return img.updateMask(img.select("the_mask_band").not()); //flip the mask so 0 is invalid
    } 
    var myMaskedCollection = mySentinel2Collection.map(applyMask);
    
  2. Do an inner join(see the documentated example). This is a bit like what combine() does (expect it only keeps matches) plus adding the secondary image as bands to the primary. It was the documentation for combine() that gave me the idea but this method would alllow for any property (ie not only 'id') to be used:

    var joinFilter = ee.Filter.equals({
      leftField: 'system:index',
      rightField: 'system:index'
    });
    var innerJoin = ee.Join.inner('primary', 'secondary');
    var myJoin = innerJoin.apply(mySentinel2Collection, myMasks, joinFilter);
    var allMasked = myJoin.map(function(feature){
      var mask = ee.Image(feature.get('secondary')).not();  //flip the mask so 0 is invalid
      return ee.Image(feature.get('primary')).mask(mask); 
    })
    

    The output of the above is an ImageCollection.

Using the combine() function is a shade shorter/easier but it still needs (in my case) a map function to apply the mask. It's all kinda obvious when you stumble on the right bit of the API! I really need to start thinking of collections more like database tables or sets.

|improve this answer|||||
  • I'm glad you found solutions. I particularly like option 1 combine(), however, I don't see its implementation in the code snippet. Would you mind editing your answer to include it. – Justin Braaten Nov 22 '19 at 16:09
  • Oops! Cut 'n' paste error - I have added it above now. Note that if combine() can't find a matching id, it copies the image from the first collection (like an outer left join). – MappaGnosis Nov 25 '19 at 9:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.