Handling null images using map function with Google Earth Engine Python API

Question

I have a cloud masked Landsat 7 image collection. For each image I want to unmask the bands with their respective mean value.

This works fine, unless an image is completely cloud masked and therefore contains no data.

I tried adapting this related answer (in JavaScript) to my needs

start_date =  "2018-01-01"
end_date =  "2021-12-31"
l7 = (ee.ImageCollection("LANDSAT/LE07/C02/T1_L2")
        .filterDate(ee.Date(start_date), ee.Date(end_date))
        .filterBounds(aoi)
        .map(lambda image: image.clip(aoi))
    )

band_names = ['SR_B1', 'SR_B2', 'SR_B3', 'SR_B4', 'SR_B5', 'SR_B7', 'QA_PIXEL']

## fn.cloudmask_LS is a wrapper for a cloud masking procedure similar to this: https://gis.stackexchange.com/q/405056/128665
l7 = l7.select(band_names).map(fn.cloudmask_LS('QA_PIXEL'))

## drop QA_PIXEL band
band_names = [b for b in band_names if not b == 'QA_PIXEL']
l7 = l7.select(band_names)

################################################
## function adapted from the other GIS SE answer
################################################
def mask_with_mean(image):
    time = image.get("system:time_start")
 
    meanDict = image.reduceRegion(reducer=ee.Reducer.mean(), geometry=aoi)
    
    outImg = ee.Image(ee.Algorithms.If(
        meanDict.values(), 
        image.unmask(ee.Image.constant(meanDict.values(band_names))),
        ee.Image(0).selfMask().set('isNull', True)
    ))
    
    return outImg.set({"system:time_start":time})

## True is an attempt to get the map function to drop nulls but it doesn't appear to be working.
l7 = l7.map(mask_with_mean, True)

But I receive the following error:

ee.ee_exception.EEException: Error in map(ID=LE07_169068_20180128): Image.constant: Invalid type for constant image value.
Expected type: Number or Array.
Actual type: Object.
Actual value: null

I have tried meanDict in the ee.Algorithm.if true clause also.

I believe the problem arises because meanDict is null for an empty image but I don't know how to catch that with the ee.Algorithm.if.

Answer 1

I guess your issue is, that any of the values in meanDict might return null while meanDict itself is always defined.

So what you could do is just remove all entries from the dict with value null and use this to unmask. This should just keep all masked values for all bands where no mean is available masked. I don't know if having the data like this is helpful for you but like this the algorithm should definitely run through:

def mask_with_mean(image):
    time = image.get("system:time_start")
 
    meanDict = image.reduceRegion(reducer=ee.Reducer.mean(), geometry=aoi).map(lambda k,v: v)
    
    outImg = image.unmask(ee.Image.constant(meanDict.values(band_names)))
    
    return outImg.set({"system:time_start":time})

If this isn't what you are looking for you could also do more complicated stuff in the map call to the dict, like checking for null and then setting another default value.

Answer 2

You should basically never use ee.Algorithms.If. If that's the tool you're reaching for, you should rethink what you're doing.

In this case, separate the reduceRegion and the unmask into separate functions, storing the result of the reduceRegion on each image. Then filter for images where the result is not null.