0

I have an Earth Engine image collection. Each image has a property that is a dictionary with multiple keys. I want to know if there are any null values in the dictionary.

My idea (code below) is to map a function over the image collection. The function gets the dictionary values and uses JavaScript indexOf function to determine if there are null values in the list. I set a new property on the image that is the true/false result.

The problem is that the indexOf function requires that the data be client-side (using getInfo) and I cannot use client-side values in a mapped function. I get the error:

A mapped function's arguments cannot be used in client-side operations

Is there a server-side way to determine whether an ee.Image dictionary property has nulls using a mapped function?

// A mocked up image collection.
var col = ee.ImageCollection.fromImages([
  ee.Image(0).set('dict', {'key1': 1000, 'key2': 200}),
  ee.Image(1).set('dict', {'key1': null, 'key2': 300})
]);

// A function constructor that returns a function to set a new image property
// that indicates whether an image property dictionary has null values. Provide
// the target dictionary property name to the `dictPropName` parameter.
function getIsNullFun(dictPropName) {
  return function(img) {
    // Get the dictionary values.
    var vals = ee.Dictionary(img.get(dictPropName)).values().getInfo();
    
    // Are there nulls?
    var hasNulls = vals.indexOf(null) !== -1;
    
    // Set an image property the indicates presence of nulls in dictionary.
    return img.set('hasNulls', hasNulls);
  };
}

// Construct the function - define what the dictionary property name is.
var isNullFun = getIsNullFun('dict');

// Map the function over the collection to set the `hasNulls` property per image.
col = col.map(isNullFun);
print('Collection w/ bool prop for "dict has null"', col);



// Do other things with the collection now that you know what images have nulls.
var notNullCol = col.filter(ee.Filter.eq('hasNulls', false));
print('Only images WITHOUT null dict values', notNullCol);

var nullCol = col.filter(ee.Filter.eq('hasNulls', true));
print('Only images WITH null dict values', nullCol);
2
  • Surprisingly, vals.indexOfSublist([null]) appears to work Sep 7, 2022 at 0:34
  • This ^ should be the accepted answer. Would you mind adding it and I'll accept it. Sep 7, 2022 at 16:04

3 Answers 3

3

Adding one more option here. This one is super simple and short. It produces the number of nulls with ee.Filter.notNull(['item']), not(), and size().

// A mocked up image with a dictionary property.
var img = ee.Image(1).set('dict', {'key1': null, 'key2': 300, 'key3':null});

// Get a list of values from the dictionary property.
var values = ee.Dictionary(img.get('dict')).values();

// Print out the number of nulls values in the list.    
print(values.filter(ee.Filter.notNull(['item']).not()).size());
3

You can use the ee.List.filter(ee.Filter.notNull) function to get a list of dictionary values that are not null and compare the list length to the length of the comprehensive list of values. Then use the ee.Algorithms.If function to get true/false for whether the lists are equal or not (contains null or not) and then set that value as the new nulls indicator property for the image. You wouldn't have to use ee.Algorithms.If if you were satisfied with 0/1 as indicators (the if just gives more flexibility for what is returned).

// A mocked up image collection.
var col = ee.ImageCollection.fromImages([
  ee.Image(0).set('dict', {'key1': 1000, 'key2': 200}),
  ee.Image(1).set('dict', {'key1': null, 'key2': 300})
]);

// A function constructor that returns a function to set a new image property
// that indicates whether an image property dictionary has null values. Provide
// the target dictionary property name to the `dictPropName` parameter.
function getIsNullFun(dictPropName) {
  return function(img) {
    // Get the dictionary values.
    var vals = ee.Dictionary(img.get(dictPropName)).values();
    
    // Get a list of not null values.
    var notNullVals = vals.filter(ee.Filter.notNull(['item']));
    
    // Get the size of values and not null values lists.
    var nVals = vals.size();
    var nNotNullVals = notNullVals.size();
    
    // Compare the list sizes; if not equal, there are nulls.
    var hasNulls = ee.Algorithms.If(nVals.neq(nNotNullVals), true, false);
    
    // Set an image property the indicates presence of nulls in dictionary.
    return img.set('hasNulls', hasNulls);
  };
}

// Construct the function - define what the dictionary property name is.
var isNullFun = getIsNullFun('dict');

// Map the function over the collection to set the `hasNulls` property per image.
col = col.map(isNullFun);
print('Collection w/ bool prop based on dict', col);



// Do other things with the collection now that you know what images have nulls.
var notNullCol = col.filter(ee.Filter.eq('hasNulls', false));
print('Only images WITHOUT null dict values', notNullCol);

var nullCol = col.filter(ee.Filter.eq('hasNulls', true));
print('Only images WITH null dict values', nullCol);
3
  • The code above compares the length of two lists to determine if there are nulls or not. It could be simplified to use the ee.List.contains() function, except that the target value is null, which the function will not except–the object argument needs to be not null. Sep 6, 2022 at 22:10
  • ee.Algorithms.If is used here. Note that it evaluates all arguments (both true and false cases). This can be computationally intensive if your alternatives are large/complex requests. Here, the results are direct boolean assignment, so no concern needed. Sep 6, 2022 at 22:14
  • Super helpful thankyou! Sep 7, 2022 at 11:09
3

Here's an alternative to Justin Braaten's answer which uses ee.Algorithms.IsEqual to find null values.


// A mocked up image collection.
var col = ee.ImageCollection.fromImages([
  ee.Image(0).set('dict', {'key1': 1000, 'key2': 200}),
  ee.Image(1).set('dict', {'key1': null, 'key2': 300})
]);


// A function constructor that returns a function to set a new image property
// that indicates whether an image property dictionary has null values. Provide
// the target dictionary property name to the `dictPropName` parameter.
function getIsNullFun(dictPropName) {
  return function(img) {
    // Get the dictionary values.
    var vals = ee.Dictionary(img.get(dictPropName)).values();
    
    var null_test = vals.map(function (value) {
      return ee.Algorithms.IsEqual(value, null);
    });

    var hasNulls = null_test.contains(true);
    
    // Set an image property the indicates presence of nulls in dictionary.
    return img.set('hasNulls', hasNulls);
  };
}

// Construct the function - define what the dictionary property name is.
var isNullFun = getIsNullFun('dict');

// Map the function over the collection to set the `hasNulls` property per image.
col = col.map(isNullFun);
print('Collection w/ bool prop for "dict has null"', col);



// Do other things with the collection now that you know what images have nulls.
var notNullCol = col.filter(ee.Filter.eq('hasNulls', false));
print('Only images WITHOUT null dict values', notNullCol);

var nullCol = col.filter(ee.Filter.eq('hasNulls', true));
print('Only images WITH null dict values', nullCol);
3
  • My answer does not need to know dictionary key names, it just gets a list of all values in the dictionary (var vals = ee.Dictionary(img.get(dictPropName)).values();) Sep 6, 2022 at 23:37
  • Whoops, missed that Sep 6, 2022 at 23:46
  • No worries, thanks for editing. Fun to see all the different ways to do this! Sep 7, 2022 at 0:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.