My question is similar to exporting table in to a drive from Google earth engine returns blank rows, but instead of getting entirely no data, only a few records come back NA where I have confirmed there is valid data (~20 NAs out of >2500 records).

Briefly, I am calculating the median of moving window and subtracting that value from each Landsat pixel. The PV is photosynthetic vegetation from a spectral unmixing.

Here are 2 screen grabs of the CSV and the console. Note the matching system:index IDs (I have triple checked the dates are the same, etc.)

enter image description here //

enter image description here

Here's a snippet of the code used to calculate the normalization and extract the data:

// Adjustment kernel function
var kernel_radius5 = 150
var PV_kernel5 = function(image,kernel_radius5) {
  return (image.select("PV").subtract(image.select("PV").focal_median(kernel_radius5,"circle","meters"))); 
var PV_normalized5 = PVdecimal.map(function(image){return PV_kernel5(image,kernel_radius5)});

// extract values from tree locations
var PVcompute_band5 = function(bandname) {
  var mapfunc = function(feat) {
    var geom = feat.geometry();
    var addProp = function(IMG, f) {
      var newf = ee.Feature(f);
      var index = ee.String(IMG.get('system:index'));
      var date = index.slice(0,20);
      var value = IMG.reduceRegion(ee.Reducer.mean(), geom, 30).get(bandname)
      return ee.Feature(ee.Algorithms.If(value,
                                         newf.set(date, ee.String(value)),
                                         newf.set(date, ee.String('NA'))));
    var newfeat = ee.Feature(PV_normalized5.iterate(addProp, feat));
    return newfeat;
  return treePoints.map(mapfunc);

// export to drive
var PVbands = "PV"
var PVtable5 = PVcompute_band5(PVbands)
  fileName+"_L8_median5", //job name
  "Ch3_GEE_exports", //folder name
  fileName+"_L8_median5"); //file name

Not sure if the problem is with the approach I've used to extract the data or in the spatial normalization function.

Has anyone else seen this behavior?

I've made quick link with a couple of the specific points exporting as NA https://code.earthengine.google.com/a5ae71cc6dd0816561b78c6a7acff614


As far as I can see, the values yo are referring to which should have correct data, are actually 0. Thus, the ee.Algorithm.If will say this is a false input and output the 'NA' in the falseCase of your if statement.

You could overcome this by first subtracting a unrealistic high value from all the images, and then add the same value in the trueCase of your if statement, so it will return the origonal value:

// the value could be 0, which is a valid input but algorithms.If will say it is not
      // therefore, we subtract 99 from the image
      var value = IMG.subtract(99).reduceRegion(ee.Reducer.mean(), geom, 30).get(bandname);
      // and here we will add 99 to the value outcome
      return ee.Feature(ee.Algorithms.If(value,
                                         newf.set(date, ee.Number(value).add(99)),
                                         newf.set(date, ee.String('NA'))));

I hope this solves your problem. Printing the feature collection in the console seems to work fine. Link script

  • Cheers @Kuik for that explanation. I guess the thing I don't understand is the data aren't zero at that location. The chance of the calculation equaling exactly zero is near impossible. If you look back to my original link script (I add that particular scene to the console) and use the inspector over those points the values are definitely non-zero. Is there some other problem with my export script that you can see, from your experience? Now I'm worried my data wrangling is wrong here. – ThrushJacket Mar 30 '19 at 14:19
  • It is indeed very strange an unmixing will produce exactly zero as outcome. I am not sure why that is. Maybe this can help you identify the images where 0, nulls and valid data is found: code.earthengine.google.com/ae05ad442bd7ef90c91b66971105b4a9 . It uses reduceRegions instead of reduceRegion, which I guess is a bit faster. – Kuik Mar 30 '19 at 16:16
  • Thanks again for looking at this. I've confirmed at least 2 of the 3 have REALLY close to zero values (-5e-16 and 4e-14) and GEE must have a significant digits rounding at some threshold. I've accepted you're answer assuming that was the issue. I'll follow up with a comment if I find anything else. Thank you again – ThrushJacket Mar 30 '19 at 22:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.