Statistical validity percentage check on raster

Question

I would like to check if my set of NDVI images with min/max values from -1 +1 is good for at least 80%. I took my cue from this other topic: here. Considering that the nodata is -32768 But the values I get on multiple images are all the same.

with rasterio.open(NDVI[0], 'r') as src:
    meta = src.meta.copy()
    meta.update({"nodata":-32768})
    data = src.read(1)
    arr=data[data>-1.0]
pct_valid = 100 * (arr != meta['nodata']).sum() / (meta['width'] * meta['height'])

print('%.4f'%pct_valid)

OUTPUT:
-6.87526

How should I interpret it for my image?

To select only those images that are 80% good instead?

if pct_valid > 80.0:
    write(raster)

Answer 1

Here is an option that uses numpy to check if NoData values account for >20% of the total pixels.

The code is designed to be included in a for loop while iterating through a directory of images. If the routine detects an image that meets the 80% threshold, it returns a True, else False. Otherwise it would be easy to convert the function to populate a list of valid images.

import os
import rasterio
import numpy as np

img = '/path/to/geotiff.tif'
nodata_value = -32768 # enter the NoData value here


def check_image(img, nodata_value):
    # Read in image as a numpy array
    src = rasterio.open(img).read(1)
    
    # Count the occurance of NoData values in np array
    nodata_count = np.count_nonzero(array == nodata_value)
    
    # Get a total pixel count
    total_count = array.shape[0] * array.shape[1]
    
    # Get a % of NoData pixels
    val = nodata_count/total_count * 100
    
    # Check if image meets threshold of 80% valid pixels
    if val < 20:
        print("Image {0} has >= 80% of pixels that are valid".format(os.path.basename(img)))
        return True
    else:
        print("Image {0} has <80% of pixels that are valid".format(os.path.basename(img)))
        return False
    
if __name__ == "__main__":
    check_image(img, nodata_value)