Dealing with NAN values in my training dataset

Question

I am new to remote sensing. I am working on a project where I need to perform land use land cover classification. For such a task I want to use machine learning (python). My features contain 'BANDS' (6 bands), 'NDVI', 'NDWI', and 'NORM' extracted from the sentinel level 1C images. I did cloud masking and then interpolation (to fill the gaps) and finally, I sampled the data and split them into training and testing datasets. I have also used a mask to eliminate samples where I have '0' which essentially means 'no-data':

# remove points with no reference from training (so we dont train to recognize "no data")
mask_train     = labels_train == 0
features_train = features_train[~mask_train]
labels_train   = labels_train[~mask_train]

# remove points with no reference from test (so we dont validate on "no data", which doesn't make sense)
mask_test     = labels_test == 0
features_test = features_test[~mask_test]
labels_test   = labels_test[~mask_test]

However, to be extra sure, I checked for the NAN values using: np.isnan(train_test).any() (for both training and testing datasets) I found that my features had NAN values. Any idea how to solve this? and why do I have NAN values in the first place?

Edit: This is what my dataset looks like ( a lot of NAN values):

Answer 1

Nodata and Nan are not the same thing. A Nodata value is a value that is assigned to certain pixels in the case there was no observation possible (for example because of cloud cover). This may be done on purpose by you or by the creator of the data that you work with.

A Nan can arise for many reasons, but most often it happens because an algorithm tries to do something on a dataset which is not possible. For example:

import numpy as np
a = np.arange(9)
b = a / 0
print (b) # will be nans or infinate (inf)

Coming back to your use case, when you calculate NDVI you might encounter that nir+red is 0. This will result in a Nan because you will divide by 0.

You can detect and deal with Nans (and Infinite) values as follows:

import numpy as np
band1 = np.random.random(10)
nodatavalue = -999
# assign values in the array to nodata, nan and inf
band1[1] = nodata
band1[2] = np.nan
band1[3] = np.inf
mask = np.where(np.logical_or(band1 == nodata, np.logical_not(np.isfinite(band1))), 1, 0)
band1_out = np.where(mask == 1, nodata, band1)
print(band1_out)