Evaluating classification accuracy and the result of kappa test in Google Earth Engine

Question

I am trying to check the efficiency of the classification. The classification accuracy and kappa test for 2020 are 0 while for 2000 is 1. I am just not able to put my finger on what is incorrect as the accuracy of 1 also seems far-fetched. This is the script used for 2000 where the result of the accuracy-test was 1.

//import the images for 2000 
var landsat_image= ee.ImageCollection("LANDSAT/LE07/C01/T1_TOA")
.filterDate('2000-01-01', '2000-12-30')
.filterBounds(watershed)
.sort('CLOUD_COVER')
.first()
//clip by asset (ROI)
var watershed_2000=landsat_image.clip(watershed)
//display the clipped region with visual parameters
Map.addLayer(watershed_2000, {bands: ['B3', 'B2', 'B1'], min:0, max: 0.5, gamma: 
1.4},'Watershed_2000')
//calculate cloud score 
var cloudScore_2000=ee.Algorithms.Landsat.simpleCloudScore(watershed_2000).select('cloud')

//mask input for clouds
var landsat_2000= 
watershed_2000.updateMask(watershed_2000.mask().reduce('min').and(cloudScore_2000.lte(40)))

//merge the landcover features 
var Landcover_2000=Vegetation.merge(builtuparea).merge(CropLand).merge(BarrenLand).merge(Waterbody).merge(Glacier)

//name of the variable
var classProperty_2000 = 'Landcover'

//bands 
var bands_2000= ['B2', 'B3', 'B4', 'B5', 'B7', 'B8']

//now train the classification points
//sample the input imagery to get a feature collection of the training points 
var training_2000= landsat_2000.select(bands_2000).sampleRegions({
  collection: Landcover_2000,
  properties: [classProperty_2000],
  scale: 30
 })

//train the classifier
var classifier_2000=ee.Classifier.smileCart().train({
  features: training_2000,
  classProperty: classProperty_2000, 
})

//classify the input imagery
var classified_2000= landsat_2000.classify(classifier_2000)

//confusion matrix about the resubstitution accuracy 
var trainAccuracy_2000 = classifier_2000.confusionMatrix()
print('2000 Resubstitution error matrix: ', trainAccuracy_2000)
print('2000 Training overall accuracy: ', trainAccuracy_2000.accuracy())
print('2000 Training Kappa index:', trainAccuracy_2000.kappa())

//the accuracy test
var withRandom_2000= training_2000.randomColumn('random')

//We want to reserve some of the data for testing, to avoid overfitting the model 
//Roughly 70% training, 30% testing.
var split = 0.7 
var trained_2000 = withRandom_2000.filter(ee.Filter.lt('random', split))
print ('Training size 2000:', trained_2000.size())
var validated_2000 = withRandom_2000.filter(ee.Filter.gte('random', split))

//classify the test FeatureCollection.
var validation_2000 = validated_2000.classify(classifier_2000)

// Get a confusion matrix representing expected accuracy.
var testAccuracy_2000 = validation_2000.errorMatrix(classProperty_2000, 'classification')
print('2000 Validation error matrix:', testAccuracy_2000)
print('2000 Validation overall accuracy:', testAccuracy_2000.accuracy())

Answer 1

Indeed as @radouxju mentions in their comment, you need to re-order your workflow a little. The idea is to:

• split your sample into training and validation sets
• train your model using only the training sample
• classify your validation set using the trained model
• examine the accuracy

// train the classification points
// sample the input imagery to get a feature collection of the training points 
var sample = landsat_2000.select(bands_2000).sampleRegions({
  collection: Landcover_2000,
  properties: [classProperty_2000],
  scale: 30
 })

// add a random column
var sample = sample.randomColumn('random')


//We want to reserve some of the data for testing, to avoid overfitting the model 
//Roughly 70% training, 30% testing.

var split = 0.7 

// split your **full** sample into training and validation points to keep them independent of each other
var training = sample.filter(ee.Filter.lt('random', split))
var validation = sample.filter(ee.Filter.gte('random', split))

//train the classifier using the **training sample**
var classifier = ee.Classifier.smileCart().train({
  features: training,
  classProperty: classProperty_2000, 
})

// classify the **validation sample**  (you can classify your training set too if you wish to see how the model performs with your training points)
var validation = validated.classify(classifier)

// Get a confusion matrix representing expected accuracy of **test data (validation sample)** 
var testAccuracy = validation.errorMatrix(classProperty_2000, 'classification')