1

I am trying to check the efficiency of the classification. The classification accuracy and kappa test for 2020 are 0 while for 2000 is 1. I am just not able to put my finger on what is incorrect as the accuracy of 1 also seems far-fetched. This is the script used for 2000 where the result of the accuracy-test was 1.

//import the images for 2000 
var landsat_image= ee.ImageCollection("LANDSAT/LE07/C01/T1_TOA")
.filterDate('2000-01-01', '2000-12-30')
.filterBounds(watershed)
.sort('CLOUD_COVER')
.first()
//clip by asset (ROI)
var watershed_2000=landsat_image.clip(watershed)
//display the clipped region with visual parameters
Map.addLayer(watershed_2000, {bands: ['B3', 'B2', 'B1'], min:0, max: 0.5, gamma: 
1.4},'Watershed_2000')
//calculate cloud score 
var cloudScore_2000=ee.Algorithms.Landsat.simpleCloudScore(watershed_2000).select('cloud')

//mask input for clouds
var landsat_2000= 
watershed_2000.updateMask(watershed_2000.mask().reduce('min').and(cloudScore_2000.lte(40)))

//merge the landcover features 
var Landcover_2000=Vegetation.merge(builtuparea).merge(CropLand).merge(BarrenLand).merge(Waterbody).merge(Glacier)

//name of the variable
var classProperty_2000 = 'Landcover'

//bands 
var bands_2000= ['B2', 'B3', 'B4', 'B5', 'B7', 'B8']

//now train the classification points
//sample the input imagery to get a feature collection of the training points 
var training_2000= landsat_2000.select(bands_2000).sampleRegions({
  collection: Landcover_2000,
  properties: [classProperty_2000],
  scale: 30
 })

//train the classifier
var classifier_2000=ee.Classifier.smileCart().train({
  features: training_2000,
  classProperty: classProperty_2000, 
})

//classify the input imagery
var classified_2000= landsat_2000.classify(classifier_2000)

//confusion matrix about the resubstitution accuracy 
var trainAccuracy_2000 = classifier_2000.confusionMatrix()
print('2000 Resubstitution error matrix: ', trainAccuracy_2000)
print('2000 Training overall accuracy: ', trainAccuracy_2000.accuracy())
print('2000 Training Kappa index:', trainAccuracy_2000.kappa())

//the accuracy test
var withRandom_2000= training_2000.randomColumn('random')

//We want to reserve some of the data for testing, to avoid overfitting the model 
//Roughly 70% training, 30% testing.
var split = 0.7 
var trained_2000 = withRandom_2000.filter(ee.Filter.lt('random', split))
print ('Training size 2000:', trained_2000.size())
var validated_2000 = withRandom_2000.filter(ee.Filter.gte('random', split))

//classify the test FeatureCollection.
var validation_2000 = validated_2000.classify(classifier_2000)

// Get a confusion matrix representing expected accuracy.
var testAccuracy_2000 = validation_2000.errorMatrix(classProperty_2000, 'classification')
print('2000 Validation error matrix:', testAccuracy_2000)
print('2000 Validation overall accuracy:', testAccuracy_2000.accuracy())
1
  • 1
    I am not so familiar with GEE, but it seems that you used the full training set (training_2000) to train your classifier (see //train classifier) then you apply this model on a subset of your training dataset . The validation is thus done with a subset of the training with a classifier (classifier_2000) trained on the full training set, which increases the risks of overoptimistic results.
    – radouxju
    Jan 14 at 11:21

1 Answer 1

1

Indeed as @radouxju mentions in their comment, you need to re-order your workflow a little. The idea is to:

  • split your sample into training and validation sets
  • train your model using only the training sample
  • classify your validation set using the trained model
  • examine the accuracy

// train the classification points
// sample the input imagery to get a feature collection of the training points 
var sample = landsat_2000.select(bands_2000).sampleRegions({
  collection: Landcover_2000,
  properties: [classProperty_2000],
  scale: 30
 })

// add a random column
var sample = sample.randomColumn('random')


//We want to reserve some of the data for testing, to avoid overfitting the model 
//Roughly 70% training, 30% testing.

var split = 0.7 

// split your **full** sample into training and validation points to keep them independent of each other
var training = sample.filter(ee.Filter.lt('random', split))
var validation = sample.filter(ee.Filter.gte('random', split))

//train the classifier using the **training sample**
var classifier = ee.Classifier.smileCart().train({
  features: training,
  classProperty: classProperty_2000, 
})

// classify the **validation sample**  (you can classify your training set too if you wish to see how the model performs with your training points)
var validation = validated.classify(classifier)

// Get a confusion matrix representing expected accuracy of **test data (validation sample)** 
var testAccuracy = validation.errorMatrix(classProperty_2000, 'classification')

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.