3

I want to assess the accuracy of the land cover classes classified by supervised classification of Landsat images. As reference data I use the aerial photography.

I sampled the classified Landsat data and at the same validation point, I identified the land cover class from aerial photography. I.e., each of my verification points have two attributes: one from Landsat (landsat), second from the aerial photography (reference).

I want to calculate an error matrix (contingency table), and accuracy assessment (overall accuracy, user's producer's accuracy) from the set of verification points.

I found a package greenbrown and asbio to assess the accuracy of classification. My final results of user's and producer's accuracy are howver switched between two packages.

Please, which package correctly calculate the values for user and producer accuracy?

Reproductible example

library(asbio)

# create dummy data
landsat <- c(1, 1, 1, 2, 2, 2, 3, 4, 5, 5, 5, 1, 1, 1, 2, 2, 2, 3, 4, 5, 5, 3, 3, 2, 2)
reference <- c(1, 2, 1, 2, 2, 2, 3, 4, 2, 2, 5, 1, 2, 2, 2, 1, 2, 3, 4, 5, 5, 3, 3, 2, 2)

# calculate Kappa statistics
asbio::Kappa(landsat,reference)  # Kappa(class1, reference)

# check out Kappa results
$ttl_agreement
[1] 76

$user_accuracy
1     2     3     4     5 
75.0  58.3 100.0 100.0 100.0 

$producer_accuracy
1     2     3     4     5 
50.0  87.5 100.0 100.0  60.0 

$khat
[1] 68.1

$table
       reference
class1  1 2 3 4 5
      1 3 3 0 0 0
      2 1 7 0 0 0
      3 0 0 4 0 0
      4 0 0 0 2 0
      5 0 2 0 0 3

# ----------------------------------------------------------------------   
# make the same calculation with the greenbrown package
# ----------------------------------------------------------------------

library(greenbrown)  
library(strucchange)
library(raster)
library(Kendall)
library(plyr)
library(bfast)
library(zoo)

# calculate the contingency table
tab <- table(landsat, reference)   data

# let's see the tab
tab

           reference
landsat   1 2 3 4 5
        1 3 3 0 0 0
        2 1 7 0 0 0
        3 0 0 4 0 0
        4 0 0 0 2 0
        5 0 2 0 0 3

# calculate the accuracy assessement
greenbrown::AccuracyAssessment(tab)

                    1        2   3   4   5 Sum UserAccuracy
1                 3  3.00000   0   0   0   6         50.0
2                 1  7.00000   0   0   0   8         87.5
3                 0  0.00000   4   0   0   4        100.0
4                 0  0.00000   0   2   0   2        100.0
5                 0  2.00000   0   0   3   5         60.0
Sum               4 12.00000   4   2   3  25           NA
ProducerAccuracy 75 58.33333 100 100 100  NA         76.0

The user and producer's accuracy are switched between two packages ! Please, which calculation and estimation of the user's and producer's accuracy is correct?

  • I come here through searching the calculation for User and Producer accuracy. The explanations are really good! However, I am not quite sure whether the confusion matrix was not correctly structured, i.e. transposed. Shouldn't the reference be put to the left (by rows) when using the greenbrown::AccuracyAssessment function? In that case, seems matching with the asbio. Please correct me if i misunderstood. – Yuk Mar 13 '18 at 15:20
  • This does not provide an answer to the question. Once you have sufficient reputation you will be able to comment on any post; instead, provide answers that don't require clarification from the asker. - From Review – Dan C Mar 13 '18 at 15:37
  • sorry for posting comments in this way, as you already noticed I could not comment on the answers:( – Yuk Mar 13 '18 at 16:13
2

The User's Accuracy is the reliability of the classes in the classified image. It is calculated as the fraction of correctly classified pixels with respect to all pixels classified as this class in the image. For instance, based on your sample data, the User Accuracy for:

UA_Class1 is : 3 / 3+3 = 50
UA_Class2 is : 7 / 7 + 1 = 87.5

Therefore, to answer your question, it would seem that the User and Producer's accuracies from the asbio package are switched.

For a thorough explanation of the Error (Confusion) Matrix and related calculations, I refer you to this excellent technical note by D. Rossiter. I have included an excerpt that formally defines the UA.

It is expressed from the point of view of the mapper. Looking across the rows (classes as mapped), an error of commission is said to occur when the mapper incorrectly mapped this class at a reference (ground truth) site where it does not exist. That is, the mapper ‘committed’ the error of over-mapping a class. This leads to a lower user’s ‘accuracy’Ci.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.