I am currently trying to compare two shapefiles (representing buildings) in order to validate simulation results. One shapefile contains real data and the other is the product of an urban simulation model.

The thing is that in most of the papers I found, researchers are using raster data (i.e. http://onlinelibrary.wiley.com/doi/10.1111/tgis.12047/full ) and apply grid-oriented indexes like the Kappa statistics. The only paper I found that compares two vector maps is the following one : http://link.springer.com/chapter/10.1007/978-3-319-46840-2_10 In this paper, "the error is computed as the rate of new real buildings that are not intersected (with a given buffer) by buildings obtained by simulation, weighted by their area" (Taillandier et al., 2016). This is a good start but I'd rather use more than one statistic.

Thus, I was wondering if anyone had any idea of how to compare two vector maps, both containing one feature class (buildings), in order to express/measure the agreement between the two maps.

I mainly work with R.


The Kappa statistic is not limited to "grid-oriented" data. It is, in essence, a comparison between two vectors representing observed verses predicted. If you distill the statistic, it is a chance-corrected percent correctly classified.

There are several implementations of the Kappa available in R, all you need to do is figure out how you are going to represent the observed and predicted. You could use the "accuracy" function in the rfUtilities package or the "confusionMatrix" function in caret. Both functions return the Kappa along with many other robust validation statistics. For multi-class validation the confusionMatrix function returns sensitivity, specificity and prevalence for each class along with the 95th percent confidence intervals. For binomial validation the accuracy function returns the AUC/ROC.

Here is a dummy example where you can see that the input data are just vectors of the classes for observed and predicted.

( obs <- iris$Species )
( pred <- sample(iris$Species) )  
caret::confusionMatrix(obs, pred)
rfUtilities::accuracy(obs, pred)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.