I have a set of raster area results, that represent a prediction for a specific loccation. I would like to measure accuracy of two methods, given the actual values, estimate-A method values and estimate-B method values.

At the moment, for each raster pixel, I'm getting the difference abs(ActualLayer.pixel.value - EstimateLayerA.pixel.value), and using the average difference for a given area as a measure of accuracy (closer to 0 would be better).

This is ok, but are there better methods to quantize the accuracy of estimates against the actual for a given area?

  • 1
    I hate to say it, but I think it depends on what you mean by better. What do you need your method to do? Do you care about a particular kind of outlier? E.g., it looks like you're using mean absolute deviation (MAD), which is sometimes used to dampen outliers. RMSE on the other hand would judge outliers more harshly. Is over or under-prediction more permissible? MAD and RMSE lose the sign, but there are measures that don't. Does noise level or systematic error matter? Is spatial dependence important, like the example here youtube.com/watch?v=SJLDlasDLEU. – user55937 Sep 12 '15 at 16:23
  • 2
    Sorry to double comment, but just to clarify, I think the appropriate error metric is really decided once you determine what kind of error is permissible and what kind of error is a deal breaker. Once you nail that down, then you can look up an error metric with the right properties that really measures what you're after. So what are trying to get at? – user55937 Sep 12 '15 at 16:41
  • 1
    A closely related question is answered at gis.stackexchange.com/questions/55507, where it is pointed out that accuracy can depend on positional errors as well as errors of estimation. I strongly agree with @user55937 that the choice of error metric depends on what you are trying to accomplish, as well as on the nature of the possible errors. – whuber Sep 12 '15 at 20:12
  • All, Thanks for the comments! I'll review the material and try to clarify. – monkut Sep 13 '15 at 0:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.