Geographic Information Systems Stack Exchange is a question and answer site for cartographers, geographers and GIS professionals. It only takes a minute to sign up.
Sign up to join this community
I would like to know if the mean error computed by QGIS georeferencer is referring to the RMSE (root-mean-square error). I calculated myself with the residual pixels X and Y but I don't have the same results. I find 4.48 with my calculation and QGIS gives me 5.79.
Looking at the source code for qgsgeorefmainwindow.cpp, function QgsGeoreferencerMainWindow::calculateMeanError in particular, the calculated "mean error" is the sum of the squares of the residuals divided by the number of points minus the minimum points required to specify the transform, i.e. adjusted for the degrees of freedom.
I haven't redone your manual calculation point by point, but a poly 2 transformation is specified by a minimum 6 points, and 4.48^2*15 approximately equals 5.79^2*(15-6), i.e. your sum of squared residuals is about 301 and the difference is the denominator.
The rationale here is that the total (squared) error should be considered not as spread over all (enabled) georeferencing points, but only over the excess points that overspecify the distribution and contribute to "error". There is of course no actual differentiation between "first" points that define the transform and "excess" points that have errors, but to have the RMSE scale properly as you add or subtract points, or be otherwise comparable between different data fitting, you need to make the adjustment.
