I would like to know if the mean error computed by QGIS georeferencer is referring to the RMSE (root-mean-square error). I calculated myself with the residual pixels X and Y but I don't have the same results. I find 4.48 with my calculation and QGIS gives me 5.79.
It depends on the quality of the data source and what your referencing data to. gis.stackexchange.com/questions/8900/…– Mapperz ♦May 21, 2021 at 16:07
Looking at the source code for qgsgeorefmainwindow.cpp, function
QgsGeoreferencerMainWindow::calculateMeanError in particular, the calculated "mean error" is the sum of the squares of the residuals divided by the number of points minus the minimum points required to specify the transform, i.e. adjusted for the degrees of freedom.
I haven't redone your manual calculation point by point, but a poly 2 transformation is specified by a minimum 6 points, and
4.48^2*15 approximately equals
5.79^2*(15-6), i.e. your sum of squared residuals is about 301 and the difference is the denominator.
The rationale here is that the total (squared) error should be considered not as spread over all (enabled) georeferencing points, but only over the excess points that overspecify the distribution and contribute to "error". There is of course no actual differentiation between "first" points that define the transform and "excess" points that have errors, but to have the RMSE scale properly as you add or subtract points, or be otherwise comparable between different data fitting, you need to make the adjustment.