Generally accepted root mean square (RMS) error for rectifying topographic maps

Question

This is a generic inquiry on the common practice/standard for determining the acceptable values of RMS when georeferencing topographic maps. Is there an absolute value?

Some literature suggests that it should be "less than or equal to 1/2 of the side of a cell which make up the total resolution of the image."

Answer 1

No, there is no absolute value for RMS, because it depends on the quality of the map being georeferenced, the quality of the target (base) map, and the purpose of the georeferencing. In particular, any advice that relates RMS to cellsize is misinformed, because cellsize reflects precision in the digital representation of an image whereas the RMS error reflects average accuracy (assuming the basemap is perfectly accurate). Although distinguishing precision and accuracy may seem like aimless pedantry, confusing them is a basic mistake with practical consequences.

All this is rather vague, so let's look at a specific example. Recently I received a series of screenshots of maps showing soil sample locations. To obtain coordinates, I planned to georeference these screenshots to an orthophoto base map and then digitize the points with heads-up digitization. Among the considerations were:

1. The orthophoto base map has 0.3 m cellsize.
2. The screenshots have approximately 2 m cellsize.
3. The soil sample locations were not surveyed; they were located "by eye" on the map when the sampler was in the field. The client estimated the accuracy was about 3 m, but 10 m is more likely.
4. The screenshots have few sharp details: they are primarily contour lines, with occasional fencelines (which are not clearly visible in the orthophoto). Thus establishing many good links would be time-consuming and error-prone.
5. There was likely some local distortion in the screenshots, meaning that high accuracy (low RMS) can be achieved only with complex transformations.
6. It was important to digitize the soil sample locations so that relative distances were fairly accurate for nearby points, but absolute accuracy was unnecessary, because one outcome of the study will be to obtain many more soil samples that refine and make more precise this preliminary survey.

To obtain an RMS of half the larger cellsize would require a high-order polynomial transformation or warping across a grid of points, calling for establishing a network of around 50 - 100 good links between the images: one to several hours of careful work, most likely, given the difficulty of even finding visible links. To obtain an RMS of half the smaller cellsize would require an order of magnitude more effort: days of work. However, for the purposes of the study an RMS of 5 m would be more than sufficient. This was achieved with 7 links and an affine transformation, just a few minutes' work. Note that this RMS is several times greater than the larger of the two cellsizes in the images.

This example illustrates how blindly following bad rules of thumb can be costly. Pay attention first to your data quality objectives; everything else follows from them.

Answer 2

The nearst book I have to hand: Geographical Information Systems in Archaeology says it "depends on the scale of the maps and the purpose to which they are being put", but recommends to aim for an error less than 1:3000, so if the original map was 1:15000, then an RMSE of 5m or less would be acceptable. Certainly anything less that 1/2 a pixel would be largely redundant, but would be nice to have.

If you're combining more than one map, then the final RMSE will be the square root of the sum of the individual RMSEs, so if one high resolution map isn't behaving, but a lower res one is, then it may not be worth spending time getting the first one to fit any better.

Answer 3

Your question has the answer that I have always gone by.

less than or equal to 1/2 of the side of a cell which make up the total resolution of the image

This is a rule of thumb. In real life, sometimes I have had to be less accurate than this for various reasons:

• Not feasible to reach these levels, with the number of tiles you have to rectify.
• Project does not require the accuracy stated by these rules (i.e. The use of them is for a strategic project and no measurements will be made off the output.
• Been impossible to reach this due to inability to plot sufficient number of confident control points.