This is a generic inquiry on the common practice/standard for determining the acceptable values of RMS when georeferencing topographic maps. Is there an absolute value?

Some literature suggests that it should be "less than or equal to 1/2 of the side of a cell which make up the total resolution of the image."

3 Answers 3


No, there is no absolute value for RMS, because it depends on the quality of the map being georeferenced, the quality of the target (base) map, and the purpose of the georeferencing. In particular, any advice that relates RMS to cellsize is misinformed, because cellsize reflects precision in the digital representation of an image whereas the RMS error reflects average accuracy (assuming the basemap is perfectly accurate). Although distinguishing precision and accuracy may seem like aimless pedantry, confusing them is a basic mistake with practical consequences.

All this is rather vague, so let's look at a specific example. Recently I received a series of screenshots of maps showing soil sample locations. To obtain coordinates, I planned to georeference these screenshots to an orthophoto base map and then digitize the points with heads-up digitization. Among the considerations were:

  1. The orthophoto base map has 0.3 m cellsize.
  2. The screenshots have approximately 2 m cellsize.
  3. The soil sample locations were not surveyed; they were located "by eye" on the map when the sampler was in the field. The client estimated the accuracy was about 3 m, but 10 m is more likely.
  4. The screenshots have few sharp details: they are primarily contour lines, with occasional fencelines (which are not clearly visible in the orthophoto). Thus establishing many good links would be time-consuming and error-prone.
  5. There was likely some local distortion in the screenshots, meaning that high accuracy (low RMS) can be achieved only with complex transformations.
  6. It was important to digitize the soil sample locations so that relative distances were fairly accurate for nearby points, but absolute accuracy was unnecessary, because one outcome of the study will be to obtain many more soil samples that refine and make more precise this preliminary survey.

To obtain an RMS of half the larger cellsize would require a high-order polynomial transformation or warping across a grid of points, calling for establishing a network of around 50 - 100 good links between the images: one to several hours of careful work, most likely, given the difficulty of even finding visible links. To obtain an RMS of half the smaller cellsize would require an order of magnitude more effort: days of work. However, for the purposes of the study an RMS of 5 m would be more than sufficient. This was achieved with 7 links and an affine transformation, just a few minutes' work. Note that this RMS is several times greater than the larger of the two cellsizes in the images.

This example illustrates how blindly following bad rules of thumb can be costly. Pay attention first to your data quality objectives; everything else follows from them.

  • 1
    Absolutely. There is no substitute for domain-specific knowledge and intuition. It never ceases to amaze me as a programmer that what should be a problem ideally suited to modern GIS, is still very much an ucomputable art. The skills really haven't changed in the past 100 years when some bright spark thought about bolting a camera to an aeroplane. That's not to belittle maning or his question in any way, but there is a reason air photo interpreters from both world wars were so highly trained and why those skills are as relevant today at they were then. Commented Apr 25, 2011 at 19:08
  • 1
    Excellent answer whuber! I pretty much follow what you are saying and the specific example you cited pretty much nailed it. @MerseyViking, I do think the question is very relevant since many colleagues fall into this rule of thumb trap.
    – maning
    Commented Apr 26, 2011 at 2:12
  • very good answer, but I disagree on the fact that RMSE reflects the pure accuracy. It indeed incorporates both the variance (precision) of the estimator and its bias (accuracy).
    – radouxju
    Commented Feb 11, 2016 at 11:08
  • @radouxju I agree: thank you for the clarification.
    – whuber
    Commented Feb 11, 2016 at 17:01

The nearst book I have to hand: Geographical Information Systems in Archaeology says it "depends on the scale of the maps and the purpose to which they are being put", but recommends to aim for an error less than 1:3000, so if the original map was 1:15000, then an RMSE of 5m or less would be acceptable. Certainly anything less that 1/2 a pixel would be largely redundant, but would be nice to have.

If you're combining more than one map, then the final RMSE will be the square root of the sum of the individual RMSEs, so if one high resolution map isn't behaving, but a lower res one is, then it may not be worth spending time getting the first one to fit any better.

  • +1 Good points. The justification for 1:3000 must be rooted in the needs of typical archaeological studies. That rule won't translate to unrelated fields.
    – whuber
    Commented Apr 25, 2011 at 15:30

Your question has the answer that I have always gone by.

less than or equal to 1/2 of the side of a cell which make up the total resolution of the image

This is a rule of thumb. In real life, sometimes I have had to be less accurate than this for various reasons:

  • Not feasible to reach these levels, with the number of tiles you have to rectify.
  • Project does not require the accuracy stated by these rules (i.e. The use of them is for a strategic project and no measurements will be made off the output.
  • Been impossible to reach this due to inability to plot sufficient number of confident control points.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.