Is there a basic or introductory study that examines and compares the precision of spatial data when working

  • with varying precison of the data input, like having 1, 2, ... decimal places?
  • with the varying implementations of floating points (float, double)?
  • with data near the equator in comparison to data near the poles?
  • with geographical distances computed with tunnel distance, great circle distance, vincenty, bowring, lambert?

All work I found so far stated that these are sources of error but don't give exact error boundaries one can expect.

  • 6
    This is a huge subject. To get a sense of it, peruse the table of contents for Accuracy 2000 (Heuvelink & Lemmens, Eds). (You may need a library for this: I cannot find any online previews.) BTW, I offer this only as a comment because for many this volume would not be considered either basic or introductory--but it is elementary.
    – whuber
    Apr 19, 2012 at 14:49
  • Thanks. A table of contents can found here. From the titles I could not discern any comparative (or introductory) studies. From what I'm reading so far I'm missing the link between the real form of the earth to the various representations and how accuracy is then measured. For example how can it be said that Vincenty is accurate to about 0.5mm?
    – oschrenk
    Apr 19, 2012 at 15:52
  • 1
    Generally 6 decimal places is acceptable - good overall accuracy whilst keeping databases (storage) reasonably small.
    – Mapperz
    Apr 19, 2012 at 16:46

2 Answers 2


Data quality standards can and frequently do vary by project. In my experience, data quality standards are set out in the project requirements, whatever they may be (government, municipal, etc.) We typically use the SDSFIE and FGDC standards which are government standards.

  • That your precision should be based on the (project's) needs is understood. I'm evaluating the different models/algorithms to get a grasp of them.
    – oschrenk
    Apr 24, 2012 at 12:38
  • Sorry. Our requirements are sometimes so rigid, the "why" and "how" of the accuracy gets a little lost.
    – bspencer
    Apr 24, 2012 at 14:54

The first two parts of your question aren't really geospatial specific, and you would need to determine how errors propagate throught the particular calculations you are performing. For example, if you are calculating distance between two points, then your error will be in units of the distance (sum), but an area will give you units of distance^2 (multiplicative effect). Any real calculations is going to have a far more complex error dependency.

I don't think number of decimal places (alone) is important - consider UTM vs lat/lon degrees - two decimal places has completely different effect.

I'd also caution that projections aren't anything like "true" - they are (at best) reasonable approximations to reality. https://www.spacecomm.nasa.gov/spacecomm/programs/system_planning/pnt/geodesy/reqts.cfm claims "the accuracy of both the International Terrestrial Reference Frame (ITRF) and the World Geodetic System 1984 (WGS 84) is estimated to be on the order of 1 to 2 parts per billion, leading to a degradation in positioning of 0.6 to 1.2 cm per year on the Earth’s surface and higher at altitude".

Reference system accuracy is also a function of time. http://www.dse.vic.gov.au/property-titles-and-maps/geodesy/geocentric-datum-of-australia-gda points out that GDA94 was once reasonably aligned to WGS84 (and ITRF), but Australia moved about a metre since then. See http://www.quickclose.com.au/stanawayssc2007.pdf for more detail on this example.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.