Geographic Information Systems Stack Exchange is a question and answer site for cartographers, geographers and GIS professionals. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am trying to understand why a floating point is inaccurate. Please bear in mind that I am not a mathematician.

What is exactly the difference between floating point and double integers? and why use one rather than the other?

Thanks for any "easy to understand" replies,


share|improve this question

closed as off topic by whuber May 10 '12 at 15:03

Questions on Geographic Information Systems Stack Exchange are expected to relate to geographic information systems within the scope defined by the community. Consider editing the question or leaving comments for improvement if you believe the question can be reworded to fit within the scope. Read more about reopening questions here.If this question can be reworded to fit the rules in the help center, please edit the question.

The difference between float4 (32-bit float) and float8 (64-bit float, or double precision) depends on the context of what it is storing and how it is to be used. For many applications, float4 is sufficient, but GIS has particular demands on extra storage precision.

Consider storing UTM coordinates to millimeter precision. You might have a Northing of 4833438.204 measured using fancy survey equipment. Stored as float4, the number truncates to 4833438, loosing the millimetres. Stored as float8, the extra digits are preserved (down to the picometre scale). The reason why storing UTM coordinates is challenging, requiring double precision storage, is that they are typically on the order of hundreds of thousands to millions in scale.

share|improve this answer
+1 for making this relevant to GIS – Scro May 10 '12 at 14:35

StackOverflow has also been asked this question (the more obvious place for it). While the question is specific to C#, the answers hold across the board:

What is the difference between Decimal, Float and Double in C#?

share|improve this answer
+1 for mentioning StackOverflow as more obvious place – atlefren May 10 '12 at 10:20

Wikipedia has a good discussion of this:

share|improve this answer

The best way to visualize how floating point numbers are inaccurate is to think in terms of base-10 "floating point" values. You can use up to ten consecutive nonzero digits, and place the decimal point arbitrarily. Examples:

1234567890     // nine digits, decimal point placed to multiply 123456789 with 10
2345678901     // ten digits
12345.6789     // nine digits, decimal point placed to divide by 10000.
0.000000123456789  // nine digits, divide by 10000000

The result of every operation must fit in the same schema:

+          0.000000123456789    // two legal inputs
= 1234567890.000000123456789    // give an unrepresentable output
= 1234567890                    // which loses the least significant digits
- 1234567890                    // subtract one of the numbers again
=          0                    // the digits are lost forever

Basically, you can represent light years and millimeters, but adding a millimeter to a light year still gives you a light year. If you need more precision, you'd use an integer that is large enough that it can represent a light year in millimeters.

share|improve this answer
While true as far as it goes, your example doesn't mention the gotcha I've seen burn people most often: That the standard computer implementation stores floating point values with a binary mantissa which has a different subset of values that can be represented exactly than if it was stored in base 10. You can't represent the base 10 value 0.1 as a non-repeating binary value; the resulting rounding means that 0.1 + 0.1 + ... + 0.1 (10 times) will not equal 1.0. – Dan Neely May 10 '12 at 14:14
Oh yeah, that too. :) – Simon Richter May 10 '12 at 17:04

Not the answer you're looking for? Browse other questions tagged or ask your own question.