If you want an accuracy of seven digits, should you then round to that value or simply just cut the numbers after that? Example:

51.1936137567 to 51.1936138 or 51.1936137567 to 51.1936137

  • 2
    Consider what the result will be of each operation. Cutting the numbers will result in a slight bias towards 0 lat 0 lon, while a rounding will not do this. However, this is to be weighted against the additional computational step in your processing. – Mikkel Lydholm Rasmussen Jan 6 '16 at 7:37

I guess that is up to you. I would round the number like I do with all numbers with too many digits, a GPS would cut the numbers. So it is a question of accuracy of the origin of your data and the accuracy you want to maintain.

  • The accuracy is supposed to be up to 2cm, so seven digits. But the GPS is still giving more digits. So you would just use them all? – Sanny Jan 6 '16 at 7:28
  • 3
    Well, I don't really see a good reason to loose accuracy. However, there are limitations to the accuracy of the GPS-device aswell, even though more digits are shown. And of course it doesn't make sence to use for example 20 digits. I think it is just important to choose one option and stay with it. And if you can't decide I guess it won't hurt to use 10 digits. Than you don't have to care to much about rounding if you need a accuracy of 7 digits. – Alex Jan 6 '16 at 7:37
  • Ok, I think that's a good way to go. Thx – Sanny Jan 6 '16 at 7:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.