3

I imported a point geometry layer into QGIS using a delimited text file (CSV) that contained the latitude and longitude values projected in CRS WGS84-EPSG:4326 with 6 decimal places (e.g. 38.145695). However, upon loading the layer, I noticed that QGIS increased the precision to 8 or 9 decimal places (which are not 0). While this doesn't cause any visual problems as it offsets input points by a small distance, it's causing issues with the computations of a plugin that I use (OpenQuake) on QGIS due to the mismatch in coordinate precision between the input data and QGIS project. I'm not very familiar with projections, so I'm not sure why QGIS adds extra decimals and how to reduce them.

1 Answer 1

4

Qgis (and with rare exception, any software) represents decimal numbers using binary floating point. In general a decimal number cannot be represented exactly by a binary floating point representation. Your CSV file entries are converted to binary floating point when read and the binary floating point value is then converted back to decimal for presentation to the user.

For example, 0.15 is converted to IEEE 754 double precision floating point as 0.0010011001100110011001100110011001100110011001100110011 which in decimal is 0.1499999999999999944488848768742172978818416595458984375

You can use this calculator to check your values if you wish.

1
  • Thank you very much! Your answer helped point me in the right direction.
    – guls
    Commented Feb 20, 2023 at 17:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.