I'm using ArcGIS Pro and recently encountered an annoying issue using Calculate Geometry with coordinates.

Let's say I create a point with DDM coordinates: 47 07.9000000E, 22 04,6000000N.

using Create Feature

If I check attributes or Coordinate Conversion tool, it shows the same coordinates as I've entered before. However if I use Calculate Geometry to fill the attrubute table, I get the coordinates like that: 47 07.8999999E, 22 04.5999999N. It never happened before in ArcGIS Desktop. I tried with different coordinate systems and options, nothing helped. It's possible to edit and round it up afterwards, but what could be the cause of the problem? table after Calculate Geometry

  • The source of these rounding errors is in the fact that the tool is a python script tool and the code that ESRI have written inherits any limitations of python. So what ever process they are using to return a string representation of X and Y is introducing the annoying rounding error. You need to approach ESRI support to get a resolution on this problem.
    – Hornbydd
    Commented Feb 16 at 10:42
  • Flating-point representation is inherently imprecise. This is a general IT issue that dates back to the 1985 IEEE standard. If you use file or enterprise geodatabase there are tweaks you can make to the spatial reference parameters before layer creation which might reduce the problem, but a ten-millionth of a minute is a very small distance (> 0.186mm at the Equator) and therefore an improper resolution for a coordinate. Ob xkcd: xkcd.com/2170
    – Vince
    Commented Feb 17 at 13:19


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.