3

I have a small issue regarding snapping accuracy in ArcMap 10.1 (Build 3035). I work on shape files in WGS84. In the editing options I did set the 'display measurements using' --> 20 decimal places. Coordinates of a vertex look like this: -70,65338002831136100000 -33,44227604994820500000. Now, if I take a vertex from a neighboring polygon and snap it to this one (-70,65338002831136100000 -33,44227604994820500000) the coordinates are set to: -70,65338002831129000000 -33,44227604994819100000. So it looks like the coordinates are aligned up to a certain decimal, but not further. Is there any setting that fixes this issue? How to handle it?

enter image description here

Improve this question
3
  • 1
    You can ignore it, you don't actually need that level of precision and whatever measurements you're using are not accurate down to that level. 20 decimal places in lat/long is down to the sub-atomic level.
    – Dan C
    Commented May 9, 2017 at 15:41
  • This causes an issue in my external tool that i use on the output shapefile (after ArcMap manipulation). So i can not simply ignore that...
    – guest
    Commented May 9, 2017 at 15:49
  • 1
    But the coordinates in the shape are stored as a 64-bit double floating point number. This type can´t have 20 decimals. It can have a mantisse of a length of 16. You need three in front, which leaves you 13 digital places.
    – Matte
    Commented May 9, 2017 at 16:06

2 Answers 2

Reset to default
4

There are several issues raised in this question:

Accuracy and precision are not equivalent

You state that you have a question about "snapping accuracy", but the rest of the question focuses on precision issues. Quoting wiki.gis.com:

Accuracy is the degree of veracity while precision is the degree of reproducibility.

That is, accuracy is the difference between the actual position and the value measured, while precision is closeness of the values when when measured multiple times.

It is not possible capture 20 places of precision in a shapefile

The IEEE standard defines how floating-point values are encoded in bytes. The mantissa of a 64-bit "double" contains 53 bits (implicitly), which only provides for 15.955 significant digits (which, depending on the magnitude, only permits 12-14 digits to the right of the decimal in decimal degree values).

It should be noted that 12 places is roughly 1.11e-08 meters (11 nanometers), which would allow you to map ultraviolet light as an area feature. Twenty places, ignoring Heisenberg's Uncertainty Principle, would allow you to map individual protons within atoms.

I should mention that any geospatial tool which cannot tolerate coordinate differences of less than a micron ought not be trusted for any purpose.

Shapefile isn't the best storage format for this sort of operation

The file geodatabase format permits you to specify precision in coordinate values in a way that shapefile does not. Geodatabase also defines the origin from which measurements are made, the absolute precision with which it is measured, and a separate value for the tolerance by which coordinates are considered equivalent. In fact, it would be possible for geodatabase to map at 1.0e-20 degree precision, but only within a single square micrometer.

I would recommend you define a custom spatial reference, with a WGS84 geographic coordinate system (GCS) an origin of {-400,-400}, an XY resolution of 0.0000001, and a XY tolerance of 0.0000002, then use that to create a feature class, and load your shapefile into that. Once you've finished snapping, export again to shapefile.

Improve this answer
2
  • +1 great explanation. Just curious though, if there's a snap-to-spatial-reference happening somewhere in there too, as described on page 13 of Esri's Understanding Coordinate Management in the Geodatabase white paper. In other words, if one has set up snapping to some other featureclass, does it snap to that other featureclass or to the spatial reference of the featureclass being edited?
    – Kirk Kuykendall
    Commented May 9, 2017 at 23:44
  • All geometries are snapped to a coordinate reference on read (because SgShape requires a CR). Which coordinate reference is where it gets fuzzy, since an FGDB FC has an SR, and the canvas has an SR, but shapefiles need to generate one on the fly (usually with one of those whacked xyunits, which generates irrational coordinates). Of course, snapping is another kindle of kittens entirely, because the snap distance is relative to the spatial reference units (coords should only be snapped once, since the snap distance itself is snapped to the SR)
    – Vince
    Commented May 10, 2017 at 0:24
3

First of all its about precision and not accuracy. Its important not to mix them. Accuracy is something that includes many topics(like origin of the data and so on). Precision is kind of the technical side to store and process the data.

The 8.th decimal is already milimeter, so there will be no task ever were a GIS might reasonably need 20 decimals. The normal snapping tolarance where the program decides if two points are on top of each other or if polygons are neighbor or not is by default at the milimeter level. If you do not change this the deviations behind the 9th decimal (rounding issues) does not matter. And even if you would set the tolerance to a much smaller value i think that the program and underlying data types can´t handle them anyway. The 14th decimal might be that spot like you found out.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.