Discrepancy between constructed line length and length recorded in attribute table in ArcGIS for Desktop?

Question

Say that in an edit session I create a line of an exact length entered by keyboard - say 10'

If I later measure that line using the measure tool, snapping to each end of the line, it reads 9.99998'

Then when I check the Attribute Table immediately after completing the line, I find that the line length is 9.99998'.

This is an actual example I just tried, but it it seems to be consistent with all such measurements. They're all off - always less than what I entered as far as I can tell - by a small amount.

Now, I realize this is a very small difference, but shouldn't the editing tool build a line that is exactly 10' if that is the length I enter, and then shouldn't the measure tool "measure" the line to be 10' if that is the length I entered when creating the line?

What is going on here? And although for what I am likely to be doing it doesn't matter, is there a way to prevent this.

To clarify: I create the line in an ArcView edit session by clicking where I want the line to start, then right clicking and selecting "exact length" to type in the length of the line I want to create. I type in 10', and the next node is dropped, in theory at least, exactly 10' from the last point. Then I click F2 to complete the drawing. When I check the attribute table it shows that that the segment is 9.99998' rather than the 10' I specify. I first noticed when I was using the measuring tool - snapping at the beginning and the end of the line. I'm not exactly sure how I could clarify the more.

I am using ArcView 10.1 SP1 Data frame and data are both NAD_1927_StatePlane_New_York_Long_Island_FIPS_3104. Data are created and stored in a File Geodatabase. Not sure what the tolerances are at the moment.

Answer 1

Lines and polygon dimensions are never exact and subject to the limitations of the feature class resolution and tolerance. Infinite precision is not possible and as a result these settings tell GIS to consider measures below those levels as "close enough" to exact measurements. There is no setting for absolutely no tolerance and infinite resolution for measurements, since the capacity of computers to store information is not infinite. Setting your tolerance and resolution to extremely small distances will reduce the potential limits of the area you can cover on the face of the globe compared to higher levels of tolerance and resolution, and may have consequences affecting performance, disk space use and snapping. This help topic provides more information about the effects of resolution and tolerance settings.

The computer has to fit the end points to a grid defined by the resolution and that takes precedence over the distance you entered. The help makes that clear. What you enter is less important than what a computer can actually efficiently store. The tool you are using to create your line will identify the grid cell that the end point of the line will fall in and then the line end point is snapped to the closest corner point of that cell. After it determines the end points it can actually store, then it recalculates the distance based on those points and reports it. The only time it might be exact is if your line was exactly aligned to the x or y plane, but probably not after you apply a projection that has to distort a 3D curved surface on to a 2D grid.

Another note is that in the background GIS is standardizing and storing all Projected Coordinate System linear positions and measurements relative to meters. All projected measurements based on English units will undergo unit conversions at some point during GIS processing. That is why when I set up my Linear Units as Foot_US in the projection I use it has to be expressed as 0.3048006096012192 in the XY Coordinate System (the equivalent of 1' expressed as a fractional portion of 1 meter). Add to this conversions between base 10 and base 2 for numbers to be manipulated and stored by the computer, and it should not be too surprising that 10' is not 10' after GIS gets through with it. The Projected Coordinate System itself probably has a larger impact on why there is distortion in the distance, but all of these complex processes and conversions can contribute some levels of variance between the user's initial input and the final output.

As a final note, a human hair is on average estimated to be about 0.1 mm thick in diameter, which is the equivalent of 0.000328 feet. So 0.00001 feet is approximately 33 times smaller than the width of an average human hair. Surveyors typically don't report distances less than 0.01 feet, which is approximately 30 times wider than an average human hair. So 0.00001 feet is a level of precision that is 1000 times finer than what is typically reported on most surveyed maps and documents. If that amount of measurement variance in the field is not negligible for your purposes, than I pity you for the amount you will have to pay for surveys.