I wonder if anyone else has run into this problem; and I'm baffled as to its reason and solution.
Whenever I do an export of features that have large-number attributes (over 14 digits) to a shapefile, they seem to lose precision. ArcGIS presents them in a default format of e-notation if they are large, but it seems to lose the last few digits of the number, for example:
728400000032249 becomes 728400000032000
Any ideas?
From the ESRI forum ArcGIS Desktop - Data Management (General) forum
Precision for shapefiles
A quote & credit to whuber (posted in 2007)
Double precision floats maintain 52 bits of precision, regardless of the sign of the number. That translates to 52 * log(2) = 15.65 decimal places. Some exceptional tiny values--less than about 10^-308 in size--have less precision; these are said to be "denormalized." Otherwise, the precision is the same for all doubles; there is no "typically" about it.
Your "bunch of extra zeros" is correct when it is understood in binary. In base 10, it is rare for a conversion from singles to doubles to result in additional zeros.
Shapefiles have to use better than single precision, which has only 23 bits of precision (about 6.92 significant decimal digits). Typical earth coordinate systems, like UTM, can have coordinates extending into the many millions of meters: that would limit shapefiles to about one meter precision when using such systems. One meter is not precise enough for large scale work. Thus, ESRI had no real choice in the matter when they designed shapefiles to use floating point coordinates rather than scaled integer coordinates.
source: http://forums.esri.com/Thread.asp?c=93&f=986&t=234634
Solution - avoid the shapefile format (stick to geodatabase) for better precision or accept the 1 meter precision.
You are hitting the limit of precision of 32-bit floating point numbers by the looks of it. The only solutions are to a) use 64-bit floats (which I suspect aren't supported in Shapefiles) or b) Create an attribute that stores the least significant digits, and scale the original value down by a few orders of magnitude, trimming off that which is stored in your new attribute, when exporting. How the value gets remultiplied and added later on depends on which software you're loading it in to.
No matter how you cut it, float32s will only ever have a maximum of 6 significant decimal figures of precision.
The answer given to this question had alternative but not the solution.
The solution is as follows:
In the Table of Contents, right-click on the layer with the Attribute Table of interest and select: Properties - Fields.
Suppose you are converting STFID to a numerical field called STFID_NUM. Then in the “Layer Properties” window that opens, scroll down to the STFID_NUM row, and in the “Number Format” column click on the gray button with 3 dots.
In the “Number Format” window that opens, select Category = “Custom”, and select String Format = “0”. Click OK, and in the Layer Properties window click Apply and then OK.
If you check the Attribute Table again, then the full digit number for STFID_NUM should now be displayed.
I agree with MerseyViking's answer,
There are a few things you can do to circumvent the problem.
Another format you could possibly use..
sdf uses sql-lite which supports blob.
Or just sql-lite
Otherwise you would need to use probably a proprietary format like esri fgdb, or a full blown rdbms to hold the blobs (binary long objects)
