0

I'm using the geometry calculator to get the geographic coordinates of a point shapefile. However when I run it, it calculates the coordinates with 4 decimal points, regardless I've chosen double an increase the precision on the field definition..

I need the coordinates with at least 8 decimal points.

Can someone offer advice?

Improve this question
12
  • Hi Astrid, welcome to GIS.SE. From memory Shapefiles store coordinates as doubles (IEEEE 32 bit floating point) en.wikipedia.org/wiki/…, depending on how many numbers are before the decimal you may not be able to get more than 8 decimal places. Why do you need to get coordinates to the nearest nanometre?
    – Michael Stimson
    Feb 18, 2015 at 0:54
  • Hi, thank you! ... I need more than 4 that is what I'm getting at the moment.. I'm working in geographic coordinates and 4 decimal is not representative enough for some points that are so close to each other.. I don't want to project them because I'm using those points to create a kriggin surface in R and I know reprojecting that raster back might result in inaccurate location. Thanks!!
    – Astrid V
    Feb 18, 2015 at 0:58
  • 2
    Make sure that you're seeing all possible digits. Right click the field and select properties. Click Numeric button to access how many digits should display. When I do it, the default is 6 digits. Does the data have a defined coordinate reference system? If not, try defining it with the appropriate geographic coordinate reference system.
    – mkennedy
    Feb 18, 2015 at 0:59
  • 1
    Then that may be all that's actually stored in the geometry values, unfortunately.
    – mkennedy
    Feb 18, 2015 at 1:12
  • 1
    @AstridV Does the shapefile have a .shp.xml? At what scale were the coordinates captured? It sounds like you have 1:10m data. Data which is accurate to 1mm is a good deal rarer than data which is precise to 1mm.
    – Vince
    Feb 18, 2015 at 1:28

1 Answer 1

Reset to default
1

As commented by @MIchaelMiles-Stimson:

There's a discussion about it here http://forums.esri.com/Thread.asp?c=93&f=986&t=234634; IEEE has been around for a long time and has defined floating points from 16 to 128 bits. That's definitely enough precision to record UTM coordinates to the mm; any more refinement on that would be generally unnecessary.

and by @Vince:

An 8-byte float pair is enough to store UTM coordinates to the Angstrom, and decimal degrees coordinates submillimeter, though it's rare that geodata is accurate to decimeter scale, so many of those digits are just wasted storage.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.