1

Can generating a shapefile from a pgdb cause topology errors due to floating point conversion? I am running into gaps in a line layer of really small distances (order of 10^-4; CRS EPSG:2227 usft).

1

Shapefiles still store numbers using 64 bits of precision, 16-16 digits. It looks like you need 11 or 12, so you should not get errors unless you are doing computations in which case floating point errors are a remote possiblility.

If anything is ever converted into a 32-bit floating point number you would definitely lose precision.

  • Great, thanks, I wanted to make sure that the gaps were real and not computational artifacts before going through with fixing them. – DLY Sep 10 '14 at 17:59
  • You mean coordinates use 64 bit precision, numbers as attributes use less digits webhelp.esri.com/arcgisdesktop/9.3/… – user30184 Sep 10 '14 at 18:51
  • @user, correct. Numeric data is stored using dBase IV format (DBF file), which means as strings of digits. – Russell at ISC Sep 10 '14 at 19:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.