3

I would like to build some queries in my addin from layers in my map. I am having problems with double numeric types. esriFieldTypeDouble has 14 - 18 decimal places or more depending on the database type. That's what I get when I build a "Select by Attributes" but when using ArcObjects with .Net I only get 12 or less decimal places (System.Collections.IEnumerator) and although I can get 14 decimal places when I retrieve a value directly from iRow.value() ,converting this number to a string in any way rounds it to 12.

I am testing this on SHAPE.STLength() and SHAPE.STArea() fields (SQLServer EXPRESS) and well as SHape_Area, SHape_Length (FGDB) and the problem is that since the value actually has more than those 12 decimal places I am getting in code but the actual value has more the query will not work.

Building the query in the attribute table's "Select by Attributes" correctly displays ALL decimal places in the Unique values list

Attempting to build the same list with the same ArcObjects that are uses to obtain this list "under the hood" System.Collections.IEnumerator rounds those to 12 places.

Is this a VB .Net issue? Is there any way to get all the decimal places to string?

Improve this question
2
  • 1
    I encountered this issue a while back, and I ended up following the advice Microsoft gives on how to compare two doubles. msdn.microsoft.com/en-us/library/ya2zha7s(v=vs.110).aspx gis.stackexchange.com/questions/118105/…
    – Dan Jurgella
    Commented Jun 16, 2016 at 17:02
  • Interesting. I need the actual values as string because I am building queries often containing 100s of values. Very simple example: Shape_Area IN (687.98923456335678) is what i expect to get. However when i build it in code I get Shape_Area IN (687.989234563357) - the number is rounded to 12 digits. Because it is no longer the same number, the query fails to retrieve the desired record. I don't am not comparing the values 2 double objects in code but rather converting these to string and concatenating.
    – Jakub Sisak GeoGraphics
    Commented Jun 16, 2016 at 18:51

1 Answer 1

Reset to default
1

I found a post with essentially at Row.Value returns rounded SHAPE.area and SHAPE.len values?

No answer is given but the comment section provides good insight and possible workrounds. This problem is related to "False Precision" and "failure to bracket floating-point comparisons".

I decided to use the rounding of value pairs workaround.

I also posted VB.Net changes double values when converted to string on the Esri Forums where I reached more less the same conclusion prior to finding the above mentioned post.

Improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.