4

I'm trying to extract attribute data from a layer in QGIS (version 2.18.2 for mac) to use it internally on my plugin (I need that due to complex matrix operations with numpy and scipy), but QGIS is always returning rounded values with different methods, e.g.:

Original data when viewing it from open attribute table option:

[3.46000000, 0.48000000, 12.86999999, 13.30000000, ...]

Using getValues() or getDoubleValues() functions:

layer = iface.activeLayer()
print layer.getValues('G2')[0]  #G2 is the field name
>>> [3.46, 0.48, 12.87, 13.3, ...]

Using getFeatures() function:

field = [f['G2'] for f in layer.getFeatures()]
print field
>>> [3.46, 0.48, 12.87, 13.3, ...]

And using dataProvider() function:

dpro = layer.dataProvider()
feat = [f['G2'] for f in dpro.getFeatures()]
print feat
>>> [3.46, 0.48, 12.87, 13.3, ...]

At the first moment I thought it could be a display limitation from console's output (I was testing there before implementing on the plugin), but running from the plugin I had the same result.

Is there a method where I can extract the data with all the decimals?

Improve this question
3
  • 2
    I tried to duplicate the problem you are seeing in 2.18.3 on a PC by creating a scratch point layer with 5 points and a field G2 with the values you have listed, and I did your example with layer.getValues... and I got all of the decimals. My return was [3.46, 12.86999999, 0.48, 3.461, 13.3]. The 3.461 value was one that I added as a test. What kind of layer are you working with and what are the properties of the field G2?
    – TJR
    Commented Mar 8, 2017 at 22:30
  • That's really strange, I'm using a normal vector layer with Census data as attributes, the details about the field are: Type: double; Type name: Real; Length: 23; Precision: 15; WMS and WFS flagged
    – Sandro Sousa
    Commented Mar 9, 2017 at 19:55
  • Just to complement, it's a polygon layer. Something curious is happening, I tested the functions with different layers and fields and they worked fine displaying all of the decimals. But on the mentioned field and layer looks like QGIS is rounding the data based on the majority of values length, most of than are like 3.46000000 but some lines has 31.53999999. Looks like it's trying to guess the data type from a sample and applying it to the rest of the lines. I thought could be a data problem but it should return the field length as it is.
    – Sandro Sousa
    Commented Mar 9, 2017 at 20:47

1 Answer 1

Reset to default
2

This looks like you have run into a very special rounding case inside of QGIS, and the short answer is that the methods that you are using are getting "all of the decimals."

If you look in your attribute table, there are probably more 9's than shown in the original question. I'm guessing it was displayed as 12.86999999... and if you widen the column you would probably see 12.869999999999999 which is the full 15 digits of precision possible.

In this particular case it seems to decide that you have so many 9's it's essentially .9 repeating and in that case it makes sense to round to 12.87 (if it really was .9 repeating, then mathematically it would be accurate to say that it is 12.87)

If you edit your attribute table and just take out one of the 9's and do a layer.getValues('G2')[0] then the return will be 12.86999999999999.

Improve this answer
1
  • Nice explanation, this really make sense, especially because I tested it on a different layer where most of the data has a long decimal length (not repeated) and it displayed the value correctly... Thanks a lot!
    – Sandro Sousa
    Commented Mar 9, 2017 at 20:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.