Field Calculator by Expression Incorrect

Question

I've created a new field as a whole integer with a length of 10. I am attempting to use the field calculator to assign values to the newly created field. However, I am certain the numbers aren't being calculated correctly and I'm not sure why. At this point I'm just attempting to do 36*1000000000 and the result QGIS gives is 1640261632.

Answer 1

I did a bit of testing, and the QGIS doesn't seem to allow you to calculate numbers that large. At first I thought this was a limitation of the shapefile format. Then I tested a scratch layer and a geopackage layer, and the limitations were the same. So this seems to be a limitation of QGIS.

Test 1

Entered various calculations into the field calculator, and compared the output preview.

Result: The largest correct calculation it would do is 2*1069999999 (output: 2139999998). Any calculation larger than that gave a wildly incorrect number, sometimes positive, sometimes negative.

A few examples:

• 3*1069999999 = -1084967299
• 3*1000000000 = -1294967296
• 700*1000000000 = -79669248
• 50*100000000 = 705032704

Test 2

Create an integer field, length 11. Type an 11 digit number into the field in the attribute table. Hit enter.

Result: the number disappears and is replaced with NULL.

Test 3

Label a feature using the calculation 36*1000000000.

Result: label reads 1640261632

Test 4

Do a calculation that internally creates a number larger than 2 million, but the final output should be less than 2 million.

Result: final output is incorrect.

Workaround

A workaround for your problem would be to limit your calculations to output numbers smaller than two million. For example, if you were calculating polygon area in square meters, try doing your calculations in hectares or square kilometers instead. Use a spreadsheet to convert the area into square meters.