Take the 2-minute tour ×
Geographic Information Systems Stack Exchange is a question and answer site for cartographers, geographers and GIS professionals. It's 100% free, no registration required.

This question already has an answer here:

NOTE: This problem is similar to one that was answered here: Why does Field Calculator return 0 values when decimals are expected?

I'm relatively new to python, and am trying to run a simple field calculation using US Census data![Screenshot of DOUBLE fields][1]

I'm trying to do a simple calculation where I sum (Cnt1+Cnt2+Cnt3)/ (Pop1+Pop2+Pop3).

I tried posting screenshots but apparently I don't have any street cred here :(

I have tried creating various types of new fields: double, long, float, and played around with the scale/precision/properties, but everytime it returns "0" values. I know the values will be very small, i.e .04 but it doesn't seem to be a decimal display problem as I tried running it with Pop1/.4 and it correctly returned a decimal. Any suggestions would be much appreciated, sorry about the lack of screenshots!

share|improve this question

marked as duplicate by whuber Apr 8 '13 at 12:50

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

In 10.0 the default interpreter is VBscript. Any chance you haven't selected Python? –  Roy Apr 5 '13 at 1:08
Hi Roy, that's a good thought; unfortunately, I double checked and Python is selected. –  David Meek Apr 5 '13 at 1:19
Do you have spaces around each mathematical operator? (the plus signs and slash) Field Calculator can be quite finicky about that. –  Erica Apr 5 '13 at 13:18
See gis.stackexchange.com/a/57400. –  whuber Apr 8 '13 at 12:45

3 Answers 3

You need to force float calculation. Right now you are doing integer math (I assume all of your source fields are short or long, hence the results you are getting), so if your result value is less than 1, it is flooring the value down to the next integer, which is zero. The type of the destination field does not matter.

This is very easy to fix. Do your calculation like this:

1.0 * (Cnt1+Cnt2+Cnt3)/ (Pop1+Pop2+Pop3)

The 1.0 out front (not 1, but 1.0) forces a float calculation without altering your end result. (Same thing happened when you did Pop1/.4 The 0.4 was a float value, so that forced float calculation.) If any of your source fields were float, you would also have ended up with float math instead of integer math (at least at 10.1 where I tested this).

share|improve this answer
You noted below that your source fields are doubles. So the consideration of source field type might be a 10.1 thing; and the fields are being coerced to integers since they contain whole numbers. –  blord-castillo Apr 5 '13 at 14:24
Many thanks blord-castillo, this worked perfectly. Problem solved. –  David Meek Apr 5 '13 at 14:26
Actually, problem temporarily solved, I was testing this out in Field Calculator, but the real objective is for it to run in a stand alone python script. However, when I run it in the script I get the following error: Traceback (most recent call last): File "C:\gislab2\Python\take_home\part1\prelim_if2.py", line 45, in <module> Field_name= 1.0*(int("Cnt1"+"Cnt2"+"Cnt3")/("Pop1"+"Pop2"+"Pop3")) ValueError: invalid literal for int() with base 10: 'Cnt1Cnt2Cnt3' I only put the (int( part in as I got an error it couldn't multiply 1.0 by string –  David Meek Apr 5 '13 at 14:31
Yes, what you are doing there is concatenating the string, not doing addition. What function are you feeding field name into? –  blord-castillo Apr 5 '13 at 15:14
@David Meek - What works in the field calculator won't usually work in a stand-alone script. If you are going stand alone, try using a cursor to grab your values, do the calculation, and update your field. check arcgis resources –  gm70560 Apr 5 '13 at 19:34

The general format for python based field calculations is as follows:

(!Cnt1! + !Cnt2! + !Cnt3!) / (!Pop1! + !Pop2! + !Pop3!)

where fields are specified as !fields! in the equation.

Also, try specifying Float for the new field you create.

share|improve this answer
Hi Aaron, thanks for your response; I have it in the format you specified (!Cnt1! + !Cnt1! + !Cnt1!) / (!Pop1! + !Pop2! + !Pop3!); also I've tried specifying the field as Float, but still no dice. –  David Meek Apr 4 '13 at 21:56
Hi Aaron, thanks for your response; I have it in the format you specified (!Cnt1! + !Cnt1! + !Cnt1!) / (!Pop1! + !Pop2! + !Pop3!); also I've tried specifying the field as Float, but still no dice –  David Meek Apr 4 '13 at 21:58

I see you said that you've played around with field properties, but you didn't say which fields you did that for. The ones I'd check for sure are the population fields because 1) they are used in the denominator and 2) population figures are usually integers. Make sure you either make your population fields float/double or cast the values as floats when you make the calculations.

share|improve this answer
Jason, good thoughts: to clarify, I've only tinkered with the properties of the new fields I've tried creating, but have checked the properties of the POP and CNT fields. They've shown DOUBLE under properties, but is it possible the data are still string in those fields? If so, would it be advisable to create a new set of POP and CNT fields where I take the original values and force double them? Thanks again, david –  David Meek Apr 5 '13 at 14:20

Not the answer you're looking for? Browse other questions tagged or ask your own question.