# Why does Field Calculator return 0 values when decimals are expected?

I realize I posed this question, and got answers, several days ago (see (Field calculator error)) , but none seem to really be working, and so am posing again.

Below are two screenshots that show the fields with field properties, and another showing the field calculator and the equation with the "answer" field populated by 0s. I apologize for them not being next to each other or in line with the appropriate text, but the web formatting keeps overlaying them.

Here is an overview of the problem: I am trying to do a in field calculator, where I'm trying to populate a new field with the values from a fairly simple equation of one field divided by another (Cnt1 by Pop1 in this case).

Both fields that I am dividing are double, and the value, while small is not infinitesimal, approx 0.01765.

I have tried creating various types of new fields: double, long, float, and played around with the scale/precision/properties, but everytime it returns "0" values. I know the values will be very small, i.e .04 but it doesn't seem to be a decimal display problem as I tried running it with Pop1/.4 and it correctly returned a decimal.

I've tested dividing the Cnt1 by .5 and that gives an appropriate decimal value, so it's not the new field format; the one thing that seems really strange is when I add (in field calculator python version): !Cnt1!+!Pop1! (which is 38+218699) I get 2.187, however, I suppose this could be related to the field being classified as float with precision=4, scale=3?

I've also tried the suggestions of @blord-castillo of doing:

``````1.0* (Cnt1/Pop1)
``````

and also 1.0* (int(Cnt1/Pop1)) But neither works, I'm at my wits end on this one, and would greatly appreciate any help!

-
In 10.0 the default interpreter is VBscript. Any chance you haven't selected Python? – Roy Apr 5 '13 at 1:08
Hi Roy, that's a good thought; unfortunately, I double checked and Python is selected. – David Meek Apr 5 '13 at 1:19
Can you add to your post the properties of each input and output field (type) ? – A.R Apr 7 '13 at 12:32
Could you post a screenshot of your attribute table so we have an idea of what you're working with? – Aaron Apr 7 '13 at 13:13
possible duplicate of Field calculator error – Aaron Apr 7 '13 at 13:15

I just ran a test at ArcGIS 10.0 SP5.

You can see my results below.

To achieve this in a file geodatabase polygon feature class, I had field cnt1 as Short Integer, pop1 as Long Integer, and Stnd3 as Double.

In the Field Calculator I used:

Stnd3=

``````float(!cnt1!) / float(!pop1!)
``````

I then used the context menu of the Stnd3 field to access the Numeric... button on its Properties to set the Number of decimal places to 9.

Is this the result you are after?

-
wow! This is perfect, it is exactly what I'm looking for. So the question for me to learn from this is why? Can you help me understand why the fields each need to be formatted differently in order to get the calculation to work? – David Meek Apr 7 '13 at 22:55
It's probably confusion over different syntaxes needed for different parsers (VB,Python) and data type (File & Personal Geodatabases, shapefiles) combinations. I stick with Python and File Geodatabases wherever possible. The other thing is to just use the Numeric... button to format how many decimal places and allow the File Geodatabase to just store Float/Double (whichever you need). I prefer to use the Python float function to convert integers rather than just multiplying by 1.0 (but I think that should have worked too). – PolyGeo Apr 7 '13 at 23:06
update is apparently this is a problem with my machine or SP or something as this still isn't working. Had someone try it on another machine and same code worked. Hmmm – David Meek Apr 8 '13 at 1:05
question that year sago solved many problems with arcgis calculations: what is your windows format (region and Language in Control Panel) ?? did anyone by mistake changed the Numbers format (open additional settings and 2-check the Numbers formats). I hope it helps – A.R Apr 9 '13 at 13:04
wow! "poly-geo" has halted premature grey hairs spreading across my head. thank you! poly-geo should start his/her own website and charge subscribers a fee. well done. – user39495 Oct 21 '14 at 8:44

You need to force float calculation. Right now you are doing integer math (I assume all of your source fields are `short` or `long`, hence the results you are getting), so if your result value is less than 1, it is flooring the value down to the next integer, which is zero. The type of the destination field does not matter.

This is very easy to fix. Do your calculation like this:

``````1.0 * (Cnt1+Cnt2+Cnt3)/ (Pop1+Pop2+Pop3)
``````

The 1.0 out front (not 1, but 1.0) forces a float calculation without altering your end result. (Same thing happened when you did `Pop1/.4` The 0.4 was a float value, so that forced float calculation.) If any of your source fields were `float`, you would also have ended up with `float` math instead of integer math (at least at 10.1 where I tested this).

-
You noted below that your source fields are doubles. So the consideration of source field type might be a 10.1 thing; and the fields are being coerced to integers since they contain whole numbers. – blord-castillo Apr 5 '13 at 14:24
Many thanks blord-castillo, this worked perfectly. Problem solved. – David Meek Apr 5 '13 at 14:26
Actually, problem temporarily solved, I was testing this out in Field Calculator, but the real objective is for it to run in a stand alone python script. However, when I run it in the script I get the following error: Traceback (most recent call last): File "C:\gislab2\Python\take_home\part1\prelim_if2.py", line 45, in <module> Field_name= 1.0*(int("Cnt1"+"Cnt2"+"Cnt3")/("Pop1"+"Pop2"+"Pop3")) ValueError: invalid literal for int() with base 10: 'Cnt1Cnt2Cnt3' I only put the (int( part in as I got an error it couldn't multiply 1.0 by string – David Meek Apr 5 '13 at 14:31
Yes, what you are doing there is concatenating the string, not doing addition. What function are you feeding field name into? – blord-castillo Apr 5 '13 at 15:14
@David Meek - What works in the field calculator won't usually work in a stand-alone script. If you are going stand alone, try using a cursor to grab your values, do the calculation, and update your field. check arcgis resources – gm70560 Apr 5 '13 at 19:34

The general format for python based field calculations is as follows:

(!Cnt1! + !Cnt2! + !Cnt3!) / (!Pop1! + !Pop2! + !Pop3!)

where fields are specified as `!fields!` in the equation.

Also, try specifying Float for the new field you create.

-
Hi Aaron, thanks for your response; I have it in the format you specified (!Cnt1! + !Cnt1! + !Cnt1!) / (!Pop1! + !Pop2! + !Pop3!); also I've tried specifying the field as Float, but still no dice. – David Meek Apr 4 '13 at 21:56
Hi Aaron, thanks for your response; I have it in the format you specified (!Cnt1! + !Cnt1! + !Cnt1!) / (!Pop1! + !Pop2! + !Pop3!); also I've tried specifying the field as Float, but still no dice – David Meek Apr 4 '13 at 21:58

I see you said that you've played around with field properties, but you didn't say which fields you did that for. The ones I'd check for sure are the population fields because 1) they are used in the denominator and 2) population figures are usually integers. Make sure you either make your population fields float/double or cast the values as floats when you make the calculations.

-
Jason, good thoughts: to clarify, I've only tinkered with the properties of the new fields I've tried creating, but have checked the properties of the POP and CNT fields. They've shown DOUBLE under properties, but is it possible the data are still string in those fields? If so, would it be advisable to create a new set of POP and CNT fields where I take the original values and force double them? Thanks again, david – David Meek Apr 5 '13 at 14:20