I have a python toolbox (.pyt) where the user selects a layer, a field from that layer and then values from that field. The tool uses this to make a selection based on the field values. It's working fine for all other datatypes (integer, text/string, float) but there is an issue with double that I think I've narrowed down to precision.

Manually making the selection by attribute the field value is 0.1000000014901162 (works). The value displayed in the tool is 0.100000001490116 (see image). enter image description here

The field is data type double with a precision of 0 and scale of 0 (see image). enter image description here I would like a solution that will work across any precision/scale. I've tried using python's decimal, but that gives too high a precision 0.100000001490116119384765. This one selection is part of a very long script but I've disassembled the functions and here are the relevant bits (not indented/functional):

    #def getParameterInfor(self):
    # FC
    landUse = arcpy.Parameter("Landuse/Greenspace Polygons",
                              "land_use", "GpFeatureLayer", "Optional", "Input")
    # Field
    LULC_field = arcpy.Parameter("Greenspace Field", "LULCFld", "Field", 
                                 "Optional", "Input")
    # List of fields from table
    landVal = arcpy.Parameter("Greenspace Field Values", "grn_field_val", 
                              "GPString", "Optional", "Input", multiValue=True)

    LULC_field.parameterDependencies = [landUse.name]
    landVal.parameterDependencies = [LULC_field.name]
    landVal.filter.type = 'ValueList'

    #def updateParameters(self, p):
    # Conditions must be met for each input to become active
    if True in [p[4].value, p[6].value]:
        p[17].enabled = True
        p[17].enabled = False
    if p[17].altered:
        p[18].enabled = True
    if p[18].altered:
        in_poly = p[17].valueAsText
        TypeField = p[18].valueAsText
        p[19].enabled = True
        p[19].filter.list = unique_values(in_poly, TypeField)

    #def main()
    landuse = params[17].valueAsText
    field = params[18].valueAsText
    fieldLst = params[19].values

    # Convert the list of unicode to float
    fieldLst = map(float, fieldLst)

    # Make selection
    arcpy.MakeFeatureLayer_management(landuse, "lyr")
    # Create where clause from list
    exp =''
    for item in fieldLst:
         if type(item) == float:
              exp += '"{}" = {} OR '.format(field, float(item))
    whereClause = exp[:-4]
    sel = "NEW_SELECTION"
    arcpy.SelectLayerByAttribute_management("lyr", sel, whereClause)

I haven't included all the code to differentiate datatypes, but the above currently works on float and other datatypes are differentiated either when converting from unicode or when creating a where clause from the list.

  • I'd use truncate, search Python sites
    – FelixIP
    Commented Jun 16, 2017 at 23:53
  • Truncating would work if values in the fieldLst were too long. The problem here is selecting features using a truncated value. The field's values could be queried, truncated and then written to a new field to use for selection, but that seems excessive, considering this is a limited issue.
    – jbosq
    Commented Jun 19, 2017 at 13:48

1 Answer 1


Instead of solving the problem in python, ROUND() allows you to make the sql selection based on a field value rounded to a specified decimal place. exp then becomes:

 exp += 'ROUND({},{}) = {} OR '.format(field, dec_place, repr(item))

where dec_place is the number of decimal places:

dec_place = len(repr(item).split(".")[1])

repr() was important as str() may round item further if performed on a float(). I chose to restrict this to instances where dec_place >= 15 (since in the OP that seems to be the .pyt cut off) foreseeing instances where rounding in the selection could cause unintended problems.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.