Problem with arcpy.ValidateFieldName and unicode characters?

Question

I am having problems with the arcpy.ValidateFieldName function in combination with unicode characters. I want to add a field to a shape field. The field name is not allowed to be bigger than 10 bytes (or at least I think so), but ValidateFieldName cuts it off at 10 characters. Since a unicode character is not necessarily one byte big, this causes problems.

Consider these three examples:

#Path to an existing shape file.
file = r"C:\path\file.shp"

#This works. The name is cut after 10 characters to "abcdefghij"
name1 = arcpy.ValidateFieldName("abcdefghijklmnop", file)
print name1
arcpy.AddField_management(in_table = file, field_name = name1, field_type = "TEXT")

#This works. The name is unchanged (since it is less than 10 characters)
#ArcMap accepts it since it is not larger than 10 bytes.
name2 = arcpy.ValidateFieldName("äbcdefghi", file)
print name2
arcpy.AddField_management(in_table = file, field_name = name2, field_type = "TEXT")

#This does not work. The name is unchanged, since it is exactly 10 characters.
#ArcMap does not accept it since it is larger than 10 bytes (?).
name3 = arcpy.ValidateFieldName("äbcdefghij", file)
print name3
arcpy.AddField_management(in_table = file, field_name = name3, field_type = "TEXT")  #19

The third examples gives me this error:

Runtime error Traceback (most recent call last): File "", line 1, in File "C:\validate.py", line 19, in arcpy.AddField_management(in_table = file, field_name = name3, field_type = "TEXT") File "c:\program files (x86)\arcgis\desktop10.2\arcpy\arcpy\management.py", line 3200, in AddField raise e ExecuteError: ERROR 999999: Error executing function. The Field type is invalid or unsupported for the operation. Failed to execute (AddField).

Strangely enough, it still adds the field but with the name "äbcdefgh_1", even though the value of name3 is "äbcdefghij".

The documentation has this to say about ValidateFieldName:

Takes a string (field name) and a workspace path and returns a valid field name based on name restrictions in the output geodatabase.

Would I be right to say that this is a bug, since the value returned in the third example is in fact not a valid field name?

Can I somehow make this work without errors? I need to be able to take the name of a field from user input and somehow make sure it get turned into a valid name.

Why is the error message complaining about the field type, even though "TEXT" should be an OK field type?

EDIT: This seems to be a bug with ArcMap. If I add the fields from ArcCatalog instead of with Python, I get a similar behavior.

Answer 1

I have been in contact with ESRI support, and they confirm that this is indeed a bug. It is fixed in ArcGIS 10.3, so the solution is to upgrade.