0

I have a JSON file containing special characters, which can be represented in Unicode. However, as soon as I export that JSON file to a file geodatabase using arcpy, the special characters all "break," i.e., they come out as something different. A Google search tells me (via this table listing string representations of special characters) that the character I'm having a problem with in my sample dataset is in UTF-8/Latin-1.

The code I'm using for this is:

arcpy.JSONToFeatures_conversion(jsonFileName, os.path.join(fgdbPath, outputName))

I have also specified the encoding for the Python (2.7) file via:

import sys
reload(sys)
sys.setdefaultencoding('utf-8')

I have also varied the above with "latin-1" and various other specifications for the encoding, mostly experimentally as I have little experience with encoding.

But regardless, where the content of my JSON file is for example "Búzi", the content of my feature (exported to an FGDB) will be "Búzi".

Nowhere can I find out about a parameter to pass to JSONToFeatures_conversion, or to CreateFileGDB_management (where I create the FGDB in question), or an Arc environment variable to set, which will allow me to specify the desired encoding.

How do I export to FGDB with the desired encoding (or providing the desired result by whatever means)?

  • Does this not address it? resources.arcgis.com/EN/HELP/MAIN/10.1/index.html#//… – Paulo Raposo Nov 30 '15 at 10:33
  • 1
    can you share your json file and check gdb config key. If your json is not encode in utf-8 sys.setdefaultencoding('utf-8') dont change the problem. you need rencode json file to utf8 before the conversion – GeoStoneMarten Nov 30 '15 at 11:10
  • The contents of the JSON file are correct. If you need to verify this, where can I share the file? I assume copy/pasting the JSON contents is not sufficient to determine if the encoding is correct? Also, the configuration keyword would probably solve this, but how can it be set for the FGDB using arcpy? – Wolfie Inu Nov 30 '15 at 11:25
  • 1
  • 1
    I have same error. I check with different params: new gdb with arcpy.env.configKeyword = "TEXT_UTF16" ; add arcpy.env.configKeyword = "TEXT_UTF16" in for workspace in script... No change. I have already check in *.mdb but there is same result. Can you test with insert cursor with a table? – GeoStoneMarten Nov 30 '15 at 14:55
1

I got an answer from ESRI, basically even though it seemed like the JSON was coming off the server correctly (via a REST URL), it was getting encoded incorrectly while being read. I used to do it this way:

import sys
reload(sys)
sys.setdefaultencoding('utf-8')

# later...
jsonResponse = urllib2.urlopen(jsonRequest)
jsonFile = codecs.open(jsonFileName, 'w', 'utf-8')
jsonFile.write(str(jsonResponse.read()))
jsonFile.close()

Even though there was a string conversion to prevent errors while writing the response to a file, I did specify the default encoding and assumed that this took care of the problem, since the encoding seemed correct when opening the saved JSON file in Notepad. However, it turns out that Notepad was being too clever for itself and doing some implicit conversions or something, because writing the JSON file with a different method solved the problem, so obviously my string conversion was a stupid thing to do.

The solution provided by ESRI was:

jsonResponse = urllib2.urlopen(jsonRequest)
with io.open(jsonFileName, 'w') as jsonFile:
    jsonFile.write(
        ''.join([i.decode('utf-8') for i in jsonResponse.readlines()]))

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.