I've found several example of script and blog posts on working with unicode characters, but I haven't been able to make anything works so far... It's a little frustrating
I have a shapefile encoded in utf-8 (exported from QGIS) and there's many É, È, À, Ô, etc. in the values of some fields.
For a geocoder, I need to normalize my fields values.
This is what I have so far:
# -*- coding: utf-8 -*-
###############################################################################
import sys
from sys import argv
import osgeo.ogr
from amtpy import EndOfScript
###############################################################################
script, src_shp, fld_shp = argv
#- Opening the shapefile
shapefile = osgeo.ogr.Open(src_shp)
layer = shapefile.GetLayer(0)
spatialRef = layer.GetSpatialRef()
#- Going through each feature, one by one
for i in range(layer.GetFeatureCount()):
print "Normalisation de la ligne %i" %(i+1)
feat = layer.GetFeature(i)
texte_norm = feat.GetField(fld_shp)
texte_norm = texte_norm.encode('utf-8')
texte_norm = texte_norm.upper()
texte_norm = texte_norm.replace(u'\00c2', 'A') #À
texte_norm = texte_norm.replace(u'\u00C9', 'E') #É
texte_norm = texte_norm.replace(u'\u00C8', 'E') #È
# I've remove 16-17 characters to replace, for the example...
texte_norm = texte_norm.decode('utf-8')
print texte_norm
#feat.SetField(fld_shp, texte_norm)
#- print EndOfScript message
print EndOfScript()
I've put a print instead of the SetField just so I could see if the replace works.
Then the script encounters the first unicode character, I have this error message: 'ascii' codec can't decode byte 0xc3 in position 5: ordinal not in range(128).
Well, I know that, since the character is a "À". I can't seem to find the right way.
Any tips?
Thanks!
texte_norm.encode('utf-8')
step and just skip straight to thereplace
function it should work:>>> s = 'Ècouter'
>>> s
'\xc3\x88couter'
>>> print s.replace('\xc3\x88','E')
Ecouter