QGIS tools change special characters (õ, à) of attributes into unreadable characters.
Example:
# load 1 wfs feature
uri = "http://geo.vliz.be/geoserver/MarineRegions/ows?service=WFS&version=1.0.0&request=GetFeature&typeName=MarineRegions:eez&Filter=%3Cogc:Filter%20xmlns:ogc=%22http://www.opengis.net/ogc%22%3E%3Cogc:PropertyIsLike%20matchCase=%22false%22%20wildCard=%22*%22%20singleChar=%22.%22%20escapeChar=%22!%22%3E%3Cogc:PropertyName%3Egeoname%3C/ogc:PropertyName%3E%3Cogc:Literal%3E*Gomera*%3C/ogc:Literal%3E%3C/ogc:PropertyIsLike%3E%3C/ogc:Filter%3E"
vlayer = QgsVectorLayer(uri, "test", "WFS")
# save as shapefile with UTF-8 encoding
QgsVectorFileWriter.writeAsVectorFormat(layer = vlayer, fileName = 'H:/Gomera.shp', fileEncoding = 'utf-8', driverName = 'ESRI Shapefile')
# run qgis tool
import processing
processing.run("native:fixgeometries", {'INPUT':'H:/Gomera.shp','OUTPUT':'H:/Gomera_fix.shp'})
In the output shapefile, the special characters have been changed into unreadables (compare left to right attribute tables):
The input shapefile and output shapefiles are both UTF-8 encoded. This happens with several tools, for example with fixgeometries, buffer or grass7:v.buffer, both in the python console as well as in the GUI.
How can I solve this problem? This seems something related to internal QGIS tools rather than problems with reading special characters as input as discussed here:
- QGIS Processing algorithm cannot deal with special characters in input
- ArcPy Field Calculation with special characters
I'm running QGIS 3.0 on Windows.
OEM 1252
. If you are willing to use hex editor you could also check the values of byte 30 (offset 29) from the beginning of the working and not working .dbf files. – user30184 Apr 18 '18 at 8:05OEM 1252
in .cpg gives the same (unreadable) result. byte 30 of the .dbf files is both00
, this is part of the header. It seems that the output .dbf has another encoding though, when I change it to 'Latin-1' as mentioned here: gis.stackexchange.com/a/255536/44980 the special characters show up fine. – Lennert Apr 18 '18 at 10:19