Hot answers tagged encoding
9
Assuming that you are on Linux and have access to a recent version of GDAL you can try the following (from this post) :
export SHAPE_ENCODING="ISO-8859-1"
ogr2ogr output.shp input -lco ENCODING=UTF-8
Note: LATIN1 should work too instead of ISO-8859-1.
In Windows, do NOT set the SHAPE_ENCODING, ogr2ogr does not recognize ISO-8859-1, nor LATIN1.
6
As Jason pointed out you've got a mismatch between the character set of the data and the character set of locator. It's probably the geodb that doesn't match the data and vice versa. Make sure the geodb is in UTF-8.
If you want to retain the accented characters and still get 100% match with the unaccented spellings, store unaccented versions of the names in ...
6
That isn't garbage, that's UTF-8 being displayed as CP1252.
This Stackoverflow topic may help you on your way, as may this one (if you're in Python).
3
Quoting from http://www.gdal.org/ogr/drv_shapefile.html :
An attempt is made to read the LDID/codepage setting from the .dbf
file and use it to translate string fields to UTF-8 on read, and back
when writing. LDID "87 / 0x57" is treated as ISO8859_1 which may not
be appropriate. The SHAPE_ENCODING configuration option may be used to
override the ...
3
You might try creating a file with the same base name as the shapefile but with a .cpg extension, containing a single line of text identifying the encoding, be it UTF-8 or something else. I am not sure if this will resolve the problem with QGIS, but I have had some success with this method to force multi-byte characters in attribute data to be interpreted ...
3
There is something fundamentally wrong with GDAL's shapefile encoding detection feature. In latest QGIS 1.9 builds, you can ignore GDAL's shapefile encoding by going into the Options window and checking the [ ] Ignore shapefile encoding. You can then set the correct encoding in the layer property window.
See ...
2
Shapefiles get their codepage either from the .dbf or from the .cpg file.
The .dbf file has a byte that represents DBF Language Driver ID. There's some discussion about these in an archived ArcGIS Desktop forum on forums.esri.com. There's a Microsoft Knowledge Base article Understanding Code Pages in Visual FoxPro which lists 19 DBF Language Driver IDs and ...
2
You are correct: the string "Bălți" would have been represented in UTF-8 format inside OpenSteetMap. If mis-interpreted as iso-8859-1 it comes out "BălÈ›i".
You can verify this at my favorite unicode tool: http://www.ltg.ed.ac.uk/~richard/utf-8.cgi
You can get proper XML dumps of OpenSteetMap data here:
http://download.geofabrik.de/osm/europe/
And then ...
2
Apparently, the issue has been fixed in the development build of QGIS.
http://hub.qgis.org/projects/quantum-gis/repository/revisions/75dc85b4d652116814873bb7674cab15ce6cde66
Installed QGIS version 1.9.0-Master and tried it out, works fine.
2
87 (0x57) is the ANSI code page ID according to this reference.
dat=open(dbf,'rb').read(30)[29:]
id=struct.unpack('B',dat)[0]
print id,hex(id),chr(id)
This prints 87 0x57 W on some random dbf I tested. For '949' you should be looking for 78 0x4E.
Edit: Below is a look up table (dict) for the code pages copied from the above reference:
lut={
1 ...
1
As a workaround you may use .cpg-files with encoding as additions to .shp (more on it here). Or hard-write encoding to the .dbf-header using Libre Office: just open it in Calc (it will ask you for encoding) and save it.
1
SOLUTION A for utf-8 encoded shapefiles
Supposed you are working on Windows, look for qgis.bat in bin directory, and open it with a text editor.
after the first line, insert
SET SHAPE_ENCODING=UTF-8
and save.
Qgis 1.8.0 has some problems with encoding, using your computers default font instead of the one you specify. It is fixed in Qgis Master (1.9.0) by ...
1
Can confirm this in python console:
>>> layer = qgis.utils.iface.activeLayer()
>>> layer.dataProvider().encoding()
PyQt4.QtCore.QString(u'UTF-8')
>>> layer.setProviderEncoding(u'latin9')
>>> layer.dataProvider().setEncoding(u'latin9')
>>> layer.dataProvider().encoding()
PyQt4.QtCore.QString(u'UTF-8')
1
Your shapefile is most probably in UTF-8, but qgis 1.8.0 has a bug in correct encoding of shapefiles. Until this is solved, you can try this workaround in Windows:
Browse to C:\Programs\Quantum GIS Lisboa\bin on Windows XP or C:\Programs (x86)\Quantum GIS Lisboa\bin on Windows 7. Look for qgis.bat and open it with a suitable editor (I have installed ...
1
Here here is my proposal:
TimeStamp and TimeSpan elements are actually defined in the OGC KML version 2.2 XSD, hopefully a glimpse at it helps you to devise a quick fix or
pick up a good XML validation tools and probe your KML files against (Google extensions to OGC) KML's XSD.
1
I got this error (Invalid byte 2 of 2-byte UTF-8 sequence) when validating the SLD in GeoServer 2.1.3, where the Danish charater Å was used as a ogc:Literal. Editing the SLD file in a text editor kind of fixed the problem. The SLD editor in Geoserver now writes Å as Ã…. But my styling is working.
1
This isn't much of an answer I'm afraid, but the File Geodatabase format expects UTF16 strings, so that might be the cause of your problem. The developer documents say:
In general wstring is used for all character parameters in the API.
This implies that we are using wchar_t. But in addition to that, we
are using UTF-16 encoded characters in the ...
1
You mention that you are converting SHP-UTF8 to SHP-ANSI. Technically you only need to do a DBF-UTF8 to DBF-ANSI conversion. Perhaps by focusing on this problem you can save a lot of time.
If you have a copy of ArcPad (you can try the 20 minute evaluation) you can try the following steps:
Start ArcPad with an empty map
Add a Shapefile that's currently in ...
Only top voted, non community-wiki answers of a minimum length are eligible
