Shapefiles *.dbf data, encoding problem

Question

I have to work with the Shapefiles, and all is fine except data in *.dbf

When I have first deserialized file, I have thought, that I'm doing smth wrong, but when any soft like (QGIS, SAGA, ESRI shapefiles reader, GRASS GIS are reading in the same result, I have thought that source file was in the incorrect format from the beggining).

Any soft shows the next encoding from *.dbf data:

http://s15.postimg.org/lji3hjywr/image.png

Then, I've read about dBase, which is used in Shapefiles and the default encoding (LATIN1), but the docs are very confusing, because somewhere I've read about LATIN1, but at other website there is info, that for Russian maps the default encoding is: OEM866.

I've tried to reencode, but again no success.

I've downloaded the shapefiles from different sources (from bbbike and other web repositories of maps), but the same result from different resources.

My aim is, to work normally with the Moscow, Russia map, which is available here: http://download.bbbike.org/osm/bbbike/Moscow/

Answer 1

The problem was because, originally data in *.dbf file for Russian langauge exactly are often saved in OEM-866 encoding format.

QGIS seems to be are marking convertions to the Unicode.

For e.g. if you want to get the normal text (in UTF-8 for e.g.) from *.dbf you can do the next (sorry, that I'm provding only C# code, but I think in other programming languages operations are similar):

static string Convert866ToUTF8(string value)
{
    var resultValue = Encoding.UTF8.GetString(Encoding.GetEncoding(866).GetBytes(value));
    return resultValue;
}

It has hepled me in developing some cleint application with C# langauge.

Answer 2

Currently, you can use any encoding in Shapefiles and the most common is Unicode. (and OpenStreeMap data use Unicode)

I have no problem to open the Moscow shapefiles in QGIS and the Cyrillic characters appears:

If you look at the properties of the layer, you can see that the encoding is 'UTF-8' (Unicode)

-> you know the encoding of your shapefile and you can change the encoding if you want.