Take the 2-minute tour ×
Geographic Information Systems Stack Exchange is a question and answer site for cartographers, geographers and GIS professionals. It's 100% free, no registration required.

Is there a way to determine the charset used for a given shapefile?

share|improve this question

4 Answers 4

up vote 8 down vote accepted

Trial and error. Try to open the .dbf file with Ms Excel or with OpenOffice using different setting until you get everything right.

Look at this post for more clues: http://stackoverflow.com/questions/319095/how-do-i-determine-the-character-set-of-a-string

share|improve this answer
    
If nothing is known about the encoding, it's worthwhile to try latin1 or UTF-8 first. –  krlmlr Mar 29 at 16:00

There are two ways for programs to determinate the character set for a shapefile.

share|improve this answer
    
+1 That link to a dBase file format page is great. However, AFAIK, codepages were never included in the dBase III format. The reference there is to a FoxPro extension of the format, which suggests not all .dbf files are going to have codepage info in them (or, if they do, it might be a result of garbage bytes appearing in a free area of the header). But if you can dig this information out, it's still a good start for a trial-and-error search. BTW, welcome to our site! –  whuber Jul 20 '11 at 13:33
    
Some python dbf libraries can read the codepage if information is there. –  Paulo Scardine Aug 6 '12 at 19:19

Another table for converting 29th byte of *.dbf to code page: http://webhelp.esri.com/arcpad/8.0/referenceguide/index.htm#locales/task_code.htm

share|improve this answer

The file utility is able to guess the encoding of a text file. Use ogr2ogr for a conversion that preserves the original encoding if there is no .cpg file:

ogr2ogr -f CSV file.csv file.dbf
file file.csv

Example output:

file.csv: ISO-8859 text

I have tested it with two of the most frequent encodings, UTF-8 and latin1. Works out of the box in Ubuntu, not sure about OS X. I'm not aware of a file utility on Windows.

NOTE: As soon as there is a corresponding .cpg file that indicates the encoding, ogr2ogr will honor it and the output will be in UTF-8. But if the CSV output looks right, you know that the information in the .cpg file is accurate.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.