I'm working in QGIS with a Government of Canada shapefile of all 338 federal electoral districts.

The encoding is UTF-8.

In the attribute table, all of the em dashes ("—") that should appear in the riding names have been replaced by question marks.

I've used the field calculator to replace the question marks with the proper em dashes, but those changes won't save properly. If I close and re-open the attribute table, the question marks re-appear.

But if I replace the question marks with non-special characters (e.g. "AAA"), there doesn't seem to be an issue. So the problem seems to be tied to the em dash.

Any ideas on what might be going on?

  • 1
    The dBase-III format predates UTF, so shapefiles are always going to have difficulty with non-ASCII codepoints. If you convert to a codepage-safe format, the data just might work. – Vince Jun 27 '19 at 17:49
  • Good to know. Thank you! Is there a particular format you would recommend? – mcsoma Jun 28 '19 at 13:59
  • File geodatabase utilizes UTF-16, so it wouldn't get tangled up, but just about every other format in creation, including CSV, is more UTF-resilient than shapefile. – Vince Jun 28 '19 at 14:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.