Given a huge shapefile which needs desolving by one attribute field to reduce redundancy. The GDAL dissolve functionality does that quite swiftly but the output attributes change the encoding so äöü become to unusable ?squares.

What i've tried so far:

  • Adding a hugeshpfile.cpg that additionaly declares UTF-8: ?squares
  • Doing the detour via Postgis: same result
  • The native QGIS dissolve function freezes no matter if shapefile or postgis
  • Doing it via SQL query in pgadmin results in a different error messages.

While trying around i didn't encounter a possibility to influence the encoding. How can that be achieved or am i missing out on some format constraints?

  • I'm not sure what the detour in postgis means. Hopefully not the same as this and it helps you out - postgis.net/docs/ST_Union.html – user92055 Mar 5 '17 at 18:36
  • Could you create a small dataset for testing? Two or three features with an attribute to use for dissolve and containing äöü should be enough. – user30184 Mar 5 '17 at 19:45
  • @user30184: Testdataset 35-3_35-4_merged.shp is a part of the merged subset. *_dissolved is from native qgis with correct encoding; *_diss_gdal is made with ST_Union – maxwhere Mar 5 '17 at 20:17
  • What shapefile presents the original data? What attribute should I use for dissolving? Is it 35-3_35-4_merged.shp and attribute "LAYER"? Or have I even understood right what you try to do? – user30184 Mar 5 '17 at 20:42
  • The original data comes in tiles of 2,5km². 35-3_35-4_merged.shp represents two of those and i wand to dissolve them by the field 'FMZK_ID'. If i do this with qgis native function it takes forever, if i do it with gdal disssolve polygons it destroys the encoding. – maxwhere Mar 6 '17 at 13:07

I solved my problem by avoiding it with a workaround that replaces all non-ascii characters in the affected fields with their compatible equivalents by using a script i derived from this answer:

def encode(shapefile):
    d = {'ä':'ae','ü':'ue','ö':'oe','ß':'ss',
    driver = ogr.GetDriverByName("ESRI Shapefile")
    dataSource = driver.Open(shapefile, 1)
    layer = dataSource.GetLayer()
    for feature in layer:
         field = feature.GetField("LAYER")
         pattern = re.compile("([" + "".join(d.keys()) + "]).")
         repl = pattern.sub(lambda x: d[x.group()], field)
         feature.SetField("LAYER", repl)

Surely this function could be better designed and although it does the job I'm still open to suggestions concerning the encoding since this workaround shouldn't have been necessary in the first place.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.