5

So I have a 'cpg' file with my 'in_file' shapefile data (definitely set to 'utf-8'). I was using 'meta' to create the same schema/crs as 'in_file'...though it runs, my text fields are borked on anything that isn't ascii (note the cjk - 'ZH_CN' - is coming back as codepoints).

ex:

    {'geometry': {'type': 'MultiPoint', 'coordinates':
    [(13531245.475704141, 2886003.2689278126)]}, 'type': 'Feature', 'id':
    '0', 'properties': OrderedDict([(u'EN_US', u'Taipei City'), (u'ZH_CN',
    u'\u53f0\u5317\u5e02'), (u'ID', 1668338.0), (u'DE_DE', u'Taipei
    City'), (u'ES_ES', u'Taipei City'), (u'FR_FR', u'Taipei City'),
    (u'JA_JP', u'Taipei City'), (u'KO_KR', u'Taipei City'), (u'PT_BR',
    u'Taipei City')])}

My first thought was to change:

with fiona.open(in_file, 'r') as input:

to:

with fiona.open(in_file, 'r', encoding='utf-8') as input:

...but that resulted in:

UnicodeEncodeError: 'latin-1' codec can't encode characters in
position 0-2: ordinal not in range(256)

Reading it without the encoding seems to be the start of my problem...however I was surprised that the error message arose when I did the above.

def process_file(self, in_file, out_file, compare_file, dist):
    # open input file and compare file
    # loop over each
    with fiona.open(in_file, 'r') as input:
        meta = input.meta
        # The outFile has the same crs, schema as inFile
        with fiona.open(out_file, 'w', **meta) as output:
            with fiona.open(compare_file, 'r') as compare:
            # Read shapely geometries from file
            # Loop through all shapely objects
            # type(item) = 'dict'
                for item in input:
                    geom = item['geometry']
                    my_shape = shape(geom)

                    # check if it is a multipoint or point
                    if geom['type'] == 'MultiPoint':
                        # break Multipoints into points
                        # i.e. <class 'shapely.geometry.multipoint.MultiPoint'>

                        for pt in my_shape:
                            single_point = self.move_point(pt, compare, dist)
                            mpoint = MultiPoint([(single_point.x, single_point.y)])
                            mpoint_for_merge = shape(mapping(mpoint))
                            # write to outfile - WHERE EVERYTHING NOW WORKS
                            output.write({'geometry':mapping(mpoint_for_merge), 'properties': item['properties']})

                    elif geom['type'] == 'Point':
                        # return of move_point is a shapely geom
                        # i.e. <class 'shapely.geometry.point.Point'>
                        my_shape = self.move_point(my_shape, compare, dist)
                        # write to outfile
                        output.write({'geometry':mapping(my_shape), 'properties': item['properties']})

                    else:
                        raise ValueError('unhandled geometry type: ' + repr(geom.type))
5

Fiona read ok utf-8 files without add nothing.

Try encoding='utf-8' only when output with fiona.

So change...

with fiona.open(out_file, 'w', **meta) as output:

To

with fiona.open(out_file, 'w', encoding='utf-8', **meta) as output:

And all must go OK.

2

Same problem occured to me when working with GeoPandas. The solution @Juanma Font has written works with GeoPandas as well:

geodataframe.to_file(filename, driver='GeoJSON', encoding='utf-8')

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.