1

What I want to do:

I want to get road graph (ways and their relations) from OSM file to write my own routing program.

What I tried to do:

I tried to use OSMtoRoadGraph, but it returns me a UnicodeDecodeError

(base) C:\Windows\system32>python 
C:\Users\Horseman.mini\Desktop\OsmToRoadGraph-master\run.py -f 
C:\Users\Horseman.mini\Desktop\OsmToRoadGraph-master\data\siberian-fed-district-latest.osm_01_01.osm -n c
starting convert_osm_to_roadgraph
selected network type: car
accepted highway tags: {'pedestrian': {'path', 'secondary_link', 'footway', 'secondary', 'tertiary_link', 'pedestrian', 'track', 'primary', 'tertiary', 'primary_link', 'unclassified', 'road', 'steps', 'service', 'living_street', 'residential'}, 'bicycle': {'secondary_link', 'secondary', 'tertiary_link', 'cycleway', 'track', 'primary', 'tertiary', 'primary_link', 'unclassified', 'road', 'path', 'service', 'living_street', 'residential'}, 'car': {'secondary_link', 'motorway', 'motorway_link', 'secondary', 'trunk', 'tertiary_link', 'trunk_link', 'primary', 'tertiary', 'primary_link', 'unclassified', 'service', 'living_street', 'residential'}}
opening file: C:\Users\Horseman.mini\Desktop\OsmToRoadGraph-master\data\siberian-fed-district-latest.osm_01_01.osm
starting read_file
starting _read_ways
0%..Traceback (most recent call last):
File "C:\Users\Horseman.mini\Desktop\OsmToRoadGraph-master\run.py", line 65, in <module>
convert_osm_to_roadgraph(filename, network_type, options)
File "C:\Users\Horseman.mini\Desktop\OsmToRoadGraph-master\utils\timer.py", line 11, in wrapper
result = function(*args, **kwargs)
File "C:\Users\Horseman.mini\Desktop\OsmToRoadGraph-master\run.py", line 25, in convert_osm_to_roadgraph
nodes, ways = osm.read_osm.read_file(filename, configuration)
File "C:\Users\Horseman.mini\Desktop\OsmToRoadGraph-master\utils\timer.py", line 11, in wrapper
result = function(*args, **kwargs)
File "C:\Users\Horseman.mini\Desktop\OsmToRoadGraph-master\osm\read_osm.py", line 11, in read_file
ways, found_node_ids = _read_ways(PercentageFile(osm_filename), parserHelper)
File "C:\Users\Horseman.mini\Desktop\OsmToRoadGraph-master\utils\timer.py", line 11, in wrapper
result = function(*args, **kwargs)
File "C:\Users\Horseman.mini\Desktop\OsmToRoadGraph-master\osm\read_osm.py", line 23, in _read_ways
parser.parse(osm_file)
File "C:\ProgramData\Anaconda3\lib\xml\sax\expatreader.py", line 111, in parse
xmlreader.IncrementalParser.parse(self, source)
File "C:\ProgramData\Anaconda3\lib\xml\sax\xmlreader.py", line 123, in parse
buffer = file.read(self._bufsize)
File "C:\Users\Horseman.mini\Desktop\OsmToRoadGraph-master\osm\xml_handler.py", line 25, in read
data = self.f.read(size)
File "C:\ProgramData\Anaconda3\lib\encodings\cp1251.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x98 in position 993: character maps to <undefined>

I think the problem is that OSM uses utf-8 format, but I see in the error this line at the very end "C:\ProgramData\Anaconda3\lib\encodings\cp1251.py", line 23, in decode so unicode error is predictable. If no ways to solve unicode problem or any ways to fix this problem using this soft - I haven't found any other ways to import roads graph from OSM.

2 Answers 2

1

osm4routing2 is a fast and simple way of extracting a graph from OSM .pbf files, written in Rust: https://github.com/Tristramg/osm4routing2

1
  • Incredible library!!!
    – hans glick
    Jul 28 at 21:29
0

I'm the maintainer of this project and would like to help you out.

UPDATE I fixed the issue in version 0.42.1

I tried to reproduce the problem locally with data I downloaded from geofabrik.de (I guessed you did the same. I got siberian-fed-district-latest.osm). However, on my Mac/Linux the reported issue doesn't appear. You can also open an issue in the Github repo if you rather want to discuss this there.

It would be great if you could somehow provide a sample of the file that produces the problem. I also have a potential way to fix it, but can't really test it, since I can't reproduce the problem. But if you want. You could do the following:

  • go to osm/ and open the file xml_handler.py
  • replace l. 18 with self.f = open(filename, encoding="utf-8")
  • try to run it again (and please tell me if it worked or not)

Lastly, I have a re-implementation (that is not yet finished) in Rust which not have this issue at all and we could try this out.

Looking forward to hear back.

Cheers, Andreas

7
  • I got pbf format, cutted it using osmconvert and then reformated to .osm. I used this files: yadi.sk/d/X71AsTwcIlNtrg
    – Alexey
    Jan 19, 2020 at 11:29
  • Yes, replacing open(... actually worked well, thank you! I will try to use it and accept your answer later. But how do I read this format? I see that I can read it as txt, but how to put it in csv?
    – Alexey
    Jan 19, 2020 at 11:59
  • glad it worked. So, there is a description of how the output format looks like in the README. But what exactly would you need as a csv file? Do you have specific software in mind to further process the data? I don't directly see how this can be easily transformed into a csv file, but maybe you can explain what you need?
    – Andreas G
    Jan 19, 2020 at 12:33
  • Btw. I updated the repository with the bugfix.
    – Andreas G
    Jan 19, 2020 at 12:36
  • Thank you! Could you tell me one extra thing, in readme line <s> <t> <length> <street_type> <max_speed> <bidirectional> - what does <s> and <t> mean? Is it like from node - to node ?
    – Alexey
    Jan 19, 2020 at 15:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.