osm2po to import whole planet

Question

My point is to use pgrouting with planet data. osm2pgrouting fails at importing whole planet, so I tried osm2po. Looked fine, it generated sql file in around 2,5h. After importing it to the db though I found there are many ways excluded from the db. Looking closer at osm2po logs:

INFO   46,421,095 of 235,414,299 ways extracted - 25,018M

So I commented out every wtr.finalMask entry in osm2po.config file. It helped a bit:

INFO   74,660,756 of 235,414,299 ways extracted - 15,761M

but still there are many ways missing. So I uncommented this one line:

disablePreFilter = true

But then right before way extracting I got this:

INFO   2,362,909,765 of 2,362,909,765 nodes extracted - 29,114M
FATAL  Exception at Thu Jul 02 07:34:50 CEST 2015- 8,932M
       Message: java.lang.RuntimeException: java.lang.ArrayIndexOutOfBoundsException: Array index out of range: -2147483648
       Class: java.lang.RuntimeException
       Stacktrace:
       de.cm.osm2po.converter.OsmXmlParser.parse(OsmXmlParser.java:83)
       de.cm.osm2po.Main.runTiler(Main.java:214)
       de.cm.osm2po.Main.main(Main.java:159)

After reading all the answers here I understand osm2po doesn't extract ways with tags it doesn't know. Do I understand correctly that disabling prefilter should extract all the ways? If so, is there any way to configure osm2po to build sql file with all the ways from osm?

I'm using osm2po 5.0.0 version. It was executed with:

java -Xmx32G -jar osm2po-core-5.0.0-signed.jar cmd=tjsp tileSize=x,c ./planet-140522.osm

on system having 64GB RAM. Map file is a little bit old, but I need exactly this version. My osm2po config is there: osm2po.config

Example of missing ways (or sections, not being sure what exactly "way" in osm means) is there: part.of.osm.file (this is part of newer osm file, not the planet, to be able to grep it reasonably fast, but it looked similar when I grepped it on planet file) leaving in the db only these 3 rows:

osm_id;osm_source_id;osm_target_id
111729593;252343523;1370502399
111729593;1370502399;252343594
111729593;252343594;252343601

Answer 1

Which osm2po version did you use and how (parameters?) did you start it? Hmmm... 74,660,756 of 235,414,299 sounds great. The rest are buildings, places, areas, boundary segments etc. and no routable roads or paths.

May I see the complete log-file to find the ArrayIndexOutOfBounds-Exception? By the way: The error says, that one array has been populated with more than 2 Billion elements!!! To bypass this issue, use the tileSize-parameter. (tileSize=30x30,1) e.g.

But does it really make sense to mix all that data (polygon-ways and line-ways) into one table? - I don't think so. The disablePreFilter option does not influence the outcome. It only deferres the filtering and leaves it to the next process step. Useful, if you want to collect infos defined in relations before, routes e.g. And most important: osm2po does a lot more than just producing geometries. Its major task is to correct ways and to find intersections (crossings) in the network. Meaning, buildings, areas, etc. may become part of the road network.