osm2pgsql import planet.osm.bz2 slowness

Question

I am trying to import whole planet data to my i7 3770k + postgres on SSD + 16gb RAM in order to render my custom tiles.

But the process is painfully slow and multi core is not helping (though you have specified)

osm2pgsql -d planet -U osm --slim -C 8000 --number-process 4 planet-120704.osm.bz2

From htop what I can see is osm2pgsql take 100% of 1 core and postgres takes only 10% of cpu.

and current prompt is:

Processing: Node(208320k 147.0k/s) Way(0k 0.00k/s) Relation(0 0.00/s)

Q: Is my computer too slow? Any method to speed up (without complexing too much hardware)?

geomajor56 · Answer 1 · 2012-08-13 14:41:14Z

up vote 1 down vote

I switched to imposm http://imposm.org/docs/imposm/latest/ for the same reason and everything worked out great. I was trying to install osm-bright for mapbox https://github.com/mapbox/osm-bright/ as all of the desired styling was present. Hope this helps out.

answered Aug 13 '12 at 14:41

geomajor56
61419

	Did the same. But don't underestimate postgres config ;) – Martin Aug 13 '12 at 14:47
	Martin, I'm a newb and always afraid to touch any config file. What can be changed in the config file? – geomajor56 Aug 13 '12 at 15:03
	try to set these 2 values as I have them. Makes pg use up to 8gb ram, which is the peak of efficency and performance in version 8.xx, shared_buffers = 4096MB, effective_cache_size = 8128MB - further don't forget kernel.shmmax, otherwise pg will not start. ;) That's it – Martin Aug 13 '12 at 15:07
	nice, thanks for the info – geomajor56 Aug 13 '12 at 17:50

1 Answer