1

I'm attempting to import the 45GB (compressed) planet-latest.osm.bz2 file downloaded from ftp://ftp.spline.de/pub/openstreetmap/planet/planet-latest.osm.bz2 but I receive parsing errors. As of now I have had two separate files fail in the same way - downloaded approximately 1 month apart.

The import command is as follows:

nohup osm2pgsql -l --bbox -180,-59,-26,85 -U gis -c -I -v --unlogged --drop -s data/planet-latest.osm.bz2 &

and the errors I receive are:

...
Entity: line -1123806515: parser error : Attribute user redefined
er="Schandlers" uid="51:48Z" version="1" changeset="29" user="OrcaDan" uid="411"
                                                                               ^
Entity: line -1123806515: parser error : Attribute uid redefined
er="Schandlers" uid="51:48Z" version="1" changeset="29" user="OrcaDan" uid="411"
                                                                               ^
data/planet-latest.osm.bz2 : failed to parse
Error occurred, cleaning up

This is a real pain as the file takes several days to download and the import can be running for several more days before the error is thrown.

I'm trying to understand what is causing the failure and what I can do to avoid it. My only theory is that the file is getting updated on the server as I am downloading it and I somehow get an invalid combination of two files. However I'm not really sure this is possible...

My question has three parts:

  1. what is causing the error I'm seeing?
  2. is there a faster way to validate a planet file than waiting for the import to fail?
  3. can anyone point me to a recent and known-good file, if the file itself is the problem?

osm2pgsql version: osm2pgsql SVN version 0.82.0 (64bit id space), installed from Ubuntu 14.04 repos.

closed as too broad by PolyGeo Mar 5 '16 at 1:35

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 1
    It's not an answer to your question, but as a workaround you could use a pbf planet dump instead of xml. The files are smaller, and faster to parse. – Jesse Crocker Sep 10 '15 at 22:01
  • 1
    Also trying a newer version of osm2pgsql might help. Version 0.82 is over two years old. – scai Sep 11 '15 at 7:23
  • @JesseCrocker thanks for the tip, it seems pretty obvious now you mention it... I'm not sure why the process I'm following uses .osm instead of .pbf but it was created several years ago and maybe things were different then. – tomfumb Sep 11 '15 at 15:10
  • As per the Tour there should be only one question asked per question. – PolyGeo Mar 5 '16 at 1:35