4

I'm trying to build a GUI that allows replacement / update of the PostGIS database that holds my OSM extract (which I use for rendering, typical mapnik / tile server arrangement).

I'd like to show the user when the data that is already loaded was "current" to. That is, if my PBF extract (from http://download.geofabrik.de/) "contains all OSM data up to 2016-07-06T19:29:02Z", then I'd like to show the "2016-07-06T19:29:02Z" part in the GUI.

However I don't know how to get that from the database.

From http://wiki.openstreetmap.org/wiki/PBF_Format I see there is a osmosis_replication_timestamp field in the header, but I don't see how to retrieve that with just osm2pgsql.

At this stage, I'm thinking about either to "shell out" to something like osmosis, or trying to parse the HeaderBlock myself. I'd then have to store that (perhaps in the database, perhaps in my local state, perhaps as a file like the mod_tile approach). Both options suffer from lack of integration with the import process (e.g. if someone does a command line import using osm2pgsql, my separate state won't reflect that import), so I end up having to track import date / timestamp pairs. Both osmosis and a parser are pretty heavy for one timestamp...

Is there a way to get the replication timestamp directly from the results of an osm2pgsql import?

Improve this question
2
  • windows or linux?
    – kttii
    Jul 20 '16 at 19:02
  • Windows in this case. Not my preferred tool, but its what the end-users have.
    – BradHards
    Jul 20 '16 at 22:07
3
+50

Using OSMConvert, the wiki shows how you can extract/display the timestamp from the file header:

osmconvert yourfile.pbf --out-timestamp

This could be included in your update process and piped into the database for your GUI to read later.

you can download it here

Using OSM2PGL, the wiki describes how the timestamp is retrieved with each object by editing the default.style file (or create a new one). A new column is created in the database for the extract. Simply unremark the timestamp at line 78 and use the following command without --hstore and include the style file reference:

osm2pgsql --extra-attributes -S default.style -d databasename yourfile.pbf

You don't have to get that from the database. When you are downloading the PBF you can retrieve the timestamp at that time.

If you are using wget then you can use the -S option (server response) which downloads and produces this result containing the last modified date:

wget response

or access the file system in the programming language of your GUI to get the last modified date:

enter image description here

Improve this answer
7
  • The problem is that there can be some (large) lag from the date that the extract is taken (the up to date in the question), to the date that it ends up on the disk of the system I'm doing the import on. So this won't solve the problem.
    – BradHards
    Jul 7 '16 at 20:28
  • @BradHards did you try wget -S ?
    – kttii
    Jul 11 '16 at 13:39
  • It does not match. -S preserves file modification date on the server (e.g. 2016-07-11 00:24 fiji-latest.osm.pbf, using UTC TZ), but the web page says all OSM data up to 2016-07-10T19:29:02Z. In any case, I don't have that much control over how the user downloads the data, or copies it (which will also change modification date) before it gets to my application.
    – BradHards
    Jul 11 '16 at 21:48
  • @BradHards Updated answer
    – kttii
    Jul 20 '16 at 19:20
  • 1
    Yes, the wiki states that a timestamp column is created in the database, indicating you would have to process the entire file, whereas OSMConvert will just retrieve the header timestamp, which you could also pipe into the database as a separate command. Updated answer.
    – kttii
    Jul 21 '16 at 14:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.