0

I would like to ask if there's an easy way to query OSM changesets' information like user, timestamp, comment and hashtags in that changeset. In particular I would like to retrieve all the user names of the users which modified an area saving their changesets with a particular hashtag (similarly to default hashtags in HOT projects https://tasks.hotosm.org/)

My idea is to download the OSM data relative to my area of interest (in this case a country, taken from https://osm-internal.download.geofabrik.de/africa/senegal-and-gambia.html#)

So what I did until now is to download the following files from GeoFabrik.de:

  • senegal-and-gambia-latest-internal.osm.pbf

  • senegal-and-gambia-internal.osh.pbf

And I tried to run the following code (taken from https://oslandia.com/en/?s=timeline+handler) using Python and pyosmium library:

import osmium as osm
import pandas as pd

class TimelineHandler(osm.SimpleHandler):
    def __init__(self):
        osm.SimpleHandler.__init__(self)
        self.elements = []

    def add_elements(self, e, elem_type):
        self.elements.append([elem_type,
                              e.id,
                              e.version,
                              e.visible,
                              pd.Timestamp(e.timestamp),
                              e.uid,
                              e.changeset]) #only changeset id

    def node(self, n):
        self.add_elements(n, 'node')

    def way(self, w):
        self.add_elements(w, 'way')

    def relation(self, r):
        self.add_elements(r, 'relation')

tlhandler = TimelineHandler()
tlhandler.apply_file("senegal-and-gambia-latest-internal.osm.pbf")
colnames = ['type', 'id', 'version', 'visible', 'ts', 'uid', 'chgset']
elements = pd.DataFrame(tlhandler.elements, columns=colnames)

This doesn't solve my problem as I'm searching for changesets' information and not OSM data itself.

Maybe I'm missing a snippet to be added in the handler similar to:

    ...

    def changeset(self, c):
        self.add_elements(c, 'changeset')

    ...

I tried to add this snippet, but the handler gives no result.

Actually I'm wondering if I'm missing something in the data structure of osm/osh .pbf files, something on pyosmium or if the changeset information I'm searching for is contained in such files.

If the task cannot be performed with pyosmium and GeoFabrik, how can I do such operation?

2

Changeset data is provided in a separate Changeset Dump: Latest Weekly Changesets (2.5 GB) on planet.openstreetmap.org (not aware of extracts).

Beside the database import tools mentioned in the Wiki, Osmium Tool can handle the changeset file, see Working with changesets (don't know about pyosmium).

  • I have problems in finding a way to play osmium on Windows 10. In any case it would be nice if it could be find a way to make pyosmium work on this, as I have the Python environment up and running. Extracts - as PlanetOSM states - are available at GeoFabrik. I cannot find references to hashtags at osmcode.org/osmium-tool/manual.html#working-with-changesets (are they embedded in the comments?) – Michael Montani Dec 12 '18 at 14:47
  • 1
    Geofabrik doesn't offer Changeset dumps, and the extracts they provide don't include changeset metadata with hashtags. Please refer to the links cited by Norbert. Hashtags are stored as tags on changeset level, see wiki.openstreetmap.org/wiki/Changese for details. – mmd Dec 12 '18 at 22:04
  • On the tangential question of "osmium on Windows 10", osmium should run under Windows subsystem for Linux on Windows 10.. – SomeoneElse Dec 13 '18 at 15:17
  • Seems like pyosmium does support changesets with osmium.SimpleHandler [1]: "The following data types are recognised: node, way, relation, area and changeset." See also "class osmium.osm.Changeset" [2]. Hashtags will be in the "comment" tag under "tags", for example see changeset 65448040 [3]. [1] docs.osmcode.org/pyosmium/latest/ref_osmium.html [2] docs.osmcode.org/pyosmium/latest/ref_osm.html [3] openstreetmap.org/api/0.6/changeset/65448040 – Norbert Renner Dec 13 '18 at 17:02
0

Eventually this code answered exactly my question:

import osmium as osm
import pandas as pd

class ChangesetHandler(osm.SimpleHandler):
    def __init__(self):
        osm.SimpleHandler.__init__(self)
        self.elements = []

    def add_elements(self, e, elem_type):
        comment=e.tags.get("comment")
        if type(comment) is not type(None) and "TheHashtag" in comment: 
        self.elements.append([e.id, #changeset id
                              e.user]) #username not user id

    def changeset(self, r):
        self.add_elements(r, 'changeset')

tlhandler = ChangesetHandler()
tlhandler.apply_file("changesets-latest.osm")
colnames = ['id','user']
elements = pd.DataFrame(tlhandler.elements, columns=colnames)
users=elements.user.unique()
print("Found %d users\n"%len(users))
print(users)

This code returns a list of all the usernames of the users having commented their changeset with certain hashtag. For sake of simplicity I supposed users didn't change their usernames in this time, otherwise one can consider to call e.uid instead of e.user in the handler.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.