2

I have ~10 mln lat/lon points, and I need to extract city and zip code from them.

I can do this with OpenStreetMap using the Nominatim API (using something like http://nominatim.openstreetmap.org/reverse?query) through geopy.

The problem is that the API takes only one lat/lon pair at a time, which, with 1 second delays between each query, would take ~3000 hours in total or maybe more.

Is there a similar API where I can submit a batch request with a few thousand pairs at the same time? Or something that I can (easily) set up locally that would give me the same information?

Also, I know that it's possible to set up Nominatim locally. However it seems quite complex, and I need only a part of the data, not everything. And it's a one-time job, so probably I won't need this in the future.

Some extra details: the lat/lon pairs are from Russia, the results I get from OSM are very good, so I'd like to have the same or something similar.

Improve this question
2
  • Are you open to downloading osm data? You could download osm Russia, then do a near analysis for city (place) and postal code (postal_code).
    – Pura Vida
    May 8, 2016 at 17:20
  • Yes, it's not a lot of data, a few GBs. Do you know how one would process the osm data to get the needed information? Thank you
    – Alexey Grigorev
    May 8, 2016 at 18:40

2 Answers 2

Reset to default
4

I discovered that the complexity of installing own instance of Nominatim can be mitigated by using a docker container where the set up script is already prepared.

I used a docker script available at https://github.com/helvalius/nominatim-docker. Then the installation is just the following: 

git clone https://github.com/helvalius/nominatim-docker.git
cd nominatim-docker
docker build --build-arg OSM=http://data.gis-lab.info/osm_dump/dump/latest/RU.osm.pbf -t nominatim
docker run nominatim

Where OSM can be replaced with an URL to any other OSM dump.

And after ~10 hours of waiting (in case of Russia), you'll have your own instance of Nominatim. Once the processing is done, the docker container can be removed along with all Nominatim dependencies such as PostgreSQL.

This is not the simple solution I wanted, but it's okay and solves the problem (although slower than I'd like!). But I still would like to know if there are simpler and faster options - in case I need this again in future.

Improve this answer
2
  • Couldn't you just grab OSM Russia and extract the city boundaries? Loop through the coordinates in Python for instance (Shapely?) and save the corresponding city. These you could then correlate with some zip code list? Would require a little bit of scripting but this could be quite quick.
    – Timothy Dalton
    May 24, 2016 at 21:11
  • Do you know if there's an example how to do it?
    – Alexey Grigorev
    May 25, 2016 at 9:54
2

If you want to stick to OSM then you will need Nominatim to process it, or one of the other OSM geocoders. Address data in OSM is build up hierarchically. Just analyzing the raw data won't help, you have to build up an address hierarchy. This is what OSM geocoders usually do. Or take a look at alternative Nominatim instances / third-party providers.

Improve this answer
2
  • If I already have nominatim running locally (see my answer), is there a way to make it batch process lat/lon pairs? Even locally it takes forever when issuing one request per pair
    – Alexey Grigorev
    May 9, 2016 at 17:46
  • 1
    I don't know how this can be achieved, sorry. But I'm pretty sure it is possible somehow. All in all you "just" need to use the same API the HTTP interface uses.
    – scai
    May 9, 2016 at 19:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.