6

I got a problem with a twitter analysis i am doing. What i want to do: Based on the country i want to make a nice graphic about how many tweets were posted per country.

Using python and tweepy i manage to receive streams of tweets in json format. In most cases those json format tweets don't have geo locations. All information I have is the user location which is represented as a simple string. Here are a few examples:

  • Berlin, Deutschland
  • France
  • Paris et plein Centre
  • Punjab, Pakistan
  • South Africa
  • Boulder, Colorado
  • NONE
  • London
  • United States
  • Ethiopia
  • Ethiopia
  • NONE
  • Paris, Mulhouse, sur la route
  • London
  • Barcelona
  • Liverpool
  • Bay Area, CA

As you can see there a many different formats in which countries and cities or states (USA) are listed. Is there a Package / Framework / Library / Algorithm i can use to get the most probable country out of these variable informations (Preferably written for python)?

I am hoping to transform TX into USA. (to state an example)

Anyone got any hints what i could use to solve my problem?

  • 3
    This is so very much a GIS task. Why the hell people vote to close this because its general IT? If you said it lacks own research, I would agree, but general IT?! – RoVo Jun 25 '18 at 6:31
7

Getting a location from a string is called Geocoding and there are a number of services offering that.

The geocoder from the OpenStreetMap project is called Nominatim and is freely available, you could simply use the API:

or you might use a ready-made module for Python (e.g. geopy).

geopy includes geocoder classes for the OpenStreetMap Nominatim, ESRI ArcGIS, Google Geocoding API (V3), Baidu Maps, Bing Maps API, Yandex, IGN France, GeoNames, Mapzen Search, OpenMapQuest, PickPoint, What3Words, OpenCage, SmartyStreets, and GeocodeFarm geocoder services

For many services, you need an API key and have some limits. If you use the open Nominatim service, you should always consider the usage policies (e.g. No heavy uses).
Especially if you have many strings to convert and/or want the best performance, you should think about installing Nominatim yourself, which should be quite easy using docker. (e.g. mediagis/nominatim).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.