Take the 2-minute tour ×
Geographic Information Systems Stack Exchange is a question and answer site for cartographers, geographers and GIS professionals. It's 100% free, no registration required.

I have list with several million place names that come from Flickr profiles. Users provided these placenames as free text, so they look like this:

Roma, Italy
Kennesaw, USA
Saginaw, MI
Rucker, Missouri, USA
Melbourne, Australia
Madrid, Spain
live in Sarnia / work in London, Canada
Valladolid, España
Italia
West Hollywood, United States

I want to disambiguate these place names. I am aware that there is in some cases no straightforward to this solution, but I am willing to live with some false disambiguation and with "no answer" for some of the places. If a place name corresponds to the name of multiple cities, then I want to assign that place to the largest city that it corresponds to.

Yahoo's place finder api would be a good solution to this problem, but I would need to make too many API calls to get through my list, so I'd like a local solution (i.e., one that does not depend on a remote api). Does anyone know of any python libraries that do this kind of thing, or any other local solutions?

(I've also asked this question on stackoverflow.)

share|improve this question

2 Answers 2

up vote 6 down vote accepted

You could try the Python library geodict. This has datasets you can download and import to a database - you can check the lists to see if they'd work well or not with your data. It works in two steps:

  1. Extracting names
  2. Matching names to a location in the lists

More details (and another online option in the comments) here.

share|improve this answer

I assume your best guess is to use a fuzzy algorithm.

Take your local dictionary of place names and administrative units and compare each word and each comma-separated block of text against this dictionary. Assign a score to each match. You might want to use a normalized search to account for spelling mistakes and have an "ignore list" for words like "live" and "work" and "in". Add the score for administrative units to the score of any smaller unit or place name in your matches that lie within this administrative unit.

Tune the scoring function with your results until you are happy. Take the best scoring match.

e.g.: Roma, Italy 
Roma matches 8 places (score according to size)
Roma matches 23 more places with normalization (lower score according to size)
Italy matches 4 places + 2 administrative units (COUNTRY, DISTRICT) (score acconding to size)
Italy matches 14 more places and units with normalization (lower score according to size)
One of the Romas lies in one of your units. -> combine scores

If you tuning is good, you will have given most points to the capital of Italy.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.