Take the 2-minute tour ×
Geographic Information Systems Stack Exchange is a question and answer site for cartographers, geographers and GIS professionals. It's 100% free, no registration required.

Here is a a little bit of a detailed question related to address parsing/geocoding which I feel should be interesting to many users.

So, essentially I am curious to know if anyone has had any experience installing, building or extending a opensource geocoding and/or address correction tool.

I am aware of geocoder:US 2.0 initiatives which I think are maintained by geocommons but I am unsure if there are better alternatives, other opensource tools, if their system can be effectively extended or if there are any developments I might not be aware of.

My goals are as follows:

  1. I need a highly accurate tool which is capable of automatically parsing out and/or standardizing location data inputted by users from a single input field all in real-time and with the highest volume possible.
  2. Input data would be one or more of the following address components: zipcode, county, city, street, address, state.
  3. Input data also needs to be able to lookup from our custom geonames database. For example he may enter the name of a neighborhood or non USPS location name which naturally are not standard address variables.

Given these goals I am well aware of the fact that when given a single form field to conduct such a lookup each user will enter his data in different formats while the other factor generally fall into misspellings.

Besides utilizing the census database as the core for the valid addresses/ranges (all which I believe Geocoder:US does, I believe some type of ability to define known "aliases" would be ideal for known misspellings of street names. The same goes for things such as a user entering Ave compared to Ave. compared to Avenue. Don't think such alias capabilities are fully possible with the Geocoder:US tool.

While the above elements may indeed solve the majority of issues I think some type of effective fuzzy matching needs to exist when the input can't be matches to high enough %age.

If input data can effectively be parsed out into individual elements based off some assumed rules and then utilizing a type of "match score" component to fuzzy match any unmatched elements would have to be based on those elements which were already "matched" with a high degree.

For example: I am going to assume for geocoding to be as effective as possible we need to extract individual data elements from the input field first in an attempt to narrow down the "area" the user is trying to find results for. In my view this means that a 5 digit number could be assumed to be a zipcode, if there is another element such as a city name that matches the zipcode the assumption that we have the "area" correct... Next we use the remaining data to try to find a full, partial or fuzzy match, score and list possible results.

In any case - I would greatly appreciate if anyone could provide some advise here along with any advise, performance stats or upcoming developments they are aware of which might adjust my direction (such as the use of postgis 2.0 as a means for enhanced matching capabilities)

share|improve this question
add comment

3 Answers

up vote 3 down vote accepted

you can try gisgraphy. it includes an address parser, a geocoder, and a reverse geocoder. (dont use the free service for batch, but install it on your server). fulltextsearch with synomyms, spellchecking can probably helps too. there is no problems if you need high volumes, because gisgraphy is available as webservices with several format (XML, JSON, PHP, Python, Ruby, YAML, GeoRSS, and Atom) so it can scale

gisgraphy

share|improve this answer
add comment

I have some experience with this. At SmartyStreets (where I work), we make address verification software called LiveAddress. (It's actually all web-based; no need to download or install it.)

The challenges of validating and standardizing addresses are plenty, I assure you. It gets even trickier when you attempt to parse the address into particular components yourself, or implementing "fuzzy search." But have no fear... we have un-officially published a basic procedure for performing free-form address validation. While our service isn't open source, we're fairly open about sharing our expertise with the community and setting new standards for quality and performance.

Anyway, I think you'll find that page somewhat helpful. An API such as ours will handle thousands upon thousands of requests per second since we're geo-distributed across three datacenters nationally. LiveAddress should be able to take care of the "fuzzy matching" for you and return only valid results, filling in the missing pieces and correcting misspellings.

It takes into account official USPS aliases and even unofficial street names or location names and matches them to official, deliverable endpoints. For your own custom names, though, you'll have to work it into your own database for that.

A final word, too: I would add that while open source tools are great and free, you will probably trade it for some aspect of service, performance, and overall quality. Even if you host the service in-house, you're responsible for maintaining it and meeting the demands of what sounds like, in your case, a heavy payload.

I'll be happy to personally answer your own questions about addresses -- I think the task before you is really quite interesting, and may seem overwhelming without the right resources.

share|improve this answer
add comment

Address standardization (AKA address correction, address normalization, address parsing) is not a simple task. If you have swift fingers and ample creativity, a very fine REGEX can be concocted that can do a remarkably good job. However, it doesn't handle very well the edge cases where the results can be ambiguous. The reason is a lack of context. You have to know what the correct result looks like in order to know that you have achieved the accuracy that you need. Certainly, taking a list of 100k addresses and being able to parse 70% of them accurately (using only REGEX) is better than not parsing any of them. But, how long does it take to parse the remaining "hard" addresses? A LONG TIME. They require a large number of specialized parsing functions because the context, or the "right answer" is unknown. This is where address verification comes in handy because the "context" is known. The fully standardized and corrected address is known and the master list can be used to compare the results.

I get asked this a lot since I work with address verification at Smartystreets.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.