13

I am working on a civics related project and I need to be able to display all the properties in the City of Philadelphia on a map, so I'll need to get the latitude & longitude for all 580,000 properties. (Only once)

Most APIs like Google/Yahoo have limits of 5,000 per day, and even BatchGeo has a similar limit.

Is there a way I can do a one-time geocoding of all these addresses?

  • 4
    The easiest way is to find a good commercial provider. – Matthew Snape Jan 14 '12 at 10:44
  • 1
    Love your username! – nmpeterson Jan 17 '12 at 16:23
14

You could try the Street Address to Coordinates tool from the Data Science Toolkit.

This API takes either a single string representing a postal address, or a JSON-encoded  
array of addresses, and returns a JSON object with a key for every address. The value 
for each key is either null if no information was found for the address, or an object 
containing location information, including country, region, city and latitude/longitude 
coordinates. Here's an example:

Not sure what the API limits are for Pete Warden's hosted copy but you could run the toolkit yourself and do your processing offline as @Devdatta suggests. There is a downloadable virtual machine that is contains all the tools in the website. Good luck :)

  • Great, thanks! I'll try it and let you know if it worked. – Tobias Fünke Jan 14 '12 at 14:26
  • It's working great so far! I'm running the VM so hopefully there are no limits. Is it pulling from a local database or is it checking some web API? – Tobias Fünke Jan 15 '12 at 18:43
  • 1
    The VM uses comes with a database of places so yeah, no limits. Coverage is great for the United States if I'm not mistaken. You could play around with the source if you're interested :) github.com/petewarden/dstk and github.com/petewarden/dstkdata – R.K. Jan 16 '12 at 0:53
  • 1
    It took eight days – Tobias Fünke Mar 10 '12 at 0:53
  • 1
    Thank you for this reference @R.K. I wonder why datasciencetoolkit isn't more popular. They didn't come up in my research past few days, had to dig deeper. Cheers! – dchhetri Feb 2 '17 at 16:44
3

With This amount of data, I would suggest that you do an offline geocoding. Just the http requests for these many records would be classified as a DoS attack by any server.

  • 4
    Not completely true. If you use a service that is designed to handle that many geocoding requests it won't be an issue. The LiveAddress API can easily handle 1000 address requests per second. That would knock out 598k in about 10 minutes. (I work at address verification and geocoding all day long - I work at SmartyStreets) – Jeffrey Mar 23 '12 at 19:17
3

Geocoding will result in points - 580,000 of them. Are you sure you want to display them all on a map? So many clustered points will likely make the map illegible. These problems assume you find a way to geocode so many records.

The City of Philadelphia's parcel records are available as a polygon layer. Furthermore, those polygons are already available as a map service. If the data/service is suitable for your needs then you don't have to worry about geocoding so many points, and the polygons will most likely look better than so many points on the map.

Information about the data (including metadata and download) and map service:

http://www.pasda.psu.edu/uci/MapService.aspx?Dataset=462

Preview of the map service (zoom in for better view)

http://maps.psiee.psu.edu/preview/map.ashx?layer=462

3

Easiest way I can think of, Get a hold with somebody with an ArcGIS license and feed it to the Geocoder in a Desktop environment.

http://www.lib.unc.edu/reference/gis/faq/geocode.html

2

You might try Geocoda: https://geocoda.com for a JSON API. Geocoda has an API that lets you get lat/lon for an address, which suits your requirements. Geocoda does not have a daily query limit, but does have an initial account limit of 10 queries per second.

Disclaimer: I work at Geocoda. Let me know if you'd like any assistance getting started, or if you'd like to preview our batch API.

1

You could try SVB Mapper which is downloadable from here and works very well with Excel and Bing Maps. You can run mapping and analysis right within Excel. You may have to check on the quota and limits.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.