2

I am building a web application in which members will be able to find other members in their area. This is your typical "nearest neighbor" search which I'll perform using their latitude and longitude. I'll do the geocoding using one of these searches: 

1. Search by street address + city + region + postal code + country
2. Search by postal code + country
3. Search by city + country

I plan to use OpenStreetMap data imported into a PostgreSQL 9.1 database using PostGIS 1.5.3. I'll use Nominatim as my geocoder because my users can be anywhere in the world. The problem is that I only have a small 8-CPU Debian 7 VPS server (on linode.com) with 1-GB of RAM while the compressed planet OSM file is around 30-GB. I'm afraid I won't be able to load that much data with so little RAM. And even if I can load it, it will take forever. What I really need is a server with 16-GB of memory but they're very expensive and I'd only need a server that size to initialize my geocoding server and perhaps to do quarterly updates.

Here are my questions:

  1. Which loader should I be using to put the data in an optimal format for the geocoding queries shown above? I've loaded a small Illinois .pbf file using osm2pgsql but there is also Osmosis and Imposim. I've read that Imposim is good if you have limited memory but I read that osm2pgsql might be best if I'm geocoding.
  2. Can I reduce the loading time with one of these loaders by only loading those elements that are needed to perform the above geocoding queries? If so, which elements are needed for the loader in question? osm2pgsql mentions "import styles" while Imposim has a mapping file.
  3. Would it perhaps be a good idea to spin up an Amazon EC2 or RDS server to download the planet file and load the data into a database and then back that data up, copy the backup over to my linode, and restore it there? This would save me the cost of upgrading to a more powerful linode.

Thanks for your response!

Improve this question
3
  • I'm not sure what you are actually asking. You want to use Nominatim which requires you to import your data into a database. All geocoding queries are then performed using this database and not the large XML/PBF file. According to the system requirements you have way too little RAM for the whole planet but I don't know if the import will fail or just take ages. Instead you could also use a public Nominatim instance for your geocoding queries, for example the one provided by MapQuest.
    – scai
    Commented Dec 10, 2013 at 7:34
  • What I'm asking is if I'll only need specific columns (i.e. tags) to perform a geocoding query and, if so, is there a way to import the .pbf file with a loader such as osm2pgsql or Imposim so as to only load those columns? E.g., osm2pgsql creates a "planet_osm_points" table which has a "culvert" column. I seriously doubt that column is required to create a geocoding query so I'd like to omit it when loading to speed up the loading process.
    – Jim
    Commented Dec 11, 2013 at 20:34
  • No you certainly don't need all tags. I recommend to take a look at Nominatim and the development notes and maybe its source code. Currently it is the most popular geocoder for OSM. Nominatim also uses a different database scheme than the one used for rendering in order to speed up geocoding queries.
    – scai
    Commented Dec 12, 2013 at 7:28

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.