Take the 2-minute tour ×
Geographic Information Systems Stack Exchange is a question and answer site for cartographers, geographers and GIS professionals. It's 100% free, no registration required.

I'm creating a custom google map that has 125 markers plotted via a cms. When loading the map I get this message:

Geocode was not successful for the following reason: OVER_QUERY_LIMIT

I'm pretty sure it's the way in which I've geocoded the markers.

How can I avoid these warnings and is there a more efficient way to geocode the results?

UPDATE: This is my attempt at Casey's answer, I'm just getting a blank page at the moment.

<script type="text/javascript"> 
(function() { 

window.onload = function() { 
 var mc;
// Creating an object literal containing the properties we want to pass to the map 
var options = { 
zoom: 10, 
center: new google.maps.LatLng(52.40, -3.61), 
mapTypeId: google.maps.MapTypeId.ROADMAP 
}; 

// Creating the map 
var map = new google.maps.Map(document.getElementById('map'), options); 

// Creating a LatLngBounds object 
var bounds = new google.maps.LatLngBounds(); 

// Creating an array that will contain the addresses 
var places = []; 

// Creating a variable that will hold the InfoWindow object 
var infowindow; 
mc = new MarkerClusterer(map);
<?php
$pages = get_pages(array('child_of' => $post->ID, 'sort_column' => 'menu_order'));
$popup_content = array();
foreach($pages as $post)
    {
    setup_postdata($post);
    $fields = get_fields(); 
    $popup_content[] = '<p>'.$fields->company_name.'</p><img src="'.$fields->company_logo.'" /><br /><br /><a href="'.get_page_link($post->ID).'">View profile</a>';
    $comma = ", ";
    $full_address = "{$fields->address_line_1}{$comma}{$fields->address_line_2}{$comma}{$fields->address_line_3}{$comma}{$fields->post_code}";
    $address[] = $full_address;
    }
wp_reset_query();
echo 'var popup_content = ' . json_encode($popup_content) . ';';
echo 'var address = ' . json_encode($address) . ';';
?>

var geocoder = new google.maps.Geocoder(); 

var markers = [];

// Adding a LatLng object for each city  
for (var i = 0; i < address.length; i++) { 
    (function(i) { 
        geocoder.geocode( {'address': address[i]}, function(results, status) {
            if (status == google.maps.GeocoderStatus.OK) {
                places[i] = results[0].geometry.location;

                // Adding the markers 
                var marker = new google.maps.Marker({position: places[i], map: map});
                markers.push(marker);
                mc.addMarker(marker);

                // Creating the event listener. It now has access to the values of i and marker as they were during its creation
                google.maps.event.addListener(marker, 'click', function() {
                    // Check to see if we already have an InfoWindow
                    if (!infowindow) {
                        infowindow = new google.maps.InfoWindow();
                    }

                    // Setting the content of the InfoWindow
                    infowindow.setContent(popup_content[i]);

                    // Tying the InfoWindow to the marker 
                    infowindow.open(map, marker);
                });

                // Extending the bounds object with each LatLng 
                bounds.extend(places[i]); 

                // Adjusting the map to new bounding box 
                map.fitBounds(bounds) 
            } else { 
            alert("Geocode was not successful for the following reason: " + status); 
            }

        });

    })(i);

} 
var markerCluster = new MarkerClusterer(map, markers); 
} 
})
(); 
</script> 

UPDATE: It doesn't really matter what the solution is as long as the markers load instantly and it's not breaking any terms & conditions. This is all new to me in the first place so why not learn some more new things!!

share|improve this question
    
You dont say why or how often you are doing these lookups. You arent trying to geocode the same points every page render are you? If thats the case you should be looking at using the api to do these lookups server side once only, when the location is first added to the database. –  Peter Sep 27 '11 at 21:49
    
If you want to try @Peter's advice, there's a simple tool here which will geocode the points in a Google spreadsheet. This would allow you write the lat/long for the cities into the CMS, so you wouldn't need to use the geocoder at run-time. –  Stephen Lead Sep 27 '11 at 23:07
    
@Peter Actually that sounds like a great idea. I'm a little torn as it will decrease the usability of my cms. If the client is having to find out lat/long per company rather than just entering the address then it's not very user friendly. Am I understanding that correctly? –  Rob Sep 28 '11 at 8:19
    
See my new answer lower down for a php function. –  Peter Oct 4 '11 at 2:19
2  
@rob FWIW I think Ragi has the right approach. Casey's answer still requires that you geocode every point, every time the map draws, which doesn't appear to be necessary for your static dataset –  Stephen Lead Oct 6 '11 at 3:29

8 Answers 8

up vote 29 down vote accepted
+25

Like everybody else, I could give you an answer with code, but I don't think somebody has explained to you that you are doing something that is fundamentally wrong.

Why are you hitting this error? Because you are calling geocode every time somebody views your page and you are not caching your results anywhere in the db!

The reason that limit exists is to prevent abuse from Google's resources (whether it is willingly or unwillingly) - which is exactly what you are doing :)

Although google's geocode is fast, if everybody used it like this, it would take their servers down. The reason why Google Fusion Tables exist is to do a lot of the heavy server side lifting for you. The geocoding and tile caching is done on their servers. If you do not want to use that, then you should cache them on your server.

If still, 2500 request a day is too little, then you have to look at Google Maps Premier (paid) license that gives you 100,000 geocoding requests per day for something around 10k a year (that is a lot - with server side caching you should not be reaching this limit unless you are some huge site or are doing heavy data processing). Without server side caching and using your current approach, you would only be able to do 800 pageviews a day!

Once you realize that other providers charge per geocode, you'll understand that you should cache the results in the db. With this approach it would cost you about 10 US cents per page view!

Your question is, can you work around the throttle limit that Google gives you? Sure. Just make a request from different ip addresses. Heck, you could proxy the calls through amazon elastic ips and would always have a new fresh 2500 allotted calls. But of course, besides being illegal (you are effectively circumventing the restriction given to you by the Google Maps terms of service), you would be doing a hack to cover the inherent design flaw you have in your system.

So what is the right way for that use-case? Before you call the google geocode api, send it to your server and query if it is in your cache. If it is not, call the geocode, store it in your cache and return the result.

There are other approaches, but this should get you started in the right direction.

Update: From your comments below, it said you are using PHP, so here is a code sample on how to do it correctly (recommendation from the Google team itself) https://developers.google.com/maps/articles/phpsqlsearch_v3

share|improve this answer
    
Have you got an example I could follow of sending it to the server etc? –  Rob Oct 5 '11 at 15:38
    
What is your backend CMS? (I assume you can/know how change it and just want a simple example that shows how to do that in your language of choice. You don't need an example of a client side js, or do you? Are you using any js framework like jquery? –  Ragi Yaser Burhum Oct 5 '11 at 16:19
    
Also, you will need to give more context about how the marker data gets into your system in the first place –  Ragi Yaser Burhum Oct 5 '11 at 16:21
    
@RagiYaserBurham I'm using wordpress with custom fields. The php bit is the part that grabs the data from the cms. A client side example of js would be handy. –  Rob Oct 5 '11 at 18:17
1  
OK. Php then. This bit from the Google Documentation should include the mysql and php part code.google.com/apis/maps/articles/phpsqlgeocode.html –  Ragi Yaser Burhum Oct 5 '11 at 18:32

Here is my solution:

dependencies: Gmaps.js, jQuery

var Maps = function($) {
   var lost_addresses = [],
       geocode_count  = 0;

   var addMarker = function() { console.log('Marker Added!') };

   return {
     getGecodeFor: function(addresses) {
        var latlng;
        lost_addresses = [];
        for(i=0;i<addresses.length;i++) {
          GMaps.geocode({
            address: addresses[i],
            callback: function(response, status) {
              if(status == google.maps.GeocoderStatus.OK) {
                addMarker();
              } else if(status == google.maps.GeocoderStatus.OVER_QUERY_LIMIT) {
                lost_addresses.push(addresses[i]);
              }

               geocode_count++;
               // notify listeners when the geocode is done
               if(geocode_count == addresses.length) {
                 $.event.trigger({ type: 'done:geocoder' });
               }
            }
          });
        }
     },
     processLostAddresses: function() {
       if(lost_addresses.length > 0) {
         this.getGeocodeFor(lost_addresses);
       }
     }
   };
}(jQuery);

Maps.getGeocodeFor(address);

// listen to done:geocode event and process the lost addresses after 1.5s
$(document).on('done:geocode', function() {
  setTimeout(function() {
    Maps.processLostAddresses();
  }, 1500);
});
share|improve this answer

It sounds like you are hitting the simultaneous request limit imposed by Google (though I cannot find a reference to what the limit actually is). You will need to space your requests out so that you do not send 125 requests all at once. Note that there is also a 2500 geocode request per day limit.

Consult the Google Geocoding Strategies document for more information.

Update: As an added solution inspired thanks to a post by Mapperz, you could think about creating a new Google Fusion Table, storing your address and any related data in the table, and geocoding the table through their web interface. There is a limit to the number of geocode requests a Fusion Table will make, however the limit is quite large. Once you reach the limit, you will need to re-submit the geocode request, and Fusion Tables will pick up where it left off. Bonus to this approach is a huge speed improvement, as you will only need to geocode ONCE, where with your current solution you will need to geocode on every load, quickly reaching daily limits.

share|improve this answer
    
I guessed I'd be hitting the limit, I'm so new to this that I think I need a code example to help. –  Rob Sep 27 '11 at 16:11
    
I've been looking into Google Fusion Tables and come across a few problems. 1. You have to manually geocode addresses (via file->geocode) in the table, can this be done automatically? 2. Some markers are only appearing at certain zoom levels while others show whatever the zoom level. This is the example - mediwales.com/mapping/example. What I eventually want to do is export my Wordpress data (ie all the company addresses) to the fusion table, geocode those automatically then grab all those geocoded addresses and plot the map instantly. –  Rob Nov 3 '11 at 11:45

using geopy geocoders I'm able to geocode a large number of addresses without a hitch (i'm not sure of the posters setup, or if he can include a python dependency to his project)

here is my test:

from geopy import geocoders

g = geocoders.Google()

for x in xrange(1,10000):
    print x
    a = g.geocode("AV JOAO NAVES DE AVILA " + str(x) + " UBERLANDIA MG BRASIL")
    print a

result snippet

7595 (u'Av. Jo\xe3o Naves de \xc1vila, 7595 - Segismundo Pereira, Uberl\xe2ndia - Minas Gerais, 38408-288, Brazil', (-18.9388154, -48.2220562))

7596 (u'Av. Jo\xe3o Naves de \xc1vila, 7596 - Segismundo Pereira, Uberl\xe2ndia - Minas Gerais, 38408-680, Brazil', (-18.938814, -48.2217423))

7597 (u'Av. Jo\xe3o Naves de \xc1vila, 7597 - Segismundo Pereira, Uberl\xe2ndia - Minas Gerais, 38408-288, Brazil', (-18.9388128, -48.2220381))

7598 (u'Av. Jo\xe3o Naves de \xc1vila, 7598 - Segismundo Pereira, Uberl\xe2ndia - Minas Gerais, 38408-680, Brazil', (-18.9388114, -48.2217242))

7599 (u'Av. Jo\xe3o Naves de \xc1vila, 7599 - Segismundo Pereira, Uberl\xe2ndia - Minas Gerais, 38408-288, Brazil', (-18.9388102, -48.2220201))

7600 (u'Av. Jo\xe3o Naves de \xc1vila, 7600 - Segismundo Pereira, Uberl\xe2ndia - Minas Gerais, 38408-680, Brazil', (-18.9388088, -48.2217062))

This is still running and I'm with 7k results.

EDIT: After a while i've hitted the end of the street and google started to give me same points, but looks like it's still making requests and giving me accurate answers.

share|improve this answer

I have tried the page now and it seems to be working right now.

While I think the the timer approach you are currently using is correct and as far as I know the only one that will work w/o requiring some server side support I should suggest the following changes (I took the liberty of rewriting your geocoding functions) to take advantage of localStorage.

LocalStorage is a relatively new technology built into most browsers that allows the web page to store data on the client. It somewhat similar to cookies, but it is much more powerful and it is not always resent to the server (unlike cookies).

My aim is to save geocoded address on the client so that if a user refreshes the page there is no need to invoke Google's geocoder functions again on them. The implementation below is crude (we should add code to check for stale records and probably need a better caching key than the index) but should be enough to get you started. Hope it helps.

(function() { 

window.onload = function() { 
 var mc;
// Creating an object literal containing the properties we want to pass to the map 
var options = { 
zoom: 0, maxZoom: 0, 
center: new google.maps.LatLng(52.40, -3.61), 
mapTypeId: google.maps.MapTypeId.ROADMAP 
}; 

// Creating the map 
var map = new google.maps.Map(document.getElementById('map'), options); 

// Creating a LatLngBounds object 
var bounds = new google.maps.LatLngBounds(); 

// Creating an array that will contain the addresses 
var places = []; 

// Creating a variable that will hold the InfoWindow object 
var infowindow; 
mc = new MarkerClusterer(map);
var popup_content = [..redacted..];

var geocoder = new google.maps.Geocoder(); 

var markers = [];

// Adding a LatLng object for each city 
function geocodeAddress(i) {
     // check if localStorage is available (it is now available on most modern browsers)
     // http://html5tutorial.net/tutorials/working-with-html5-localstorage.html
     // NB: this *must* be improved to handle quota limits, age/freshness, etc
     if(localStorage && localStorage['address_'+i]) {
        places[i]=JSON.parse(localStorage['address_'+i]);
        addPlace(i);
     } else {
      geocoder.geocode( {'address': address[i]}, function(results, status) {
        if (status == google.maps.GeocoderStatus.OK) {
            places[i] = results[0].geometry.location;
            if(localStorage) {
               // cache result locally on the browser, this will help reducing the number of requests
               // to the google geocoder in case the user refreshes the page
               // remember: the more he will refresh, the more he's likely to hit the limit
               // (this is one case where refreshing or closing the browser does not work)
               localStorage['address_'+i]=JSON.stringify(results[0].geometry.location);
            }

            addPlace(i);
        } else { 
            console.log("Geocoding of address "+address[i]+" failed");
        }
    })
}

function addPlace(i) {
    // Adding the markers 
    var marker = new google.maps.Marker({position: places[i], map: map});
    markers.push(marker);
    mc.addMarker(marker);

    // Creating the event listener. It now has access to the values of i and marker as they were during its creation
    google.maps.event.addListener(marker, 'click', function() {
        // Check to see if we already have an InfoWindow
        if (!infowindow) {
            infowindow = new google.maps.InfoWindow();
        }

        // Setting the content of the InfoWindow
        infowindow.setContent(popup_content[i]);

        // Tying the InfoWindow to the marker 
        infowindow.open(map, marker);
    });

    // Extending the bounds object with each LatLng 
    bounds.extend(places[i]); 

    // Adjusting the map to new bounding box 
    map.fitBounds(bounds);
}

function geocode() {
    if (geoIndex < address.length) {
        geocodeAddress(geoIndex);
        ++geoIndex;
    }
    else {
        clearInterval(geoTimer);
    }
}
var geoIndex = 0;
var geoTimer = setInterval(geocode, 600);  // 200 milliseconds (to try out)

var markerCluster = new MarkerClusterer(map, markers); 
} 
})
(); 
share|improve this answer
    
Thanks very much, just giving something else a go but will try this in a minute. Sounds like a good idea. –  Rob Oct 5 '11 at 13:48
    
Just tried it out (have now reverted back to previous), it worked the first time then when I refreshed it just showed a blank page. –  Rob Oct 5 '11 at 13:59
    
try now, there was an error, also please note that the popup_content variable has been redacted for brevity. –  unicoletti Oct 5 '11 at 14:07

I think Sasa is right, on both counts.

In terms of sending all your addresses at once, one option is to send the requests at intervals. In the past when using JavaScript I have opted to delay requests by 0.25 (seems to work!) seconds using the

setTimeout( [FUNCTION CALL] , 2500 )

method.

In .NET i have opted for:

System.Threading.Thread.Sleep(250);

Seems to work.

EDIT: Cant really test it, but this should/might work!

Javascript example. The addressArray holds strings that are addresses...

for (var i = 0; i < addressArray.length; i++0 
{

setTimeout('googleGeocodingFunction(' + addressArray[i] + ')' , 2500);

}

EDIT:

for (var i = 0; i < address.length; i++) {
    function(i) {
        setTimeout(geocoder.geocode({ 'address': address[i] }, function(results, status) {
            if (status == google.maps.GeocoderStatus.OK) {
                places[i] = results[0].geometry.location;

                var marker = new google.maps.Marker({ position: places[i], map: map });
                markers.push(marker);
                mc.addMarker(marker);
                google.maps.event.addListener(marker, 'click', function() {
                    if (!infowindow) {
                        infowindow = new google.maps.InfoWindow();
                    }

                    // Setting the content of the InfoWindow
                    infowindow.setContent(popup_content[i]);

                    // Tying the InfoWindow to the marker 
                    infowindow.open(map, marker);
                });

                // Extending the bounds object with each LatLng 
                bounds.extend(places[i]);

                // Adjusting the map to new bounding box 
                map.fitBounds(bounds)
            } else {
                alert("Geocode was not successful for the following reason: " + status);
            }
        }), 2500);// End of setTimeOut Function - 2500 being a quarter of a second. 
    } 
}
share|improve this answer
    
Thanks for the reply. How can I get that into my code. This is all a bit new so need some hand holding. You should be able to view my source code. –  Rob Sep 27 '11 at 16:10
    
Which bit? What language are you using? –  CatchingMonkey Sep 27 '11 at 16:11
    
I'm using javascript, how could I set an interval? –  Rob Sep 27 '11 at 16:14
    
Edited my answer. –  CatchingMonkey Sep 27 '11 at 16:17
    
Also realsed i meant setTimeout()... Thanks Casey. –  CatchingMonkey Sep 27 '11 at 16:24

This php function (i dont know which language youre working in, but you can pick it apart im sure) uses google geolocator api. Given an address, it will return the normalised address and latlong. HTH.

// GEOLOCATEBYADDRESS
// @arg (string)address
// @return (array) ((int)flag,(array)address,array(rawresponse))

function geolocatebyaddress($lookupaddress){

    $lookupaddress=trim("$lookupaddress New Zealand");
    $lookupaddress=urlencode($lookupaddress);

    //send off google api lookup request
    $apiurl="http://maps.google.com/maps/api/geocode/json";
    $apiurl.="?address=$lookupaddress";
    $apiurl.="&region=NZ";
    $apiurl.="&sensor=false";

    if (function_exists("curl_init")) {
        $ch = curl_init();
        curl_setopt($ch, CURLOPT_URL, $apiurl);
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
        $json = curl_exec($ch);
        curl_close($ch);
    }
    else     $json= file_get_contents($apiurl);

    //process response
    $response= json_decode($json,true);
    $approxflag=0;
    if ($response['status']=='OK' and count($response['results'])) {

        $aa= array();
        foreach($response['results'] as $cc=>$result) {
            //set keys
            foreach(array('street_number','road','suburb','city','postcode','region','lat','long') as $t){
                $aa[$cc][$t]='';
            }
            if ($result['geometry']){
                $aa[$cc]['lat']= $result['geometry']['location']['lat'];
                $aa[$cc]['long']=$result['geometry']['location']['lng'];
         }

            if (count($result['address_components'])){
                foreach ($result['address_components'] as $acs){
                    if ($acs['types'][0]=='street_number')               $aa[$cc]['street_number']= $acs['long_name']; 
                    if ($acs['types'][0]=='route')                       $aa[$cc]['road']=      $acs['long_name']; 
                    if ($acs['types'][0]=='sublocality')                 $aa[$cc]['suburb']=   $acs['long_name']; 
                    if ($acs['types'][0]=='locality')                    $aa[$cc]['city']=    $acs['long_name']; 
                    if ($acs['types'][0]=='postal_code')                 $aa[$cc]['postcode']= $acs['long_name']; 
                    if ($acs['types'][0]=='administrative_area_level_1') $aa[$cc]['region']=   $acs['long_name']; 
                }    
            }
            //successful?
            if ($result['geometry']['location_type']=='APPROXIMATE') $approxflag++;
            if (isset($result['partial_match']))  $approxflag++;
            if (!$aa[$cc]['street_number'])         $approxflag++;
        }
        if ($approxflag) return (array(1,$aa,$response));
        else return (array(2,$aa,$response));
    }
    else return (array(0,array(),$response));
}    
share|improve this answer
    
Thanks for the reply, how would I incorporate this into my code? –  Rob Oct 4 '11 at 10:54
    
Sorry i presumed you were coding the site. That function is used as part of server side scripting that for instance takes a user input address, uses the google geocoder api to get the coords. You can then put those coords in your db, and when you need to produce a map, just query the coords and produce the map with markers. That way you wont be abusing the geocoder for every page render as dicsussed. (did you downote my comment? I am trying to help, and penalising me isnt going too encourage me at all). Also i forogt to mention that the api requires you to use it with a gmap. –  Peter Oct 8 '11 at 1:29

Here is how I throttle my geocode requests in javascript, addresses being an array containing each address to geocode:

for (i=0;i<=addresses.length;i++) {
    setTimeout( function () {
            geocoder.geocode( { 'address': addresses[i]}, function(results, status) {

                if (status == google.maps.GeocoderStatus.OK) { 

                    //create and add marker to map based off geocode result
                    var marker = new google.maps.Marker({  
                        map: map,
                        title: results[0].formatted_address,
                        position: results[0].geometry.location
                    });

                 } //EDIT, was missing the closing bracket here
            });

    }, i * 1500);
}

This essentially adds a second and half between each geocoding request.

share|improve this answer
    
Thanks for the answer. I keep getting a blank page when trying to integrate into my code - mediwales/mapping –  Rob Sep 27 '11 at 19:28
    
Are you using any debugging tools that could point out the error? If you aren't, try using the FireFox plugin firebug. It can be pretty handy in situations like this. –  Casey Sep 27 '11 at 19:47
    
I'm not no. I'm not confident I've put the code in the right place, I'll update my question with my attempt. –  Rob Sep 27 '11 at 19:55
    
I was missing a closing bracket in my code sample. I edited my answer and added it, with a comment. However, it looks like this wasn't an issue in your sample code. Are you getting a blank page or a blank map? A blank page would indicate there is a syntax error in the javascript code. A blank map (no markers) would indicate that the geocoding code has an issue. –  Casey Sep 27 '11 at 20:26
1  
Hi. I've had another go at your answer. The delay is working now but can't get the markers to plot. From my code these are the important bits that plot the markers - (function(i) { and })(i); without those it won't plot, even without adding the delay. –  Rob Oct 5 '11 at 10:33

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.