2

I have a data size of 1.87GB containing 14962000 rows and 9 columns. And I want to apply GeoPy but I got the errors.

My code:

import pandas as pd 
from geopy.geocoders import Nominatim
from geopy.extra.rate_limiter import RateLimiter

df = pd.read_csv("mydata_name.csv")

df["Geo"] = df["latitude"].astype(str)+ ',' + df["longitude"].astype(str)
geolocator = Nominatim(user_agent="MyApp", timeout=10)

rgeocode = RateLimiter(geolocator.reverse, min_delay_seconds=0.001)

df['address'] = df['Geo'].apply(rgeocode)

When executing my code I get the following errors. I want to get an address for all rows. But the code takes 5 hours to execute and I get this:

RateLimiter caught an error, retrying (0/2 tries). Called with (*('24.807412,46.803666',), **{}).
Traceback (most recent call last):
  File "C:\Users\ibrahim\Anaconda3\lib\site-packages\geopy\geocoders\base.py", line 360, in _call_geocoder
    result = self.adapter.get_json(url, timeout=timeout, headers=req_headers)
  File "C:\Users\ibrahim\Anaconda3\lib\site-packages\geopy\adapters.py", line 373, in get_json
    resp = self._request(url, timeout=timeout, headers=headers)
  File "C:\Users\ibrahim\Anaconda3\lib\site-packages\geopy\adapters.py", line 401, in _request
    raise AdapterHTTPError(
geopy.adapters.AdapterHTTPError: Non-successful status code 502

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\ibrahim\Anaconda3\lib\site-packages\geopy\extra\rate_limiter.py", line 136, in _retries_gen
    yield i  # Run the function.
  File "C:\Users\ibrahim\Anaconda3\lib\site-packages\geopy\extra\rate_limiter.py", line 274, in __call__
    res = self.func(*args, **kwargs)
  File "C:\Users\ibrahim\Anaconda3\lib\site-packages\geopy\geocoders\nominatim.py", line 359, in reverse
    return self._call_geocoder(url, callback, timeout=timeout)
  File "C:\Users\ibrahim\Anaconda3\lib\site-packages\geopy\geocoders\base.py", line 378, in _call_geocoder
    self._adapter_error_handler(error)
  File "C:\Users\ibrahim\Anaconda3\lib\site-packages\geopy\geocoders\base.py", line 392, in _adapter_error_handler
    raise exc_cls(str(error)) from error
geopy.exc.GeocoderServiceError: Non-successful status code 502
RateLimiter caught an error, retrying (0/2 tries). Called with (*('24.650473,46.669335',), **{}).
Traceback (most recent call last):
  File "C:\Users\ibrahim\Anaconda3\lib\site-packages\geopy\geocoders\base.py", line 360, in _call_geocoder
    result = self.adapter.get_json(url, timeout=timeout, headers=req_headers)
  File "C:\Users\ibrahim\Anaconda3\lib\site-packages\geopy\adapters.py", line 373, in get_json
    resp = self._request(url, timeout=timeout, headers=headers)
  File "C:\Users\ibrahim\Anaconda3\lib\site-packages\geopy\adapters.py", line 401, in _request
    raise AdapterHTTPError(
geopy.adapters.AdapterHTTPError: Non-successful status code 502

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\ibrahim\Anaconda3\lib\site-packages\geopy\extra\rate_limiter.py", line 136, in _retries_gen
    yield i  # Run the function.

2 Answers 2

2

Your code looks okay to me. The only thing that I suggest is to wrap your geocoding procedure into a try/except, i.e.

from geopy.point import Point

def reverse_geocoding(lat, long):
    point = Point(lat, lon)
    try:
        location = locator.reverse(point)
        return location.raw
    except (AttributeError, KeyError, ValueError):
        return f"No result for {point}"

Regarding the 502 Bad Gateway error:

GeocoderServiceError: Non-successful status code 502

it can occur because of several reasons:

  1. As mentioned in the Nominatim's documentation:

    No heavy uses (an absolute maximum of 1 request per second).

    So, you probably exceeded the daily limit, which is 86400 requests per day (1sec * 60 * 60 * 24).

    The easiest way to guarantee you don't make more than one request per second is

    time.sleep(1)
    

    while interacting through the requests, before making a request.

    In total it might be needed about 174 days for proceeding 14962000 rows.

  2. Another assumption (which is less likely) is that the web service stopped working. It was described in this issue on GitHub.


References:

1
  • Why does AttributeError, KeyError, ValueError work here when the error says AdapterHTTPError?
    – filifunk
    Apr 6, 2023 at 0:34
1

An HTTP 502 status implies something went wrong in your internet connection, it's possible you exceeded some limit on the geocoder.

2
  • Thanks for response. How can I know the maximum limits May 4, 2021 at 16:09
  • That would probably be covered in the documentation
    – Ian Turton
    May 4, 2021 at 17:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.