I have one dataset with about 10 million points (lat, long). I would like to select the points that fall within a map.
I have this map as a shapely object (
.shp). To do this, I transformed the localization of each point into a Point object using
points_from_xy. So I transformed my DataFrame into a GeoDataFrame.
My map is in the form of a Multipolygon object. So I used
geometry.unary_union to convert it into a unified polygon (I'm not sure if it is correct). Then I used the
within method in Geopandas to selects the Points inside the map.
import pandas as pd import geopandas as gpd from tqdm import tqdm map = gpd.read_file('foo.shp') df = pd.read_csv('foo1.csv') points = gpd.GeoDataFrame(df, geometry=gpd.points_from_xy(df.longitude, df.latitude)) map = map.geometry.unary_union within_points = [points.geometry[i].within(map) for i in tqdm(range(points.geometry.count()))] within_points = points[within_points]
My problem is that this process takes TOO long (about 5 days on my core i5 laptop). I would like to know if there is any way to accelerate it?