I have a feature class of points, a few thousands I have now, but I expect to deal with hundreds of thousands of points. I would like find pair of points that is within a specific distance. The points I have is typically spread apart spatially so I don't expect to find many pairs. How I can find those pairs.

Right now the points are originally from ESRI shape file, and read into geopandas geodataframe. I tried to tbl = df['geometry'].distance(df['geometry']) see if I can get distance of all pairs. But I got the error "python: GeometryComponentFilter.cpp:34: virtual void geos::geom::GeometryComponentFilter::filter_ro(const geos::geom::Geometry*): Assertion `0' failed.". Beside But, I don't want to get full set of distance as I am only interested in points that are close to each other.

I have a version of code which uses ArcGIS, and GenerateNearTable_analysis is what I used for the purpose. I want to run code with open source code now. Although I am reading shapefile into geopandas, it doesn't have to be, and recommendation for some other way to work with vector dataset is appreciated as well. It has to be python and should run on Linux.