I'd like to confirm the performance I am observing when utilizing
distance from a Geopandas
Time to complete operations appears to be far greater than comparable operation in PostGIS. I would like to understand if this is known and, if so, if there are suggestions as to methods for making Geopandas more performant, particular regarding geo operations like buffering.
Given a set of geometries, I would like to calculate some aggregate (e.g.
sum()) for each geometry plus those that fall within a given distance of the reference geometry.
Given a set of geometries, I would like to calculate a quarter mile buffer (402 meters) around them and gather the sum the attribute
du (dwelling units).
Current method, utilizing solely the centroids in an attempt to be performant:
# precompute the centroids centroids = df['wkb_geometry'].centroid def test_measure(row): center = row['wkb_geometry'].centroid return df.loc[centroids.distance(center) < 402, 'du'].sum() df.apply(lambda row: test_measure(row), axis=1)
Precomputing centroid and then using distance does appear to introduce some efficiencies compared to buffering and using
within operation. Cost, regardless of methodology, grows at rate
n^2 due to the fact that
n times where
n is the row count and must be run once for each of the
- 100 rows: 0.17s
- 1000 rows: 11.11s
Prior, with buffering and
within it took about:
- 100 rows: 1s
- 1000 rows: 100s
Prior method, not using centroids:
def test_measure(row): buffer_shape = row['wkb_geometry'].buffer(500) return df.loc[df['wkb_geometry'].within(buffer_shape), 'du'].sum()
Were I to perform a similar operation (using
st_dwithin) in PostGIS, I would be able to run the operation in the following times.
- 100 rows: not run
- 1000 rows: not run
- 24000 rows: 75s
For reference, here is an example of that sort of SQL query:
CREATE OR REPLACE FUNCTION agg_within_dist( in_id int, in_geometry geometry, OUT id int, OUT du float) AS $$ SELECT $1 AS geography_id, SUM(CAST(ref.du AS float)) AS du FROM s1.s1_scenario_final AS REF WHERE st_dwithin($2, ref.wkb_geometry, 402); $$ COST 10000 LANGUAGE SQL STABLE strict; SELECT (f).* FROM ( SELECT agg_within_dist(geography_id, wkb_geometry) AS f FROM scenario ) s