GeoPandas Dissolve takes long time to execute

Question

I have a dataframe in GeoPandas Python. The data frame contains around 3800 rows (geometry type is Multiplygon). Some polygons are overlapping each other. I want to dissolve those overlap. I used following code to dissolve overlapping polygons.

import geopandas as gpd
overlapping_areas = gdf[gdf.geometry.overlaps(gdf.unary_union)]

if not overlapping_areas.empty:
    dissolved_coverage = gdf.dissolve(by='LEGEND')
    dissolved_coverage = dissolved_coverage.reset_index()
else:
    dissolved_coverage = gdf

where gdf is the GeoPandas dataframe and 'LEGEND' is one of the columns in GeoPandas dataframe that contains the unique values.

The above code has been running for more than two hours (still running as I write). I am not sure how long further it takes to compelte the execution. Is there any better way to program it faster?

Answer 1

This will be very slow: gdf[gdf.geometry.overlaps(gdf.unary_union)]. You are unioning the entire dataframe 3800 times, once for each row.

Try dissolving to dissolve overlaps, then explode to separate those polygons not overlapping:

import geopandas as gpd
file = r"/home/bera/Desktop/gistest/overlaps.gpkg"
df = gpd.read_file(file, layer="overlaps")
dissolved = df.dissolve(by="kkod")
exploded = dissolved.explode(index_parts=True)

Answer 2

I suppose this construction

overlapping_areas = gdf[gdf.geometry.overlaps(gdf.unary_union)]

if not overlapping_areas.empty:

is meant as a performance optimisation?

If that is the (only) reason, a good start is to just remove that, that should give a huge speedup and will avoid wrong results.

Some reasons why it is counterproductive for performance:

1. the gdf.unary_union will be executed for every row, so 3800 times.
2. even if you would perform gdf.unary_union only once, dissolve also does a unary_union under the hood, so you are doing the same thing twice. Worse: performing unary_union on a list of geometries becomes worse than linearly slower depending on the number of geometries. Because your dissolve has a groupby, the number of geometries in each group will probably be a lot smaller than the gdf.unary_union being performed on all geometries in one go, so the gdf.dissolve(by=...) will typically be (a lot) faster than performing gdf.unary_union once.
3. gdf.unary_union will probably create a very complex polygon, so performing overlaps with it for every geometry will be relatively slow as well.
4. overlaps is a bit of an unclear name for this predicate and will result in other results than you probably expect: shapely.overlaps. In addition, the gdf.unary_union contains ALL geometries, including the one you are checking for if there are any intersections, so there will always be at lease an intersection with itself.