I am trying to dissolve grid spaces by category using GeoPandas' dissolve function, but it doesn't finish after hours of running and I want to make sure I'm not doing something wrong and/or there isn't some way to speed it up.

Here's what I've got so far:

outputData = gp.GeoDataFrame(outputData[['category','geometry']])
outputData['geometry'] = outputData['geometry'].apply(wkt.loads)
outputDataFrame = outputData.dissolve(by='category')

Here's the breakdown: I've got a Dataframe called outputData with many variables/columns, so I limit it to the two I need and convert it into a geoDataFrame. In order to use the geometry column, I need to apply wtk.loads else I get an error about it being a str type. There are 12 values in the category column (0-11) and around 200,000 rows in the DataFrame.

I've run dissolve before (e.g. to get region boundaries from counties) but never on so many geometries. I'm thinking that because in this case the geometries are all simple and non-overlapping (a grid) there may be a better way to combine them. But if dissolve is just groupby + unary_union, then I don't know what could be better than that.

Is this a known unsolvable problem with large datasets or are there any tricks to speed it up significantly (e.g., by setting an option or using a different library/function)?

  • 1
    Are you using a recent version of geopandas and its dependencies? – bugmenot123 Jun 20 '19 at 8:28
  • Does the same code work on a smaller dataset with similar characteristics? – PolyGeo Jun 20 '19 at 8:32
  • I was using geopandas 0.4.0, so I upgraded to 0.5.0. After doing that, I was able to dissolve the first 500 rows in 0.10 seconds. If it were linear timing, the full dataset would complete in about 40s, but it hasn't finished after many minutes. – Aaron Bramson Jun 20 '19 at 9:03
  • 1
    Please Edit the question to contain this information. – Vince Jun 20 '19 at 10:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.