2

I'm trying to get the area of some flooding data. Overall, the dataset is huge and hardly to handle in QGIS. The dataset comes in several shapefiles ranging from 60 Mb to 4 Gb and include several overlapping polygons that are classified.

In addition, the data is not really well optimized. Doing operations like "dissolve" tell me that the geometry is invalid. If I go for "repair geometry" it takes ages. Literally everything takes ages although the amount of objects is not that high. Even if I do some testing on the smallest dataset, it takes a lot longer than usual for everything. Could it be the amount of vertices that crash everything? Here's an example:

enter image description here

enter image description here

So is there any way to just get the outline? So far I was only successful doing "dissolve" after "repair" but now I'm stuck on the largest datasets. And even with the data that I already processed (the data that is already dissolved) there is no way using it in further spatial analysis due to performance issues. I already tried buffering, simplify, filtering it in several steps and even went for a conversion to lines. Only thing I need is a rough outline of the area.

3
  • 1
    Shapefiles exceeding 2GiB are non-conformant, and therefore not shapefiles.
    – Vince
    Feb 6 at 20:44
  • 2
    Your polygons seem to have been generated by brute force from raster data. I would try the Simplify tool which could radically reduce your point count.
    – Llaves
    Feb 7 at 2:10
  • Simplify alters the data heavily. But this is a step I use later. So far there is no answer to my question (there was one but it got deleted?). The process just takes time and I'm not very positive that I could handle the largest dataset. The most time consuming algorithm is dissolve which is also one of the most important. I'd rather use raster data but this is not available unfortunalely. Feb 10 at 16:12

1 Answer 1

0

If you are able to run standalone python scripts you could try using geofileops. Especially for dissolve it should be a lot faster.

Small code snippet showing how you can use geofileops.dissolve:

import geofileops as gfo

gfo.dissolve(
    input_path="...",
    output_path="...",
    explodecollections=True,
)

Disclaimer: I'm the developer of geofileops

1
  • I'll try that when I have the time. Thanks for your answer. Feb 19 at 20:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.