3

I'm trying to apply the 'join attributes by nearest' function on a quite big dataset: A multipolygon layer with ~ 415.000 features (1.11GB) and a point layer with ~ 305.000 features (~320 MB). Just to give you an idea on the data, I have added a small snippet of how the data looks like.

enter image description here

As can be seen the polygon layer is pretty complex, and I think, this is the problem.

I tried the 'join attributes by nearest' on a small clip of the data, and it works like a charm and gives exactly the output I need. However, when I run it with the whole data set, the first 50% are pretty fast (it takes a couple of seconds to reach up to 50%), and then it gets more or less stuck and afterwards takes ~ 1h per 1%.

I know this is a broad question, however, if someone has experience how to accelerate the processing of this function (join attributes by nearest) I would be very glad to hear about! I'm of course also open for other methods to use for this kind of problem.

I hope this question doesn't get closed as I understand, that it is relatively broad. I just hoped, there would be someone who had to solve a similar problem with the same function already, too.

Windows 10
QGIS 3.28 Firenze

5
  • 2
    Try the three steps in: gis.stackexchange.com/questions/440360/… and set a max distance
    – BERA
    May 23, 2023 at 10:34
  • 1
    I've also had problems with large datasets using QGIS and I've only been able to solve them using PostGIS.
    – jpinilla
    May 23, 2023 at 10:38
  • Thank you so much for this fast answer :) I will come back when I tried it out!
    – i.i.k.
    May 23, 2023 at 10:39
  • 2
    Ensure also that both layers have spatial indexes
    – Matt
    May 23, 2023 at 10:52
  • 1
    @BERA Thank you very much, the data now, after using 'multipart to singlepart' on the polygon layer it only took ~ 3-4 MIN when applying the 'join by nearest' !! Before that, the process was running for almost 17 HOURS when I stopped it. Thank you so, so much :) Why does this have such a better performance?
    – i.i.k.
    May 23, 2023 at 11:28

1 Answer 1

4

Based on the answer of @BERA I first applied the 'Multipart to Singlepart' tool on the Polygon layer, and then creating spatial indices for both layer. When I then apply the 'join attributes by nearest' it only takes 3-4 min to get the result.

As I had set spatial indices also before applying the Multipart-to-Singlepart'-tool, and the performance was still very bad, the real difference here makes creating singlepart-Polygons.

Another update for even bigger data: After applying 'Multipart to singlepart' apply 'subdivide', then again 'multipart to singlepart' on the subdivided singleparts, and of course, in the end, create a spatial index.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.