3

I have successfully done spatial joins with QGIS many times. However, with two shapefiles QGIS hangs or never seems to complete. (I’ve let the process run for over two hours.)

I’ve tried all the obvious things that I can find by searching and reading other posts, e.g., making sure the two shapefiles have the same CRS (link), but it always hangs. What’s more confusing is that I can spatially join both shapefiles with the same other shapefile, but I can’t join them directly. IOW, spatial-join(A,B) and spatial-join(B-C) both work, but spatial-join(A,C) does never completes.

Do you have any tips, suggestions, insights about what might be going on?

DETAILS

I have three layers for a state with successively smaller shapes:

  • Congressional Districts (CDs) — order a dozen shapes
  • Precincts — order 2,500 shapes
  • Blocks — order 300,000 shapes

The CRS for all three layers is EPSG:4269, NAD83. However, the CRS for the Precincts and CDs shapefiles that I got from the state was something else. So, I changed the CRS to EPSG:4269, NAD83 using “Save As …” (per this).

I want to assign Precincts to Blocks and CDs to Blocks, using Vector > Data Management Tools > Join Attributes By Location … Again, I’ve done this kind of operation with many shapefiles successfully, so I believe there’s something subtle going on somewhere that I’m missing.

Vector > Data Management Tools > Join Attributes By Location ... works for

  • Blocks and Precincts layer with “Target vector layer” = Blocks and “Join vector layer” = Precincts; and
  • Precincts and CD layer with “Target vector layer” = Precincts and “Join vector layer” = CDs

However, it hangs, never completes, for

  • Blocks and CDs layer with “Target vector layer” = Blocks and “Join vector layer” = CDs

The first two spatial joins complete relatively quickly, taking no more than a minute or two. The latter, transitive spatial join never completes though. As I said, I’ve let it run for two hours to no avail.

NOTE: The only reason I spatially joined Precincts and CDs was to see if it would work. (It does.) A fraction of Precincts actually span CDs which is why need to do this transitive join.

In the process of making sure everything was set up right, I made the shapes in the coarsest layer (CDs) transparent, so I could see the smaller shapes (blocks) through them, and the align correctly. So, I believe the coordinate systems are set up correctly.

I’m running QGIS 2.8 on Mac OS X 10.11.3 (El Capitan).

Improve this question
9
  • Alec, have you checked the shapefiles' geometry? I've experienced the kind of problems that you've described, only to have them solved by checking, then repairing geometry. Bad geometry is often unnoticeable until some particular action is attempted with the data. There is a topology checker plugin available.
    – Stu Smith
    Feb 10, 2016 at 2:18
  • Thanks for the tip, Stu!. I haven't checked the geometry, but I will look into the Topology Checker Plugin. I'll see what the Web says, but LMK if there are particular rules you configure the checker to check for. Thanks!
    – Alec Ramsay
    Feb 10, 2016 at 14:45
  • I found this [post] (faunaliagis.wordpress.com/2013/08/14/…) that referenced a built-in tool: Vector > Geometry Tools > Check Geometry Validity ... I ran that on both layers, and it didn't find any problems.
    – Alec Ramsay
    Feb 10, 2016 at 16:21
  • Then I ran the TopologyChecker plug-in on both layers. No errors on the CDs layer (w/ rules must not have duplicates | must not have gaps | must not have invalid geometries | must not have multi-part geometries | must not overlap). No errors on the Blocks layer, except for several multi-part geometries for islands. Are they a problem for spatial join? If so, how can spatial-join(A,B) and spatial-join(B,C) work but spatial-join(A,C) not, if A is the Blocks layer with some multi-part geometries and C is the much coarser CDs layer? In the meantime, I will explore making single parts.
    – Alec Ramsay
    Feb 10, 2016 at 16:28
  • There's a couple of other spatial join tools available which you could try (Spatial Join from the MMQGIS plugin; and Join attributes by location from the Processing Toolbox). If these also fail then perhaps you could share your data publicly so that others could test it (if the data is not sensitive ofcourse).
    – Joseph
    Feb 11, 2016 at 11:08

1 Answer 1

Reset to default
1

It turns out that there wasn’t a problem, that QGIS didn’t hang, it just appeared to.

When I changed my settings to prevent my machine from going to sleep and let the process run for a long time (over night), it completed. I have no idea how long it actually took, but it was more than an hour or two of the CPU being pegged near 100% utilization. That shows up in Activity Monitory like the app is hung.

I don't know what the algorithm is, but some combination of the performance of the algorithm and the large size of the data sets made this take a long, long time and appear hung. The progress bar also never showed any progress while I watched it, so I had no indication that QGIS was actually doing anything.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.