0

I'm working with a very large shapefile in QGIS 3.4. I only need one section of the layer (near the main city, not the areas outside it) and I'm trying to delete the large sections which I don't need. Unfortunately, QGIS keeps crashing (it's fine with all other operations). I got the following error message on the last crash:

Stack Trace


RaiseException :
_TypeMatch :
RtlCaptureContext :
QgsEllipseSymbolLayer::properties :
QgsLineString::points :
QgsCurvePolygon::coordinateSequence :
QgsGeometry::convertPolygon :
QgsGeometry::asMultiPolygon :
QgsGeometryValidator::run :
QThread::start :
BaseThreadInitThunk :
RtlUserThreadStart :




QGIS Info
QGIS Version: 3.2.3-Bonn
QGIS code revision: 9b176802e5
Compiled against Qt: 5.9.2
Running against Qt: 5.9.2
Compiled against GDAL: 2.2.4
Running against GDAL: 2.2.4



System Info
CPU Type: x86_64
Kernel Type: winnt
Kernel Version: 10.0.17134

So far I've tried the following:

1) Deleting vertices in small batches 2) Deleting vertices in large batches 3) Exporting to GeoJSON and reloading the GeoJSON and repeating the above 4) Creating a new shapefile of a bounding polygon for the area I need and using the "clip" geoprocessing tool to isolate the section of the original shapefile I need in a new layer 5) Setting the canvas to the area I need and saving as a new layer using the "map view extent" option under "Layer" > "Save As"

None of the above have worked. The first three all result in QGIS crashing, the fourth does not create a correctly clipped new layer (to get this to run I have had to change "Invalid Features Filtering" to "Ignore features with invalid geometries"), and the fifth doesn't save the map view extent but instead saves the whole of the original shapefile.

I'm thinking option five is the best, but I cannot get it to save just the "map view extent". I have searched the forums for "map view" "map view extent" "canvas extent" but couldn't find anything about the issue. I've also tried this on two different computers but the results are the same.

Does anyone have any ideas?

  • QGIS 3.4 crashes fairly often for me. You might have better success with the old LTR, version 2.18. – csk Jan 18 at 17:23
  • Have you tried selecting the features you want and exporting them to a new layer? – csk Jan 18 at 19:14
  • Are you using 64bit QGIS? 64bit apps normally have access to more memory. – klewis Jan 18 at 20:22
  • I'd love to test this - is your input data public? – ndawson Jan 18 at 23:06
  • 1
    I'd also suggest upgrading to 3.4.3 - 3.2 is no longer supported – ndawson Jan 18 at 23:06
1

The file isn't that big but contains "invalid geometries" (as Gabriel suggested). Grass V.Clean can be used to sort out these issues. However, I managed to clip the area around San Francisco using the GDAL tool "Clip vector to extent" without problem [and without using v.clean first]. I made a shapefile copy of the original layer first as I'm more familiar with this format a. By the way, I was certain there was clip tool that allows you to choose to ignore invalid geometries, but I cant see that option.

  • Thanks very much - this worked perfectly. I really appreciate you taking the time to help. Thanks again! – Gary Jan 23 at 19:45
  • You're welcome Gary. These problems can be a right PITA without some input from another angle. You've got lots of good advice in other replies here for when other issues crop up. If I run into problems I often try a GRASS or SAGA alternative to the native QGIS functions, they've had a longer development time (I believe 30 years in the case of GRASS and originally developed for the US Military) nd sometimes give additional feedback as to what might be going wrong. – SteveW Jan 25 at 17:34
4

Just for completing the previous answers, let me add my points (are general points and apply always, but in your particular case seems to me you should adjust to all three) to be taken into account:

  1. Use valid (and not null nor duplicated) geometries in your layers.

When you encounter a serious error when processing a layer, that layer is likely to contain invalid geometries.
Create a new project, add the conflicting layer and run the Check validity tool with the QGIS method enabled.
Then, and unless you need null geometries for some features that only have attributes in the table, run the Remove null geometries tool on the 'Valid' layer.
Finally, and unless you know that you need duplicate geometries because they have different associated attributes, run the Delete duplcated geometries tool on the 'Non null geometries' layer.
Export the 'Cleaned' output layer and work with that layer instead.

  1. Use the same CRS for all your layers and your project.

Avoid QGIS having to reproject your layers on the fly, either to render them or to run tools on them.
Although the vertices of the polygons can be transformed correctly in the reprojection on the fly, the spatial location of the lines that form its boundaries changes according to whether it is reprojecting it on the fly or not.

  1. Use the latest stable versions of QGIS.

The latest stable versions of QGIS are 3.4.3 for Current, and 2.18.27 for LTR. But you are using 3.2.3.
Many bugs were fixed since then.
Since you use QGIS in Windows, I recommend you do not use stand-alone installers.
Many versions of them, including 3.4.3, have compilation issues that are fixed a few days after the installer is released, but you can not implement them but only by making manual installations from the console.
Use the OSGeo4W Network Installer instead.
Then, running the Setup app in Advanced mode, you can keep the versions of all the packages you have installed up-to-date. And you can use the latest Current and LTR versions without problems in the same machine.

  • Thanks very much for the help. I'm using QGIS 3.4 as well as 3.2 (two different computers) but the results are the same. I've also checked and made sure the project and layer CRS are the same, which they are. Interestingly, when I check the validity, the process runs but does not complete - it stays at 99% and never completes. I'm running this on a very new computer so I'm beginning to think the shapefile might just be too big. Is this possible? The data is from NOAA however so I would be surprised if it were of poor quality. Thanks again for the help. – Gary Jan 22 at 23:57
  • 1
    Thanks very much for the assistance; I really appreciate it. Fortunately, SteveW's solution below using the GDAL Clip Vector to Extent has worked and the issue has been resolved. I learned new things from your suggestions however and I'm grateful for the help. Thanks again! – Gary Jan 23 at 19:49
3

Instead of trying to remove data from the large shapefile, simply export a subset of the data to a new shapefile.

Select the features you want. Right click on the layer name > Export > Save selected features as...

If the layer has attributes you don't need, exclude those attributes from the export. enter image description here

If QGIS crashes when exporting, here are some things you can try before exporting (in increasing order of inconvenience):

  • Turn off rendering before attempting the export (bottom-right corner of the main project window)

enter image description here

  • Un-check the option to "add saved file to map" when you export. After the export finishes, remove the large shapefile from your project, then import the new, smaller shapefile.

  • Start a new QGIS project. Only import the large shapefile, and if necessary one other layer that shows where the city boundary is.

  • Revert to the old LTR, version 2.18.

  • Nice answer. That is pretty much my way to handle this situation too. – Pooneil Jan 20 at 15:35
  • Thanks for the help. I've gone through these steps, but still no success I'm afraid. I appreciate you taking the time to help. Unfortunately the shapefile only has one huge feature so selecting it doesn't make a difference to the process. – Gary Jan 22 at 23:50
  • 1
    @Gary, "one huge feature" is a problem. Parts of a Multipart feature are supposed to be isolated: parts can not be adjacent, parts can't intersect, superimpose, etc. Run the Multipart to single parts tool and check the validity of the singlepart features. I will try to download the data and replicate it. – Gabriel De Luca Jan 23 at 0:12
  • Thanks very much for the help. I have run the Multipart to single parts tool but the outcome of the validity check is still the same unfortunately: it just stays at 99% and never completes (I gave it over 25 minutes to run). Thanks again. – Gary Jan 23 at 0:47
1

I've had a similar problem with large datasets of points (Qgis 2.18). Selecting features manually bogged down the computer and caused crashes. In the end I turned of the projections on the fly option. Then created a new vector layer with a rectangle covering the area of interest in the same CRS as the points. I was then able to clip the points to this rectangle and create a smaller dataset to work from. I found the native Qgis clipping tool more likely to crash than one in the toolbox, I think I used the SAGA one in the end. As CSK said - turn off the "add saved file to map" option as well and only have open the one shapefile.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.