Problem editing, clipping, and saving large shapefile QGIS 3.4

Question

I'm working with a very large shapefile in QGIS 3.4. I only need one section of the layer (near the main city, not the areas outside it) and I'm trying to delete the large sections which I don't need. Unfortunately, QGIS keeps crashing (it's fine with all other operations). I got the following error message on the last crash:

Stack Trace


RaiseException :
_TypeMatch :
RtlCaptureContext :
QgsEllipseSymbolLayer::properties :
QgsLineString::points :
QgsCurvePolygon::coordinateSequence :
QgsGeometry::convertPolygon :
QgsGeometry::asMultiPolygon :
QgsGeometryValidator::run :
QThread::start :
BaseThreadInitThunk :
RtlUserThreadStart :




QGIS Info
QGIS Version: 3.2.3-Bonn
QGIS code revision: 9b176802e5
Compiled against Qt: 5.9.2
Running against Qt: 5.9.2
Compiled against GDAL: 2.2.4
Running against GDAL: 2.2.4



System Info
CPU Type: x86_64
Kernel Type: winnt
Kernel Version: 10.0.17134

So far I've tried the following:

1) Deleting vertices in small batches 2) Deleting vertices in large batches 3) Exporting to GeoJSON and reloading the GeoJSON and repeating the above 4) Creating a new shapefile of a bounding polygon for the area I need and using the "clip" geoprocessing tool to isolate the section of the original shapefile I need in a new layer 5) Setting the canvas to the area I need and saving as a new layer using the "map view extent" option under "Layer" > "Save As"

None of the above have worked. The first three all result in QGIS crashing, the fourth does not create a correctly clipped new layer (to get this to run I have had to change "Invalid Features Filtering" to "Ignore features with invalid geometries"), and the fifth doesn't save the map view extent but instead saves the whole of the original shapefile.

I'm thinking option five is the best, but I cannot get it to save just the "map view extent". I have searched the forums for "map view" "map view extent" "canvas extent" but couldn't find anything about the issue. I've also tried this on two different computers but the results are the same.

Does anyone have any ideas?

Answer 1

The file isn't that big but contains "invalid geometries" (as Gabriel suggested). Grass V.Clean can be used to sort out these issues. However, I managed to clip the area around San Francisco using the GDAL tool "Clip vector to extent" without problem [and without using v.clean first]. I made a shapefile copy of the original layer first as I'm more familiar with this format a. By the way, I was certain there was clip tool that allows you to choose to ignore invalid geometries, but I cant see that option.

Answer 2

Just for completing the previous answers, let me add my points (are general points and apply always, but in your particular case seems to me you should adjust to all three) to be taken into account:

1. Use valid (and not null nor duplicated) geometries in your layers.

When you encounter a serious error when processing a layer, that layer is likely to contain invalid geometries.
Create a new project, add the conflicting layer and run the Check validity tool with the QGIS method enabled.
Then, and unless you need null geometries for some features that only have attributes in the table, run the Remove null geometries tool on the 'Valid' layer.
Finally, and unless you know that you need duplicate geometries because they have different associated attributes, run the Delete duplcated geometries tool on the 'Non null geometries' layer.
Export the 'Cleaned' output layer and work with that layer instead.

2. Use the same CRS for all your layers and your project.

Avoid QGIS having to reproject your layers on the fly, either to render them or to run tools on them.
Although the vertices of the polygons can be transformed correctly in the reprojection on the fly, the spatial location of the lines that form its boundaries changes according to whether it is reprojecting it on the fly or not.

3. Use the latest stable versions of QGIS.

The latest stable versions of QGIS are 3.4.3 for Current, and 2.18.27 for LTR. But you are using 3.2.3.
Many bugs were fixed since then.
Since you use QGIS in Windows, I recommend you do not use stand-alone installers.
Many versions of them, including 3.4.3, have compilation issues that are fixed a few days after the installer is released, but you can not implement them but only by making manual installations from the console.
Use the OSGeo4W Network Installer instead.
Then, running the Setup app in Advanced mode, you can keep the versions of all the packages you have installed up-to-date. And you can use the latest Current and LTR versions without problems in the same machine.

Answer 3

Instead of trying to remove data from the large shapefile, simply export a subset of the data to a new shapefile.

Select the features you want. Right click on the layer name > Export > Save selected features as...

If the layer has attributes you don't need, exclude those attributes from the export.

If QGIS crashes when exporting, here are some things you can try before exporting (in increasing order of inconvenience):

• Turn off rendering before attempting the export (bottom-right corner of the main project window)

• Un-check the option to "add saved file to map" when you export. After the export finishes, remove the large shapefile from your project, then import the new, smaller shapefile.

• Start a new QGIS project. Only import the large shapefile, and if necessary one other layer that shows where the city boundary is.

• Revert to the old LTR, version 2.18.

Answer 4

I've had a similar problem with large datasets of points (Qgis 2.18). Selecting features manually bogged down the computer and caused crashes. In the end I turned of the projections on the fly option. Then created a new vector layer with a rectangle covering the area of interest in the same CRS as the points. I was then able to clip the points to this rectangle and create a smaller dataset to work from. I found the native Qgis clipping tool more likely to crash than one in the toolbox, I think I used the SAGA one in the end. As CSK said - turn off the "add saved file to map" option as well and only have open the one shapefile.