I have been trying to perform a intersect (then tried using clip) on a rather large dataset on QGIS. I have been able to complete similar geoprocessing using slightly smaller datasets and the layer. For this layer no errors are being reported but the clip/intersect function seems to be unable to move past 28% complete (tried multiple times). Does anyone have any advice or help they can offer beyond trying to maximize my computers available processing capability?
-
1+1! QGIS usually hasn't crashed unless there is a "crash dump written" error message. Some tasks just really take quite some time...– SAnderkaCommented Nov 10, 2014 at 14:31
-
@SAnderka, good catch with the "crash dump" message!– JosephCommented Nov 10, 2014 at 14:35
-
Thanks Joseph and SAnderka, I had initially thought it was a case of giving it time but left it running over three hours to find no further progress.– SarahCommented Nov 10, 2014 at 15:53
-
I once had dataset over 500 MB and it took over 6 hours to complete. I've read people having to process large datasets for days! If you want to run an analysis for a large dataset, it's probably best to leave it running at night so that hopefully when you come back to it the next day, it's been fully processed.– JosephCommented Nov 10, 2014 at 15:57
-
What do you mean with a rather large dataset? Million or 100 million polygons? Do you plan to clip with a simple rectangle or more complicated geometry? From shapefile or database?– user30184Commented Nov 10, 2014 at 21:38
Add a comment
|
2 Answers
Usually QGIS tends to look as if it's stopped processing or crashed ("Not responding" message) but is in actual fact still processing the data. You can check this via the Task Manager while running your intersect/clip function and see if qgis-bin.exe is using CPU usage. If it is then you may have to wait a while, particularly with chunky datasets.
Depending on the size of your dataset, it is common for processes to last hours if not days. There are ways to decrease processing time such as:
- Upgrading your CPU
- Adding more RAM to your Computer
- Delete unwanted fields/columns in your layers
Also thanks to @SAnderka who also mentioned that if QGIS crashes then there should be a "crash dump written" pop-up error message.
For large datasets, I suggest to use the equivalent GDAL functions.
GDAL has a progress bar, and does not waste RAM for a nice looking GUI. It is always delivered with any QGIS version.
-
1+1 for mentioning GDAL, I should also look into this when dealing with large datasets!– JosephCommented Nov 10, 2014 at 16:35