2

I have been trying to perform a intersect (then tried using clip) on a rather large dataset on QGIS. I have been able to complete similar geoprocessing using slightly smaller datasets and the layer. For this layer no errors are being reported but the clip/intersect function seems to be unable to move past 28% complete (tried multiple times). Does anyone have any advice or help they can offer beyond trying to maximize my computers available processing capability?

  • 1
    +1! QGIS usually hasn't crashed unless there is a "crash dump written" error message. Some tasks just really take quite some time... – SAnderka Nov 10 '14 at 14:31
  • @SAnderka, good catch with the "crash dump" message! – Joseph Nov 10 '14 at 14:35
  • Thanks Joseph and SAnderka, I had initially thought it was a case of giving it time but left it running over three hours to find no further progress. – Sarah Nov 10 '14 at 15:53
  • I once had dataset over 500 MB and it took over 6 hours to complete. I've read people having to process large datasets for days! If you want to run an analysis for a large dataset, it's probably best to leave it running at night so that hopefully when you come back to it the next day, it's been fully processed. – Joseph Nov 10 '14 at 15:57
  • What do you mean with a rather large dataset? Million or 100 million polygons? Do you plan to clip with a simple rectangle or more complicated geometry? From shapefile or database? – user30184 Nov 10 '14 at 21:38
1

Usually QGIS tends to look as if it's stopped processing or crashed ("Not responding" message) but is in actual fact still processing the data. You can check this via the Task Manager while running your intersect/clip function and see if qgis-bin.exe is using CPU usage. If it is then you may have to wait a while, particularly with chunky datasets.

Depending on the size of your dataset, it is common for processes to last hours if not days. There are ways to decrease processing time such as:

  • Upgrading your CPU
  • Adding more RAM to your Computer
  • Delete unwanted fields/columns in your layers

Also thanks to @SAnderka who also mentioned that if QGIS crashes then there should be a "crash dump written" pop-up error message.

1

For large datasets, I suggest to use the equivalent GDAL functions.

GDAL has a progress bar, and does not waste RAM for a nice looking GUI. It is always delivered with any QGIS version.

  • 1
    +1 for mentioning GDAL, I should also look into this when dealing with large datasets! – Joseph Nov 10 '14 at 16:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.