3
registry = QgsProject.instance()

first_layer = registry.mapLayersByName('buffered')[0]
second_layer = registry.mapLayersByName('buffered')[0]

areas = []
instrument_number = "tin_ttl_in" 
proprietor = "tin_ttl_re" #input field name for the registered proprietor


SLayerList = second_layer.getFeatures()

for first_feature in SLayerList:
    tempLayer =first_layer.getFeatures(QgsFeatureRequest().setFilterRect(first_feature.geometry().boundingBox()))
    for second_feature in tempLayer:
       if first_feature.geometry().overlaps(second_feature.geometry()):
           if (first_feature[instrument_number] == second_feature[instrument_number]) or (first_feature[proprietor] == second_feature[proprietor]):
            areas.append(first_feature.id())

first_layer.select(areas)

I'm having trouble with this script running very slow and I believe it could be because I'm making too many calls to getFeatures() or having a nested for loop? But I am not hugely experienced in Python or pyqgis (only started in the last month).

The data set I'm using currently only has around 35,000 polygons in it and it takes probably 20-30 mins to complete but I plan to be able to run this script on data sets of around 200,000 so definitely need some other option.

I'm not aiming to have this run in seconds as that's unrealistic, but do have a goal of having it run in around 5 minutes.

  • Speed isn't python's strong point. Do you have any other programming languages available that compile? like C++, C#. To speed this up I think you need to leave QGIS and python and run directly with OGR. – Michael Stimson Jan 16 at 0:25
  • Well I don't have any experience using those languages and not 100% how OGR works but i'll have a search. – yeslek66 Jan 16 at 0:50
  • C# is easier to learn than C++ but isn't necessarily free, Microsoft had an express version that required registration as a sampler; C# is only available in Windows but C++ exists in Linux and Mac. C# (and VB.net, both get compiled to common run-time) libs for GDAL/OGR can be gained from gisinternals.com ; you can employ multiprocessing though that's not advisable for a beginner. – Michael Stimson Jan 16 at 1:10
4

You could get a HUGE speed boost by iterating through the features in the first layer and building a QgsSpatialIndex on them. Then query the index when iterating through the second layer to determine what features possibly overlap.

Secondly, you can get another huge speed boost by preparing the geometry from the features from your outer loop. Since these are potentially compared against many other features, preparing them once will lead to a speedup of an order of magnitude or more. This is done by calling:

geom = first_feature.geometry()
engine = QgsGeometry.createGeometryEngine(geom.constGet())
engine.prepareGeometry()

...

if engine.overlaps(second_feature.geometry().constGet()):
    ...
  • I will give these two ideas a try. Sounds like that would work excellently. Appreciate the help! – yeslek66 Jan 16 at 19:34
1

One simple thing that might help is to swap your if statements order around. Checking geometry for overlaps is computationally more expensive than what I assume are string comparisons you do to compare 'instrument_number' and 'proprietor'.

If this can reduce a few of the geom checks by exiting before they are executed it may shave off some time. How well this works is dependent on how much it reduces the next overlap step.

Further to this I would try to include the check of 'instrument_number' and 'proprietor' as another filter when you create 'tempLayer' as you have all the required information from 'first_feature' at that point. I'm not sure the best way to go about this as I'm not familiar with pyQGIS.

None the less by doing this you would reduce the amount of times you pass through that second for-loop. It moves the overhead of the if-statement from the more expensive Python code and push it to the underlying QGIS code which is probably C or C++ already.

  • That is a good point! I hadn't thought of that but I agree that looking into the attribute check in a filter may be more beneficial to cutting out unnecessary features. Cheers for the help! – yeslek66 Jan 16 at 19:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.