Gdal is known for being the fastest library for geospatial data processing. I'm using it in my python script and I'm having issues with performance on a specific task which is Erasing a layer from another layer which takes too much time!

I used first this prebuilt function:

inputLayer.Erase(maskLayer, ResultLayer)

This works fine for a layer with average number of objects but when running it with a layer that has a 200,000 features it takes forever!

So I managed to write this bit of code:

        for featureI in inputLayer:
            geomI = featureI.GetGeometryRef()
            for featureM in maskLayer:
                geomM = featureM.GetGeometryRef()

But this will last 1200 min which is too long for my case!

Is there any way to speed things up? Do you have any suggestions to make for me?

I'm using python3.5 with GDAL2,

  • no it's not ! the other post solution is my problem ! can u read the two posts so that u can have a better understanding ? – Abdelaziz Nov 17 '17 at 10:23
  • If you're interested in performance code in C++ or C#/VB.net, there are GDAL/OGR bindings available for these languages. You could be experiencing overhead by the to-and-fro from python interpretation to compiled C libs; I can personally attest that GDAL/OGR in C++ is significantly faster than in python, so much so that I only use GDAL in python now for very simple/small tasks.. – Michael Stimson Nov 17 '17 at 11:19
  • 2
    python as a language isn't built for speed, its main focus was/is ease of use, to get any real kind of performance a compiled language is necessary... after all you don't jump on a bicycle and expect to keep up with the traffic on the freeway. – Michael Stimson Nov 17 '17 at 11:32
  • 1
    There's this post gis.stackexchange.com/questions/151699/… but I'm not sure that is of much help. It seems that -clipsrc will only retain features within the clip geometry. You could try sub-setting the features in the layer and run over many cores, I would need to know much more about your features and their storage type to guide you in this. – Michael Stimson Nov 17 '17 at 12:09
  • 1
    i resolved my performance issue by using parallel processing for now. next time i will be building a batch i'll think of using a compiled programming language , thanks Michael – Abdelaziz Nov 29 '17 at 9:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.