1

I have a issue about memory management in PyQGIS during loop.

I'm writing a code where there is a while loop.

During the operatione inside the loop that are almost the same,I noticed that the execution time increases and also the RAM occupation too (from 200MB to 128GB). So I tried to write a simplified test script that always executes the same instructions 10.000 times but I notice once again that the execution time increases and the RAM too.

The test script:

  • input1: dots shape (23.000 points)
  • input2: lines shape (115.000 lines)
  • execution: extracts a point from the points shape (always the
    same)-->buffer of 10.000m around the point-->extraction of the lines in the buffer. Cycle 10.000 times.

Below you can find the code and a graph of the times laps for each cycle.

Can you explain me why time increases even if the instructions for each cycle are always the same?

I assume that the memory is not freed every cycle.

Is there a way to fix and keep the time the same every loop?

    #1A -  spatial index on lines
    alg_params = {
        'INPUT': parameters['inputline']#input lines
    }
    outputs['ASpatialIndexOnLines'] = processing.run('native:createspatialindex', alg_params, context=context, is_child_algorithm=True)
    feedback.setCurrentStep(1)
   
    n_loop=0
    while n_loop<10000: #loop 10.000 times
        start_time=time.time()
        
        # 1 - extract random point
        alg_params = {
            'FIELD': 'k',
            'INPUT': parameters['inputpoints'],#input points
            'OPERATOR': 0,  # =
            'VALUE': 100, #extract always point with k=100
            'OUTPUT': QgsProcessing.TEMPORARY_OUTPUT
        }
        outputs['ExtractRandomPoint'] = processing.run('native:extractbyattribute', alg_params, context=context,  is_child_algorithm=True)

        feedback.setCurrentStep(2)
        

        # 2 - Buffer 10000m on random point
        alg_params = {
            'DISSOLVE': False,
            'DISTANCE': 10000,
            'END_CAP_STYLE': 0,  # Arrotondato
            'INPUT': outputs['ExtractRandomPoint']['OUTPUT'],
            'JOIN_STYLE': 0,  # Arrotondato
            'MITER_LIMIT': 2,
            'SEGMENTS': 5,
            'OUTPUT': QgsProcessing.TEMPORARY_OUTPUT
        }
        outputs['Buffer10000mOnRandomPoint'] = processing.run('native:buffer', alg_params, context=context,  is_child_algorithm=True)

        feedback.setCurrentStep(3)
        

        # 3 - spatial index on buffer
        alg_params = {
            'INPUT': outputs['Buffer10000mOnRandomPoint']['OUTPUT']
        }
        outputs['SpatialIndexOnBuffer'] = processing.run('native:createspatialindex', alg_params, context=context,  is_child_algorithm=True)

        feedback.setCurrentStep(4)
       

        # 4 - extract lines in buffer
        alg_params = {
            'INPUT': outputs['ASpatialIndexOnLines']['OUTPUT'],
            'INTERSECT': outputs['SpatialIndexOnBuffer']['OUTPUT'],
            'PREDICATE': [0],  # interseca
            'OUTPUT': QgsProcessing.TEMPORARY_OUTPUT
        }
        outputs['ExtractLinesInBuffer'] = processing.run('native:extractbylocation', alg_params, context=context,  is_child_algorithm=True)
       
        
        end_time=time.time()
        delta_time=round(end_time-start_time,3)
        feedback.pushInfo('{},{}'.format(delta_time,n_loop)) #write time and loop
        n_loop+=1
        if feedback.isCanceled():
            return {}
        
        
        
    # when loop finish-->out_test, last extraction
    alg_params = {
        'INPUT': outputs['ExtractLinesInBuffer']['OUTPUT'],
        'OUTPUT': parameters['out_test']
    }
    outputs['fixGeometries'] = processing.run('native:fixgeometries', alg_params, context=context, is_child_algorithm=True)    
        
        
    return {}

Time(s) Vs Loop

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.