2

I wrote this script to classify a shapefile in 5 classes, this works for small files (minor 8000 features), for larger files, the process has stopped. Can make this better?

#Load the shape
vrAdubo = QgsVectorLayer(vetor_saida_adubo, "Adubo", "ogr")

if not vrAdubo:
   progress.setText("Erro ao carregar shape resultante Adubo")


#Insert fields area_ha,  classe e kmph
provVrAdub = vrAdubo.dataProvider()
provVrAdub.addAttributes([QgsField("area_ha", QVariant.Double), QgsField("classe", QVariant.Int), QgsField("kmph", QVariant.Double), QgsField("area_class", QVariant.Double),
QgsField("perc_class", QVariant.Double)])
vrAdubo.updateFields()

#Set fields valures
vrAdubo.startEditing()
iterVrAdub = vrAdubo.getFeatures()

#variaveis areas totais por classe
area_cls1 = 0.000
area_cls2 = 0.000
area_cls3 = 0.000
area_cls4 = 0.000
area_cls5 = 0.000

#Setting filds
for featAdubo in iterVrAdub:
    geom = featAdubo.geometry()
    featAdubo['area_ha'] = geom.area() / 10000
    areatt_ad += featAdubo['area_ha'] #area total mapeada
    ct_pol_adu += 1 #contagem de poligonos
    featAdubo['kmph'] = featAdubo['Speed'] * 3.6  

#Classify by ranges
    if featAdubo['AppldRate'] > 0.0000:
        if featAdubo['AppldRate'] < 400.000:
            featAdubo['classe'] = 1
            area_cls1 += featAdubo['area_ha']
        elif featAdubo['AppldRate'] >= 400.000 and featAdubo['AppldRate']< 450.000:
            featAdubo['classe'] = 2
            area_cls2 += featAdubo['area_ha']
        elif featAdubo['AppldRate'] >= 450.000  and featAdubo['AppldRate']< 550.000:
            featAdubo['classe'] = 3
            area_cls3 += featAdubo['area_ha']
        elif featAdubo['AppldRate'] >= 550.000  and featAdubo['AppldRate']< 600.000:
            featAdubo['classe'] = 4
            area_cls4 += featAdubo['area_ha']
        else:
            featAdubo['classe'] = 5
            area_cls5 += featAdubo['area_ha']
    else:
        progress.setText("Aplicacao zerada em AppIdRate: %" % featAdubo.id()) 

    vrAdubo.updateFeature(featAdubo)

vrAdubo.commitChanges()
  • What do you mean with: 'the process has stopped'. Which error message do you get? Without knowing this, I would suggest to call changeAttributeValue() instead of updateFeature(). Even better in this case: provVrAdub.changeAttributeValues({featAdubo.id(): {fieldindex, class_value}}) with fieldindex as provVrAdub.fieldNameMap()['classe'] – Detlev Mar 23 '16 at 19:16
  • @Detlev I sorry, my english is bad, the process "not responding" in taskmanager and was running a few hours, the file is a shapefile with 180.000 polygons. So 180.000 registries. Any error messages. Just stops. – Franklin Januário Mar 23 '16 at 19:28
  • Ok. Please try to change your code to change attributes on the dataProvider. The documentation states the updateFeature() '... needs to query the datasource on every call' - this sounds like poor performance. And writing to dataProvider lacks the overhead of editing the layer. – Detlev Mar 23 '16 at 19:35
  • For testing I try this:fin_id = 1 for featAdubo in iterVrAdub: provVrAdub.changeAttributeValue(featAdubo.id(), featAdubo['id'].index, fin_id, QVariant()) fin_id = fin_id + 1 vrAdubo.commitChanges()But return QgSVectorProvider don't have changeAttributeValue – Franklin Januário Mar 23 '16 at 20:11
  • changeAttributeValues(). Did you miss the trailing "s"? – Detlev Mar 23 '16 at 20:19
1

Using QgsMapLayer.updateFeature() on many features is time consuming. The documentation states that updateFeature() '... needs to query the datasource on every call'. In your case that is 180.000 times a maximum of 180.000 features.

Another aspect is to edit a layer. Editing layer brings some overhead for transaction management. I think in your use case this is not necessary.

You gain performance with methods writing once to the data provider, namely changeAttributeValues(). Replace the if..else construct and updateFeatures() in your code with following lines (elif must be filled in):

fieldindex = provVrAdub.fieldNameMap()['classe']
update_dict = {}

for featAdubo in iterVrAdub:
    # ...
    if featAdubo['AppldRate'] < 400.000:
        class_value = 1
    # elif ...
    # else:
    #     ...
    update_dict[featAdubo.id()] = {fieldindex: class_value}

provVrAdub.changeAttributeValues(update_dict) 

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.