0

The writing of processing scripts in QGIS3 changed substantially. On the basis of many tutorials/blog posts and answers on this site I got a running processing script in QGIS3. But it was, for a lousy programmer like me, a hard piece of work, because of many errors in logic and code I produced.

Here is what I tried:

  • To avoid painfull testing inside QGIS, I decided to import modules into my processing script, that I can test outside of QGIS (I am using PyScripter). To get this working I had to manipulate sys.path, because local imports (in the same (sub-)directory) seem to do not work in processing scripts.
  • When changing something in an imported module, QGIS3 doesn't notice the change in the code, it seems not to re-import when starting a processing script. OK, I maybe can avoid that, by using reload - but I didn't test it
  • I also tried to use a standalone pyqgis script to have the chance to debug it. It works, but it is very confusing, because the stack gets very "long" and while stepping through function calls, I easily loose orientation and can't find out, why an error occured.

So I'm looking for a method or procedure to ease the development of processing scripts.

My processing script is quite straightforward, based on the template by underdark and the foss4g workshop materials and at last a tutorial by Ujaval Gandhi!

Here are my imports and the head of that script

from PyQt5.QtCore import QCoreApplication, QVariant
from qgis.core import (QgsApplication,
                       QgsProcessing,
                       QgsFeatureSink,
                       QgsProcessingException,
                       QgsProcessingAlgorithm,
                       QgsProcessingParameterFeatureSource,
                       QgsProcessingParameterFeatureSink,
                       QgsFeatureRequest)
from qgis.core import (QgsProcessingParameterString,
                       QgsProcessingParameterEnum,
                       QgsCoordinateReferenceSystem,
                       QgsWkbTypes)
import processing, sys

# use this to import local modules
sys.path.append(QgsApplication.qgisSettingsDirPath() + '\\processing\\scripts')
from pomodules.poqgsstations import PoQgsStations
from pomodules.poqgscurrentw import PoQgsCurrentW

class PegelOnlineLoader(QgsProcessingAlgorithm):
    INPUT = 'Query'
    OUTPUT = 'Result'
    QUERY_OPTIONS = ['Stations', 'Waterlevels']

as you can see, i have a directory, called pomodules with modules doing the "real" work (download json data from a rest-api and turn them into a list of QgsFeatures, po stands for PegelOnline, a german [website] (https://www.pegelonline.wsv.de/gast/start) providing the REST-Service)

The here is the main part of my processing script:

    def initAlgorithm(self, config=None):
        """
        Here we define the inputs and output of the algorithm, along
        with some other properties.
        """
        self.addParameter(
            QgsProcessingParameterEnum(self.INPUT,
                                         self.tr(self.INPUT),
                                         self.QUERY_OPTIONS))
        self.addParameter(
            QgsProcessingParameterFeatureSink(self.OUTPUT,
                                              self.tr('Output layer'),
                                              QgsProcessing.TypeVectorAnyGeometry))


    def processAlgorithm(self, parameters, context, feedback):
        """
        Here is where the processing itself takes place.
        """
        qx = self.parameterAsInt(parameters, self.INPUT, context)
        feedback.pushInfo(str(qx))
        query = self.QUERY_OPTIONS[qx]
        feedback.pushInfo(query)

        # we're loading from PegelOnline
        if query == 'Stations':
            stat = PoQgsStations()
        elif query == 'Waterlevels':
            stat = PoQgsCurrentW()

        features = stat.getFeatures()
        # field definitions for stations
        fields = stat.fields
        crs = stat.crs

        # our output to qgis
        sink, dest_id = self.parameterAsSink(parameters, self.OUTPUT, context, fields, QgsWkbTypes.Point, crs)

        feedback.pushInfo(str('Total Length {}'.format(len(features))))

        for f in features:
            sink.addFeature(f, QgsFeatureSink.FastInsert)

        return {self.OUTPUT: dest_id}

As i said the script is working, but I'm unsure if this is good practise. As described above, I also wrote a standalone script for debugging purposes using the answer by Germán Carrillo here. I run this in PyScripter using the debug mode:

import sys
import os
from qgis.core import QgsApplication, QgsProcessingFeedback

pre = os.environ['QGIS_PREFIX_PATH']

QgsApplication.setPrefixPath(pre, True)

qgs.initQgis()
print (QgsApplication.qgisSettingsDirPath())

# Append the path where processing plugin can be found
sys.path.append(pre + '\\python\\plugins')

import processing
from processing.core.Processing import Processing
Processing.initialize()

# parameters needed by the algorithm
params = {
    'Query' : 0,
    'Result' : 'c:/temp/stations.shp'
}
feedback = QgsProcessingFeedback()

res = processing.run('script:PegelOnlineLoader', params, feedback=feedback)
res['Result'] # Access your output layer

You find the complete example on github. I hope you don't mind my long text (the topic is quite new and needs some explanation, I think).

How can I ease the development of processing scripts and get a better way of debugging?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.