5

I am trying to output the result of a QGIS 3 model as a CSV file. At first I came across How to export csv/excel in QGIS3.0 model processor? but the accepted answer is not satisfying to me. Further researches only provided the code for QgsVectorFileWriter and how to output a CSV in general, but no information about the vectorFileName to use in QGIS models.

Unfortunately there is no predefined algorithm similar to "package layers" for CSV files.

So I was trying to modify the script itself using QgsVectorFileWriter like this:

QgsVectorFileWriter.writeAsVectorFormat(results, r"C:\tmp\xy.csv", "utf-8", None, "CSV", layerOptions ='GEOMETRY=AS_WKT')

Unfortunately I dont know much about Python yet, so I am struggling to implement this correctly. I think the main issue is which vectorFileName (e.g. results or finaloutput) I should use and where to place this output code correctly.


I now managed to create a partially working script (simplified model which adds fid and id field to points as example):

from qgis.core import QgsProcessing
from qgis.core import QgsProcessingAlgorithm
from qgis.core import QgsProcessingMultiStepFeedback
from qgis.core import QgsProcessingParameterVectorLayer
from qgis.core import QgsProcessingParameterFeatureSink
from qgis.core import QgsVectorFileWriter
from qgis.core import QgsVectorLayer
from qgis.core import QgsCoordinateReferenceSystem
import processing


class Model(QgsProcessingAlgorithm):

    def initAlgorithm(self, config=None):
        self.addParameter(QgsProcessingParameterVectorLayer('input', 'input', types=[QgsProcessing.TypeFile], defaultValue=None))
        #self.addParameter(QgsProcessingParameterFeatureSink('Finaloutput', 'finaloutput', type=QgsProcessing.TypeFile, createByDefault=True, defaultValue=None)) #Using this, no CSV will be created
        self.addParameter(QgsProcessingParameterFeatureSink('Finaloutput', 'finaloutput', type=QgsProcessing.TypeFile, createByDefault=True, defaultValue='C:/tmp/xy.csv')) #Using this, two CSV will be created
        #self.addParameter(QgsProcessingParameterFeatureSink('Finaloutput', 'finaloutput', type=QgsProcessing.TypeVectorAnyGeometry, createByDefault=True, defaultValue=None)) #Using this no CSV will be created

    def processAlgorithm(self, parameters, context, model_feedback):
        # Use a multi-step feedback, so that individual child algorithm progress reports are adjusted for the
        # overall progress through the model
        feedback = QgsProcessingMultiStepFeedback(1, model_feedback)
        results = {}
        outputs = {}

        # result
        alg_params = {
            'FIELDS_MAPPING': [{'expression': '$id', 'length': 10, 'name': 'fid', 'precision': 0, 'type': 4},{'expression': '$id-1', 'length': 10, 'name': 'id', 'precision': 0, 'type': 4}],
            'INPUT': parameters['input'],
            'OUTPUT': parameters['Finaloutput']
        }
        outputs['Result'] = processing.run('qgis:refactorfields', alg_params, context=context, feedback=feedback, is_child_algorithm=True)
        results['Finaloutput'] = outputs['Result']['OUTPUT']
        layer = QgsVectorLayer(results['Finaloutput'], "name", "ogr")
        crs = QgsCoordinateReferenceSystem("EPSG:4326")
        QgsVectorFileWriter.writeAsVectorFormat(layer, r'c:\tmp\ab.csv', "utf-8", crs, "CSV", False, [""], layerOptions=["GEOMETRY=AS_YX","SEPARATOR=SEMICOLON"])
        return results

    def name(self):
        return 'model'

    def displayName(self):
        return 'model'

    def group(self):
        return ''

    def groupId(self):
        return ''

    def createInstance(self):
        return Model()

So first it turned out I need to create a QgsVectorLayer such as layer = QgsVectorLayer(results['Finaloutput'], "name", "ogr") first.

Secondly, the API docs as well as the given error messages are really confusing. The main issue was that arguments for QgsVectorFileWriter.writeAsVectorFormat were given the wrong way. First it needs a CRS given as 4th argument instead of None and secondly the 7th argument needs to be a list and not a string. Also the layerOptions needs to be a list and not a string. Using @Fran Raga's hint I figured out this real error (First I thought I need to output the CSV after return because the error then disappeared...).

At last, even more confusing: Using @Joseph's hint it now works partially. But only when setting a defaultvalue like

self.addParameter(QgsProcessingParameterFeatureSink('Finaloutput', 'finaloutput', type=QgsProcessing.TypeFile, createByDefault=True, defaultValue='C:/tmp/xy.csv'))

It then outputs two CSV: "xy.csv" (comma as separator) as well as "ab.csv" (semicolon as separator). When leaving

self.addParameter(QgsProcessingParameterFeatureSink('Finaloutput', 'finaloutput', type=QgsProcessing.TypeFile, createByDefault=True, defaultValue=None)) 

no CSV at all is written. Overall, no WKT or YX geometry is written.


How can I properly export the result of a QGIS 3 model to a CSV using pyqgis with predefined settings such as path, delimiter and geometrytype?

  • 3
    You has the call return before creating the csv, it never executes the writing of the csv. Put the return after creating the csv file – Fran Raga Oct 6 at 18:10
  • This throws the error TypeError: QgsVectorFileWriter.writeAsVectorFormat(): arguments did not match any overloaded call: overload 1: argument 1 has unexpected type 'dict'. When using results['Finaloutput'] I get TypeError: QgsVectorFileWriter.writeAsVectorFormat(): arguments did not match any overloaded call: overload 1: argument 1 has unexpected type 'str'. – MrXsquared Oct 6 at 18:38
  • 2
    @MrXsquared - You could hardcode the path when setting the defaultValue parameter of the final output: self.addParameter(QgsProcessingParameterFeatureSink('Finaloutput', 'finaloutput', type=QgsProcessing.TypeFile, createByDefault=True, defaultValue='C:/tmp/xy.csv')) – Joseph Oct 7 at 10:14
  • @Joseph thanks to your suggestion setting a different defaultvalue QgsVectorFileWriter.writeAsVectorFormat(layer, r'c:\tmp\ab.csv', "utf-8", crs, "CSV", False, [""], layerOptions=["GEOMETRY=AS_YX","SEPARATOR=SEMICOLON"]) now works, it creates two csv: one with the correct separator (semicolon) "ab.csv" and one with comma "xy.csv". Weird thing is, it does not export any csv when leaving self.addParameter(QgsProcessingParameterFeatureSink('Finaloutput', 'finaloutput', type=QgsProcessing.TypeFile, createByDefault=True, defaultValue=None). Any idea why? Also it still does not export geometry. – MrXsquared Oct 7 at 20:32
3
+50

I think you can do that just with the great internal tool from QGIS3 and without any knowledge of python. All within the graphical interface. You can access to the importing format (extension) by clicking in the '...' in the output element. Here you have the complete video with the process (I can't even upload here the complete gif...).

Here are some screenshot with those tool: enter image description here enter image description here


EDIT: Script changes

I've changed your code a little bit to keep your vectorlayer in memory after refactor tool and just then save it using your own CSV options. As you see, the output parameter is now a string instead of a feature sink. I hope it does the job.

from qgis.core import QgsProcessing
from qgis.core import QgsProcessingAlgorithm
from qgis.core import QgsProcessingMultiStepFeedback
from qgis.core import QgsProcessingParameterVectorLayer
from qgis.core import QgsProcessingParameterFeatureSink
from qgis.core import QgsVectorFileWriter
from qgis.core import QgsVectorLayer
from qgis.core import QgsCoordinateReferenceSystem
from qgis.core import QgsProject
from qgis.core import QgsProcessingParameterString
import processing


class Model(QgsProcessingAlgorithm):

    def initAlgorithm(self, config=None):
        self.addParameter(QgsProcessingParameterVectorLayer('Input', 'input', types=[QgsProcessing.TypeFile], defaultValue=None))
        self.addParameter(QgsProcessingParameterString('Finaloutput','finaloutput', 'C:/tmp/xy.csv'))

    def processAlgorithm(self, parameters, context, model_feedback):
        # Use a multi-step feedback, so that individual child algorithm progress reports are adjusted for the
        # overall progress through the model
        feedback = QgsProcessingMultiStepFeedback(1, model_feedback)
        results = {}
        outputs = {}

        # result
        alg_params = {
            'FIELDS_MAPPING': [{'expression': '$id', 'length': 10, 'name': 'fid', 'precision': 0, 'type': 4},{'expression': '$id-1', 'length': 10, 'name': 'id', 'precision': 0, 'type': 4}],
            'INPUT': parameters['Input'],
            'OUTPUT': 'memory'
        }
        results['Result'] = processing.run('qgis:refactorfields', alg_params, context=context, feedback=feedback, is_child_algorithm=True)
        outputs['Output'] = results['Result']['OUTPUT']
        layer = QgsVectorLayer(outputs['Output'], 'name', 'ogr')
        crs = QgsCoordinateReferenceSystem("EPSG:4326")
        QgsVectorFileWriter.writeAsVectorFormat(layer, parameters['Finaloutput'], "utf-8", crs, "CSV", False, [""], layerOptions=["GEOMETRY=AS_YX","SEPARATOR=SEMICOLON"])  
        return outputs

    def name(self):
        return 'model'

    def displayName(self):
        return 'model'

    def group(self):
        return ''

    def groupId(self):
        return ''

    def createInstance(self):
        return Model() 
  • I see the workaround to export geometry informations. But how can I specify separators (like semicolon, tab, comma, etc)? – MrXsquared Oct 11 at 17:29
  • works like a charm. can you add an explanation to your changes so we (I) can understand how and why this works? – MrXsquared Oct 12 at 21:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.