Exporting result of QGIS model to CSV

Question

I am trying to output the result of a QGIS 3 model as a CSV file. At first I came across How to export csv/excel in QGIS3.0 model processor? but the accepted answer is not satisfying to me. Further researches only provided the code for QgsVectorFileWriter and how to output a CSV in general, but no information about the vectorFileName to use in QGIS models.

Unfortunately there is no predefined algorithm similar to "package layers" for CSV files.

So I was trying to modify the script itself using QgsVectorFileWriter like this:

QgsVectorFileWriter.writeAsVectorFormat(results, r"C:\tmp\xy.csv", "utf-8", None, "CSV", layerOptions ='GEOMETRY=AS_WKT')

Unfortunately I dont know much about Python yet, so I am struggling to implement this correctly. I think the main issue is which vectorFileName (e.g. results or finaloutput) I should use and where to place this output code correctly.

---

I now managed to create a partially working script (simplified model which adds fid and id field to points as example):

from qgis.core import QgsProcessing
from qgis.core import QgsProcessingAlgorithm
from qgis.core import QgsProcessingMultiStepFeedback
from qgis.core import QgsProcessingParameterVectorLayer
from qgis.core import QgsProcessingParameterFeatureSink
from qgis.core import QgsVectorFileWriter
from qgis.core import QgsVectorLayer
from qgis.core import QgsCoordinateReferenceSystem
import processing


class Model(QgsProcessingAlgorithm):

    def initAlgorithm(self, config=None):
        self.addParameter(QgsProcessingParameterVectorLayer('input', 'input', types=[QgsProcessing.TypeFile], defaultValue=None))
        #self.addParameter(QgsProcessingParameterFeatureSink('Finaloutput', 'finaloutput', type=QgsProcessing.TypeFile, createByDefault=True, defaultValue=None)) #Using this, no CSV will be created
        self.addParameter(QgsProcessingParameterFeatureSink('Finaloutput', 'finaloutput', type=QgsProcessing.TypeFile, createByDefault=True, defaultValue='C:/tmp/xy.csv')) #Using this, two CSV will be created
        #self.addParameter(QgsProcessingParameterFeatureSink('Finaloutput', 'finaloutput', type=QgsProcessing.TypeVectorAnyGeometry, createByDefault=True, defaultValue=None)) #Using this no CSV will be created

    def processAlgorithm(self, parameters, context, model_feedback):
        # Use a multi-step feedback, so that individual child algorithm progress reports are adjusted for the
        # overall progress through the model
        feedback = QgsProcessingMultiStepFeedback(1, model_feedback)
        results = {}
        outputs = {}

        # result
        alg_params = {
            'FIELDS_MAPPING': [{'expression': '$id', 'length': 10, 'name': 'fid', 'precision': 0, 'type': 4},{'expression': '$id-1', 'length': 10, 'name': 'id', 'precision': 0, 'type': 4}],
            'INPUT': parameters['input'],
            'OUTPUT': parameters['Finaloutput']
        }
        outputs['Result'] = processing.run('qgis:refactorfields', alg_params, context=context, feedback=feedback, is_child_algorithm=True)
        results['Finaloutput'] = outputs['Result']['OUTPUT']
        layer = QgsVectorLayer(results['Finaloutput'], "name", "ogr")
        crs = QgsCoordinateReferenceSystem("EPSG:4326")
        QgsVectorFileWriter.writeAsVectorFormat(layer, r'c:\tmp\ab.csv', "utf-8", crs, "CSV", False, [""], layerOptions=["GEOMETRY=AS_YX","SEPARATOR=SEMICOLON"])
        return results

    def name(self):
        return 'model'

    def displayName(self):
        return 'model'

    def group(self):
        return ''

    def groupId(self):
        return ''

    def createInstance(self):
        return Model()

So first it turned out I need to create a QgsVectorLayer such as layer = QgsVectorLayer(results['Finaloutput'], "name", "ogr") first.

Secondly, the API docs as well as the given error messages are really confusing. The main issue was that arguments for QgsVectorFileWriter.writeAsVectorFormat were given the wrong way. First it needs a CRS given as 4th argument instead of None and secondly the 7th argument needs to be a list and not a string. Also the layerOptions needs to be a list and not a string. Using @Fran Raga's hint I figured out this real error (First I thought I need to output the CSV after return because the error then disappeared...).

At last, even more confusing: Using @Joseph's hint it now works partially. But only when setting a defaultvalue like

self.addParameter(QgsProcessingParameterFeatureSink('Finaloutput', 'finaloutput', type=QgsProcessing.TypeFile, createByDefault=True, defaultValue='C:/tmp/xy.csv'))

It then outputs two CSV: "xy.csv" (comma as separator) as well as "ab.csv" (semicolon as separator). When leaving

self.addParameter(QgsProcessingParameterFeatureSink('Finaloutput', 'finaloutput', type=QgsProcessing.TypeFile, createByDefault=True, defaultValue=None)) 

no CSV at all is written. Overall, no WKT or YX geometry is written.

---

How can I properly export the result of a QGIS 3 model to a CSV using pyqgis with predefined settings such as path, delimiter and geometrytype?

Answer 1

I think you can do that just with the great internal tool from QGIS3 and without any knowledge of python. All within the graphical interface. You can access to the importing format (extension) by clicking in the '...' in the output element. Here you have the complete video with the process (I can't even upload here the complete gif...).

Here are some screenshot with those tool:

---

EDIT: Script changes

I've changed your code a little bit to keep your vectorlayer in memory after refactor tool and just then save it using your own CSV options. As you see, the output parameter is now a string instead of a feature sink. I hope it does the job.

from qgis.core import QgsProcessing
from qgis.core import QgsProcessingAlgorithm
from qgis.core import QgsProcessingMultiStepFeedback
from qgis.core import QgsProcessingParameterVectorLayer
from qgis.core import QgsProcessingParameterFeatureSink
from qgis.core import QgsVectorFileWriter
from qgis.core import QgsVectorLayer
from qgis.core import QgsCoordinateReferenceSystem
from qgis.core import QgsProject
from qgis.core import QgsProcessingParameterString
import processing


class Model(QgsProcessingAlgorithm):

    def initAlgorithm(self, config=None):
        self.addParameter(QgsProcessingParameterVectorLayer('Input', 'input', types=[QgsProcessing.TypeFile], defaultValue=None))
        self.addParameter(QgsProcessingParameterString('Finaloutput','finaloutput', 'C:/tmp/xy.csv'))

    def processAlgorithm(self, parameters, context, model_feedback):
        # Use a multi-step feedback, so that individual child algorithm progress reports are adjusted for the
        # overall progress through the model
        feedback = QgsProcessingMultiStepFeedback(1, model_feedback)
        results = {}
        outputs = {}

        # result
        alg_params = {
            'FIELDS_MAPPING': [{'expression': '$id', 'length': 10, 'name': 'fid', 'precision': 0, 'type': 4},{'expression': '$id-1', 'length': 10, 'name': 'id', 'precision': 0, 'type': 4}],
            'INPUT': parameters['Input'],
            'OUTPUT': 'memory'
        }
        results['Result'] = processing.run('qgis:refactorfields', alg_params, context=context, feedback=feedback, is_child_algorithm=True)
        outputs['Output'] = results['Result']['OUTPUT']
        layer = QgsVectorLayer(outputs['Output'], 'name', 'ogr')
        crs = QgsCoordinateReferenceSystem("EPSG:4326")
        QgsVectorFileWriter.writeAsVectorFormat(layer, parameters['Finaloutput'], "utf-8", crs, "CSV", False, [""], layerOptions=["GEOMETRY=AS_YX","SEPARATOR=SEMICOLON"])  
        return outputs

    def name(self):
        return 'model'

    def displayName(self):
        return 'model'

    def group(self):
        return ''

    def groupId(self):
        return ''

    def createInstance(self):
        return Model()