Executing QGIS tools in Python from Linux Shell [duplicate]

Question

I would like to execute a Python script which uses QGIS Processing tools.

On Ubuntu 20.x I installed qgis, qgis-common, python3-qgis and python3-qgis-common.

My objective is to execute the tool Exploe HStore Field from shell within a python script.

The following lines are working well within the QGIS-GUI Python Console

from qgis.core import processing
from qgis.core import *
processing.run("qgis:explodehstorefield", { 
    'INPUT' : '/home/myacc/geodata.gpkg|layername=points',
    'FIELD' : 'other_tags', 
    'EXPECTED_FIELDS' : '', 
    'OUTPUT' : 'ogr:dbname=\'/home/myacc/geodata_explode.gpkg\' table=\"points\" (geom)' })

My problem is, that processing.run is not recognized outsite the QGIS-GUI Python console.

I tried to create a pre-build script from the modeller, but there is the same problem.

Answer 1

You should set Python path to be able to import QGIS specific modules. Compare the Python path in the terminal window and from the QGIS Python console.

Python path in the terminal window:

python3
>>> import sys
>>> print(sys.path)
['', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/siki/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages']

Python path from the QGIS Python console

>>> import sys
>>> print(sys.path)
['/usr/share/qgis/python', '/home/siki/.local/share/QGIS/QGIS3/profiles/default/python', '/home/siki/.local/share/QGIS/QGIS3/profiles/default/python/plugins', '/usr/share/qgis/python/plugins', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/siki/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages', '/home/siki/.local/share/QGIS/QGIS3/profiles/default/python', '/home/siki/pyqgis']

Add the necessary missing QGIS specific directories to the Python path in your code, for example:

sys.path.insert(0, '/usr/share/qgis/python')

You should import QgsProcessing from qgis.core:

from qgis.core import QgsProcessing

Answer 2

I posted this answer because it was a little different in my Linux Shell with Debian 10. Based in this question and in this answer, I adapted following code for a 'qgis:creategrid' processing algorithm and saved it as test.py.

def warn(*args, **kwargs):
    pass

import warnings
warnings.warn = warn

import sys

from qgis.analysis import QgsNativeAlgorithms
from qgis.core import (
     QgsApplication, QgsRasterLayer
    )

QgsApplication.setPrefixPath('/usr', True) #for avoiding "Application path not initialized"

# Append the path where processing plugin can be found
sys.path.append('/usr/share/qgis/python/plugins')

from processing.core.Processing import Processing

Processing.initialize()
QgsApplication.processingRegistry().addProvider(QgsNativeAlgorithms())
import processing

uri = '/home/zeito/pyqgis_data/utah_demUTM2.tif'

layer = QgsRasterLayer(uri,
                       'raster')

extent = layer.extent()

xmin, ymin, xmax, ymax = extent.toRectF().getCoords()

extent = str(xmin)+ ',' + str(xmax)+ ',' +str(ymin)+ ',' +str(ymax) 

output = '/home/zeito/pyqgis_data/my_grid.shp'

processing.run('qgis:creategrid', 
               {'TYPE':2, 
                'EXTENT':extent, 
                'HSPACING':5000,
                'VSPACING':5000,
                'HOVERLAY':0,
                'VOVERLAY':0,
                'CRS':layer.crs().authid(), 
                'OUTPUT':output})

print("Done!")

After running it in Linus shell as:

python3 test.py
Done!

produced layer was loaded as expected in QGIS 3. It worked for me in this way.