Looking for manual on how to properly setup standalone PyQGIS without GUI

Question

My goal is to create a standalone PyQGIS script that is able to run QgsProcessingAlgorithm scripts without GUI. My setup runs on Windows and my IDE of choice is VS Code.

I've followed the Windows instructions from the official documentation and added those environment variables to my system.

I'd like to be able to simply run the following two lines from VS Code:

from qgis.core import *
import processing

without getting into problems. This looks easy, right? Wrong. First thing you run into when running this code is that it is missing PyQt.

No module named 'PyQt5.QtCore'

There is no mention of this in the documentation, so naturally you might be inclined to think: okay, let's install that using pip.

Now the first line from qgis.core import * runs fine, but the second doesn't.

No module named 'processing'

To let this work, apparently a lot more environment paths (again undocumented) need to set. So with the help of @Kadir's answer I created the following batch file:

SET OSGEO4W_ROOT=C:\OSGeo4W
call %OSGEO4W_ROOT%\bin\o4w_env.bat

path %PATH%;%OSGEO4W_ROOT%\apps\qgis\bin
path %PATH%;%OSGEO4W_ROOT%\apps\Qt5\bin
path %PATH%;%OSGEO4W_ROOT%\apps\Python39\Scripts
set QGIS_PREFIX=%OSGEO4W_ROOT%\apps\qgis
set PYTHONHOME=%OSGEO4W_ROOT%\apps\Python39
set PYTHONPATH=%OSGEO4W_ROOT%\apps\qgis\python;%OSGEO4W_ROOT%\apps\qgis\python\plugins;%PYTHONPATH%
set QT_QPA_PLATFORM_PLUGIN_PATH=%OSGEO4W_ROOT%\apps\Qt5\plugins\platforms
set QT_PLUGIN_PATH=%OSGEO4W_ROOT%\apps\qgis\qtplugins;%OSGEO4W_ROOT%\apps\qt5\plugins

set GDAL_FILENAME_IS_UTF8=YES
set VSI_CACHE=TRUE
set VSI_CACHE_SIZE=1000000

Next I restarted VS Code and executed the initial two lines again. But immediately got into problems. This time PyQT complains:

qgis._gui cannot import type '����' from PyQt5.QtCore

After looking at various posts here and reported issues on GitHub, it turns out that that error represents a mismatch between the just installed PyQt5, and the one that is apparently shipped with QGIS itself.

Long story short: don't use pip to install PyQt5. So I uninstalled it and retried. Unfortunately I once again got the error:

No module named 'PyQt5.QtCore'

even after running that batch file. So I found another batch file, but that one didn't help either. I ended up with the same No module named 'PyQt.QtCore' error and after fiddling with combining the two batch files I even managed to get this error:

No module named 'osgeo'

somewhere deep in the import processing line. And that's when I concluded that solving this problem is way beyond my capabilities.

Would someone be able to write a step-by-step instruction on how to let those two lines of code run just fine in VS Code, from a fresh installation of QGIS?

Answer 1

Thanks to @bugmenot123 I found an outdated PowerShell script and VS code settings file but after a bit of fiddling, I was able to get it working!

And to ensure future versions of me, and other people who would love to not go through all these loops and hoops and instantly test their scripts in PyQGIS without GUI, I've created a GitHub repository with a simple example.

https://github.com/MarByteBeep/pyqgis-standalone

It doesn't rely on a batch file to launch VS Code and I modified the settings in such a way that it would be relatively easy to update it in the future.

TL; DR;

Should the repo ever get offline, here is a step-by-step version of the answer:

1. Install the full version of OSGeo4W LTR and put it in C:/OSGeo4W

2. Create a .env file with the following contents and put it in the root of your VS Code project folder:

# Configure these
OSGEO4W_ROOT=C:/OSGeo4W
QGIS_CONFIG=qgis-ltr
PYTHON_VERSION=Python39

# Variables below will be generated from config above
# Path
PATH=${OSGEO4W_ROOT}/apps/${QGIS_CONFIG}/bin;${OSGEO4W_ROOT}/apps/${PYTHON_VERSION};${OSGEO4W_ROOT}/apps/${PYTHON_VERSION}/Scripts;${OSGEO4W_ROOT}/apps/qt5/bin;${OSGEO4W_ROOT}/apps/Python27/Scripts;${OSGEO4W_ROOT}/bin;C:/WINDOWS/system32;C:/WINDOWS;C:/WINDOWS/system32/WBem;${OSGEO4W_ROOT}/apps/${PYTHON_VERSION}/lib/site-packages/pywin32_system32;${OSGEO4W_ROOT}/apps/${PYTHON_VERSION}/lib/site-packages/numpy/.libs
# Python
PYTHONHOME=${OSGEO4W_ROOT}/apps/${PYTHON_VERSION}
PYTHONPATH=${OSGEO4W_ROOT}/apps/${QGIS_CONFIG}/python;${OSGEO4W_ROOT}/apps/${QGIS_CONFIG}/python/plugins;${PYTHONPATH}
# GDAL
GDAL_DATA=${OSGEO4W_ROOT}/share/gdal
GDAL_DRIVER_PATH=${OSGEO4W_ROOT}/bin/gdalplugins
GDAL_FILENAME_IS_UTF8=YES
# GeoTIFF
GEOTIFF_CSV=${OSGEO4W_ROOT}/share/epsg_csv
# Qt
O4W_QT_BINARIES=${OSGEO4W_ROOT}/apps/Qt5/bin
O4W_QT_DOC=${OSGEO4W_ROOT}/apps/Qt5/doc
O4W_QT_HEADERS=${OSGEO4W_ROOT}/apps/Qt5/include
O4W_QT_LIBRARIES=${OSGEO4W_ROOT}/apps/Qt5/lib
O4W_QT_PLUGINS=${OSGEO4W_ROOT}/apps/Qt5/plugins
O4W_QT_PREFIX=${OSGEO4W_ROOT}/apps/Qt5
O4W_QT_TRANSLATIONS=${OSGEO4W_ROOT}/apps/Qt5/translations
QT_PLUGIN_PATH=${OSGEO4W_ROOT}/apps/${QGIS_CONFIG}/qtplugins;${OSGEO4W_ROOT}/apps/qt5/plugins
# QGIS
QGIS_PREFIX_PATH=${OSGEO4W_ROOT}/apps/${QGIS_CONFIG}
# Cache
VSI_CACHE=TRUE
VSI_CACHE_SIZE=1000000"

3. Create a .vscode/settings.json with the following content:

{
    "python.envFile": "${workspaceFolder}/.env",
    "python.defaultInterpreterPath": "C:/OSGeo4W/apps/Python39/python.exe"
}

4. Create main.py like this and feel free to replace Proximity.py with whatever script you are testing.

from qgis.core import *

qgs = QgsApplication([], False)
qgs.initQgis()

# import processing *after* initializing the application
import processing
from processing.core.Processing import Processing
Processing.initialize()

from scripts.proximity import Proximity

# I load the VectorLayer in code and pass it as an argument; obviously you could also just pass the
# file name to 'INPUT'. This is just a demonstration that you can have full control over the input
# layer before you send it off to the processing script

inLayer = QgsVectorLayer('test/nl_airports.osm|layername=multipolygons')
if not inLayer.isValid():
    raise Exception("Layer failed to load!")

# Create the Proximity algorithm
alg = Proximity()

# Set the params needed for this algorithm
params = {
    'INPUT': inLayer,
    'DISTANCE': 10000, # in meters
    'OUTPUT': 'test/rasterized.tif'
}

# Run the algorithm as you would from inside the QGIS GUI
alg.initAlgorithm()
ctx = QgsProcessingContext()
feedback = QgsProcessingFeedback()
alg.prepareAlgorithm(params, ctx, feedback)
alg.processAlgorithm(params, ctx, feedback)

# All done
print("Done")

qgs.exitQgis()

5. And now launch this main.py from VS Code and off you go.

Enjoy!

Answer 2

Finally, I got a solution to be able to run the "standalone PyQGIS"* example "Proximity" (above mentioned by "Mar Tijn" and provided by "MarByteBeep).

This solution was possible without needing to launch the configuration file "launch.json" as described in the "README.md" file (under the heading: "run example").
I just first added the following two code-lines (see here line 2 and 3) in the python file "main.py" so as to be able to use the plugin "PROCESSING" (initially line 8 of the "main.py" file), then I store it and finally I directly ran it.

Line 1: from qgis.core import

Line 2: import sys
Line 3: sys.path.append('C:\Program Files\QGIS 3.24.1\apps\qgis\python\plugins')

Line 4: qgs = QgsApplication([], False)
Line 5: ...

. * By "Standalone PyQGIS" I refer to code/scripts that can be run outside the QGIS-GUI (=> QGIS-Desktop/Server Application). In my case under the external Editor VS Code