Starting QGIS 2.18 processing algorithms from .bat-file

Question

I have written a bunch of Python scripts within the QGIS 2.18 processing framework to process a sequence of tasks. What I am trying to achieve now is to call the python scripts from a .bat-file without opening QGIS 2.18. The .bat-file is executed within a Windows 10 cmd window.

I have already tried multiple things with an easy Python script which just prints "Hello QGIS!". But I cannot get it to work correctly. It does not print "Hello QGIS!" into my cmd console window or into the file.

I have first tried it with a standalone installation of QGIS2.18 but as this wasn't working, I followed the advice from @wondim and installed QGIS2.18 over the OSGeo4W installer. I downloaded the OSGeo4W (64-Bit version) installer from this site https://trac.osgeo.org/osgeo4w/. I double-clicked the downloaded setup file (called osgeo4w-setup-x86_64.exe), checked "advanced installation"->Next. On the following pages, I left the default options checked and just clicked "Next" until I got to the page where I need to select the packages I want to install. I expanded the "Desktop" drop-down menu and I see that all options are on "Skip". Now I am checking the 2.18.0-1 version (qgis-full-rel-dev) and click next. As of my understanding, QGIS2.18 should now get installed via OSGeo4W installer. But if I do this, it installs QGIS3 and Python3 because there are certain dependencies. After the installation, I have QGIS3 installed and in the OSGeo4W folder is the Python37 folder (which is not the correct python folder, I need Python27).

So, with help of the OSGeo4W installer, I installed some libs which I believe where needed to generate the Python27 folder. The libs I installed are called python-six, python-future and pyqt4.

I have created a .bat-file named launch.bat which includes the following lines (updated following @wondim's answer):

REM Change OSGEO4W_ROOT to point to the base install folder
SET OSGEO4W_ROOT=C:\OSGeo4W
SET QGISNAME=qgis-rel-dev
SET QGIS=%OSGEO4W_ROOT%\apps\%QGISNAME%
set QGIS_PREFIX_PATH=%QGIS%
REM Gdal Setup
set GDAL_DATA=%OSGEO4W_ROOT%\share\gdal\
REM Python Setup
set PATH=%OSGEO4W_ROOT%\bin;%QGIS%\bin;%PATH%
SET PYTHONHOME=%OSGEO4W_ROOT%\apps\Python27
set PYTHONPATH=%QGIS%\python;%PYTHONPATH%

REM Launch python job
python "C:/Users/myuser/Desktop/hello_qgis.py"
pause

And my hello world python script looks like this (updated following @wondim's answer):

import sys

# Add the path to Processing framework
sys.path.append('C:/OSGeo4W/apps/qgis-rel-dev/python/plugins')
filename = "C:/Users/myuser/Desktop/logging/log.txt"

def write_log_message(message, tag, level):
    with open(filename, 'a') as logfile:
        logfile.write('{tag}({level}): {message}'.format(tag=tag, level=level, message=message))

from qgis.core import *  # This is where the error appears
        
QgsMessageLog.instance().messageReceived.connect(write_log_message)

# Initialize QGIS Application
QgsApplication.setPrefixPath("C:/OSGeo4W/apps/qgis-rel-dev", True)

# This line causes QGIS to crash without any error message
# app = QgsApplication([], True) 

QgsApplication.initQgis()


# Import and initialize Processing framework
from processing.core.Processing import Processing
Processing.initialize()
import processing

print 'Hello QGIS!'
write_log_message("Hello QGIS\n", 'hey', 1)

When I am executing the launch.bat, I get the following output in my command line window:

C:\Users\myuser\Desktop>REM Change OSGEO4W_ROOT to point to the base install folder    
C:\Users\myuser\Desktop>SET OSGEO4W_ROOT=C:\OSGeo4W    
C:\Users\myuser\Desktop>SET QGISNAME=qgis-rel-dev    
C:\Users\myuser\Desktop>SET QGIS=C:\OSGeo4W\apps\qgis-rel-dev    
C:\Users\myuser\Desktop>set QGIS_PREFIX_PATH=C:\OSGeo4W\apps\qgis-rel-dev    
C:\Users\myuser\Desktop>REM Gdal Setup    
C:\Users\myuser\Desktop>set GDAL_DATA=C:\OSGeo4W\share\gdal\    
C:\Users\myuser\Desktop>REM Python Setup    
C:\Users\myuser\Desktop>set PATH=C:\OSGeo4W\bin;C:\OSGeo4W\apps\qgis-rel-dev\bin;C:\Users\myuser\AppData\Local\Microsoft\WindowsApps;C:\Users\myuser\AppData\Local\Programs\MiKTeX 2.9\miktex\bin\x64\;C:\Progra~1\Java\jdk1.8.0;C:\BigData\hadoop-2.9.1;C:\BigData\hadoop-2.9.1\bin;C:\BigData\hadoop-2.9.1\sbin;    
C:\Users\myuser\Desktop>SET PYTHONHOME=C:\OSGeo4W\apps\Python27    
C:\Users\myuser\Desktop>set PYTHONPATH=C:\OSGeo4W\apps\qgis-rel-dev\python;    
C:\Users\myuser\Desktop>REM Launch python job

C:\Users\myuser\Desktop>python "C:/Users/myuser/Desktop/hello_qgis.py"
Traceback (most recent call last):
  File "C:/Users/myuser/Desktop/hello_qgis.py", line 11, in <module>
    from qgis.core import *
  File "C:\OSGeo4W\apps\qgis-rel-dev\python\qgis\__init__.py", line 70, in <module>
    from qgis.PyQt import QtCore
  File "C:\OSGeo4W\apps\qgis-rel-dev\python\qgis\PyQt\QtCore.py", line 24, in <module>
    from PyQt5.QtCore import *
ImportError: No module named PyQt5.QtCore

C:\Users\myuser\Desktop>pause
Drücken Sie eine beliebige Taste . . .

Any suggestions on how to solve this problem? It seems like it is trying to import PyQt5 things, but I thought QGIS 2 is using PyQt4 or am I wrong here? Have I forgotten to install some more Python27 related libs?

I am very grateful for any advice you could possibly give me as I have already invested a lot of time into this and I believe the solution cannot be that difficult.

Please consider that I have already browsed lots of other similar questions which did not helped me in finding the solution. I would also like to solve this by using a standalone installation of QGIS 2.18 if this is possible.

UPDATE: I wasn't successful installing QGIS2.18 via the OSGeo4W installer following this tutorial: https://www.giscourse.com/install-qgis-through-osgeo4w/ It always installs QGIS3 and Python37 by default. I have to install some libs in addition for it to generate the Python27 folder. But that does not change the fact that it does not install QGIS2.18.

UPDATE: Under "Desktop": I marked the 2.18.0 version as this is the last released version von QGIS2.18. Furthermore, under "Libs", I have installed the python-six: Python 2 and 3 compatibility utilities and pyqt4:Python binding for Qt4 packages. Now I have the Python27 package in my OSGeo4W/apps/ folder (and the Python37 folder too).

Answer 1

I think the problem here is the order of where you wrote sys.path.append('C:/OSGeo4W/apps/qgis/python/plugins'). This code should fix the latest issue.

import sys

sys.path.append('C:/OSGeo4W/apps/qgis/python/plugins')

filename = "C:\Users\myuser\Desktop\logging\log.txt"

def write_log_message(message, tag, level):
    with open(filename, 'a') as logfile:
        logfile.write('{tag}({level}): {message}'.format(tag=tag, level=level, message=message))

from qgis.core import *
QgsMessageLog.instance().messageReceived.connect(write_log_message)

# Initialize QGIS Application
QgsApplication.setPrefixPath("C:/OSGeo4W/apps/qgis", True)

# Line 8 causes QGIS to crash without any error message
# app = QgsApplication([], True) 

QgsApplication.initQgis()

# Add the path to Processing framework

# Import and initialize Processing framework
from processing.core.Processing import Processing
Processing.initialize()
import processing

print 'Hello QGIS!'
write_log_message("Hello QGIS\n", 'hey', 1)

Edit: QGIS 2 uses PyQt4 so you have to remove pyqgis 3 from OSGEO4W.