Import error for qgis.core when running OSGeo4w shell script

Question

I've been trying, along with this post, to run a script in OSGeo4w Shell, outside of QGIS. But I get the following error:

ImportError: No module named qgis.core

I have also read the following posts and tried to import various modules but to no avail:

• How to run sextante algorithms outside of QGIS python console?
• How to run a simple python script for QGIS from outside (e.g. Sublime Text)?
• Writing standalone Python scripts using PyQGIS?

Here is a simple script which creates a grid and clips a polygon shapefile onto it.

Note: This script has been tested and works successfully when running in QGIS.

##Test=name

import os
import glob
import sys

sys.path.append("C:\Program Files\QGIS Brighton\lib;%OSGEO4W_ROOT:\=/%/apps/qgis;%OSGEO4W_ROOT%\apps\qgis\bin;%OSGEO4W_ROOT%\apps\grass\grass-6.4.3\lib;%PATH%")

from PyQt4.QtCore import *
from PyQt4.QtGui import *
from qgis.core import *

QgsApplication.setPrefixPath("C:\Program Files\QGIS Brighton\apps\qgis", True)
QgsApplication.initQgis()

from os.path import expanduser
home = expanduser("~")

#   Folder path of the Results for shapefiles
path_dir = home + "\Desktop\Test\\"
path_res = path_dir + "Results\\"

def run():
#   Set directory, search for all polygon .shp files and run the Create Grid and Clip algorithms then output results into Results folder
    os.chdir(path_dir + "Shapefiles\\")
    for fname in glob.glob("*.shp"):
            outputs_1=processing.runalg("qgis:creategrid", 1000, 1000, 24108, 18351.157175, 258293.802316, 665638.226408, 1, 'EPSG:7405',  None)
            outputs_2=processing.runalg("qgis:clip", outputs_1['SAVENAME'], fname, path_res  + "/"+ fname)
run()

QgsApplication.exitQgis()
#   Remove the above line when running in QGIS

---

Following the answer and the script posted by @gcarrillo, I can finally import the qgis.core. modules successfully. The script provided by @gcarrillo runs but I receive a Traceback error:

Traceback (most recent call last):
  File "Test.py", line 55, in <module>
    run()
  File "Test.py", line 53, in run
    algClip.processAlgorithm(progress)
  File "C:\Users\username\.qgis2\python\plugins\processing\algs\qgis\ftools\Clip.py", line 59, in processAlgorithm
    layerA.pendingFields(),
AttributeError: 'NoneType' object has no attribute 'pendingFields'

Answer 1

Finally found the proper way of running processing algorithms in PyQGIS standalone scripts.

This answer is based on answers to Problem with import qgis.core when writing a stand-alone PyQGIS script and to Error: Algorithm not found, which is in turn based on a Qgis-dev mailing-list discussion.

I suggest you to follow the work flow given in Problem with import qgis.core when writing a stand-alone PyQGIS script to enable your QGIS libraries in your OSGeo4W Shell. Once you have your QGIS libraries working properly, we can proceed to the 2nd part of your question: running processing algorithms in a standalone PyQGIS script.

I've modified your original script a bit and tested it on Windows 7 and GNU/Linux. I use processing version 2.2.0-2 and suggest you to use this version, which is the current one at the moment of writing the answer.

import os, sys, glob

# Prepare the environment
from qgis.core import * # qgis.core must be imported before PyQt4.QtGui!!!
from PyQt4.QtGui import *
app = QApplication([])
QgsApplication.setPrefixPath("C:\\Program Files\\QGIS Brighton\\apps\\qgis", True) # The True value is important
QgsApplication.initQgis()

from os.path import expanduser
home = expanduser("~")

#   Folder path of the Results for shapefiles
path_dir = home + "\Desktop\Test\\"
path_res = path_dir + "Results\\"

# Prepare processing framework 
sys.path.append( home + '\.qgis2\python\plugins' )
from processing.core.Processing import Processing
Processing.initialize()
from processing.tools import *

def run():
    outputs_1=general.runalg("qgis:creategrid", 1000, 1000, 24108, 18351.157175, 258293.802316, 665638.226408, 1, 'EPSG:7405',  None)
    #   Set directory, search for all polygon .shp files and run the Create Grid and Clip algorithms then output results into Results folder
    os.chdir(path_dir + "Shapefiles\\")
    for fname in glob.glob("*.shp"):        
        outputs_2=general.runalg("qgis:clip", outputs_1['SAVENAME'], fname, path_res  + "/"+ fname)

run()

QgsApplication.exitQgis()
#   Remove the above line when running in QGIS

Note that I have taken the grid creation out of the for loop, as you don't really need a new grid to perform each clip.

This should do the trick!

Answer 2

This answer is based on answers to Problem with import qgis.core when writing a stand-alone PyQGIS script and to How can I access `processing` with Python?.

I suggest you to follow the work flow given in Problem with import qgis.core when writing a stand-alone PyQGIS script to enable your QGIS libraries in your OSGeo4W Shell. Once you have your QGIS libraries working properly, we can proceed to the 2nd part of your question: running processing algorithms in a standalone PyQGIS script.

As in How can I access `processing` with Python?, I'll give you a workaround until I'm able to run algorithms by name (e.g., processing.runalg('provider:algorithm_name')). I use processing version 2.2.0-2 and suggest you to use this version.

We can use the QGIS Python console to figure out where an algorithm script is located in processing plugin folders. For example, to know where to import qgis:creategrid from, write in QGIS Python console:

from processing.core.Processing import Processing
Processing.getAlgorithm('qgis:creategrid')

You should obtain:

<processing.algs.qgis.mmqgisx.MMQGISXAlgorithms.mmqgisx_grid_algorithm instance at 0xae7382c>

which is enough for us to notice both the module path (processing.algs.qgis.mmqgisx.MMQGISXAlgorithms) and the algorithm class (mmqgisx_grid_algorithm). You'll use this information in the script below.

I've modified your script a bit and tested it on Windows 7. You might need to adjust paths in order to run the script in your own environment.

import os
import glob
import sys

from PyQt4.QtCore import *
from PyQt4.QtGui import *
from qgis.core import *

app = QApplication([])
QgsApplication.setPrefixPath("C:\\Program Files\\QGIS Brighton\\apps\\qgis", True)
QgsApplication.initQgis()

from os.path import expanduser
home = expanduser("~")

#   Folder path of the Results for shapefiles
path_dir = home + "\Desktop\Test\\"
path_res = path_dir + "Results\\"

# Tell Python where you will get processing from
sys.path.append(home + "\.qgis2\python\plugins")

# Reference the algorithms you want to run
from processing.algs.qgis.mmqgisx.MMQGISXAlgorithms import *
from processing.algs.qgis.ftools.Clip import *
algGrid = mmqgisx_grid_algorithm()
algClip = Clip()

from processing.core.SilentProgress import SilentProgress
progress = SilentProgress()

def run():
    # Create a grid 
    grid = path_dir + "Grids\grid.shp"
    algGrid.setParameterValue('HSPACING', 1000)
    algGrid.setParameterValue('VSPACING', 1000)
    algGrid.setParameterValue('WIDTH', 24108)
    algGrid.setParameterValue('HEIGHT', 18351.157175)
    algGrid.setParameterValue('CENTERX', 258293.802316)
    algGrid.setParameterValue('CENTERY', 665638.226408)
    algGrid.setParameterValue('GRIDTYPE', 1)
    algGrid.setParameterValue('CRS', 'EPSG:7405')
    algGrid.setOutputValue('SAVENAME', grid)
    algGrid.processAlgorithm(progress)

    # Set directory, search for all polygon .shp files 
    os.chdir(path_dir + "Shapefiles\\")    
    for fname in glob.glob("*.shp"):
        # Run the Clip algorithm, then output results into Results folder
        algClip.setParameterValue('INPUT', grid)
        algClip.setParameterValue('OVERLAY', fname)
        algClip.setOutputValue('OUTPUT', path_res  + "/"+ fname)
        algClip.processAlgorithm(progress)

run()
QgsApplication.exitQgis()

This should do the trick!

As you can see, I've created a Test/Grids folder so that you store a single grid Shapefile instead of creating a temporal file in every for loop, which doesn't seem to be necessary.

Answer 3

I had to make minor changes to the script provided by @gcarrillo to include the OSGEO4W64 path (I had to re-install QGIS via the OSGEO4W64 installer as I used the standalone installer initially) and to include double-slashes. Here is the final script and many thanks to everyone for their help:

import os, sys, glob

# Prepare the environment
from qgis.core import * # qgis.core must be imported before PyQt4.QtGui!!!
from PyQt4.QtGui import *
app = QgsApplication([]) # instantiation of QgsApplication
QgsApplication.setPrefixPath("C:\\OSGeo4W64\\apps\\qgis", True) # The True value is important
QgsApplication.initQgis()

from os.path import expanduser
home = expanduser("~")

#   Folder path of the Results for shapefiles
path_dir = home + "\Desktop\Test\\"
path_res = path_dir + "Results\\"

# Prepare processing framework 
sys.path.append( home + '\.qgis2\python\plugins' )
from processing.core.Processing import Processing
Processing.initialize()
from processing.tools import *

def run():
    outputs_1=general.runalg("qgis:creategrid", 1000, 1000, 24108, 18351.157175, 258293.802316, 665638.226408, 1, 'EPSG:7405',  None)
    #   Set directory, search for all polygon .shp files and run the Create Grid and Clip algorithms then output results into Results folder
    os.chdir(path_dir + "Shapefiles\\")
    for fname in glob.glob("*.shp"):        
        outputs_2=general.runalg("qgis:clip", outputs_1['SAVENAME'], fname, path_res  + "/"+ fname)
run()

QgsApplication.exitQgis()
#   Remove the above line when running in QGIS