10

I want to write a few stand-alone scripts that use the processing toolbox of Qgis.

I have read a few threads (here and here, e.g.) but couldn't find a working solution yet.

Using Qgis 2.16.1 on Ubuntu Xenial 16.04 LTS

The import section of my script looks like this:

# Python modules
import sys
import time
import os

# Qgis modules
from qgis.core import *
import qgis.utils
from PyQt4.QtCore import QFileInfo, QSettings

Does anyone know what is missing for me to be able to import the processing module?

With a simple import processing, I get this:

Original exception was:
Traceback (most recent call last):
 File "/home/steph/Documents/Projets/20141227-CIM_Bishkek/Scripts/python/00-projets/20160811-AnalysesUAVs/20160811-UAVAnalyse.py", line 36, in <module>
import processing
 File "/usr/lib/python2.7/dist-packages/qgis/utils.py", line 607, in _import
mod = _builtin_import(name, globals, locals, fromlist, level)
ImportError: No module named processing

EDIT (after Joseph's comment)

I have tried like this:

# Python modules
import sys
import time
import os

# Qgis modules
from qgis.core import *
import qgis.utils
from PyQt4.QtGui import *
app = QApplication([])
QgsApplication.setPrefixPath("/usr", True)
QgsApplication.initQgis()
from PyQt4.QtCore import QFileInfo, QSettings
#from PyQt4.QtGui import *

# Prepare processing framework 
sys.path.append('/home/steph/.qgis2/python/plugins')
from processing.core.Processing import Processing
Processing.initialize()
from processing.tools import *

but the behavior is weird: my script runs until the end without error but seems to "jump" over the the tasks it's supposed to perform :-) In other words, it runs until the end but without doing anything.

I admit that my explanation is not very clear... I will investigate further but if anyone has a magical solution (not a workaround) to import this module, then please!

EDIT 2: adding my whole script. Sorry if it's a bit long....

# -*- coding: cp1252 -*-
########################################################
## Name: Performs various analyses on UAV imagery using Qgis
## Source Name: UAVanalyse.py
## Version: Python 2.7
## Author: Stephane Henriod
## Usage: Performs a set of analyses on UAV imagery
## Date 11.08.2016
## Modified: 
########################################################


# Import required modules

# Python modules
import sys
import time
import os

# Qgis modules
from qgis.core import *
import qgis.utils
from PyQt4.QtCore import QFileInfo, QSettings

# Custom modules
from config_work import *
import display_msg as disp
import clean_time as cl

def make_raster_layer(raster, log_file):
    """Creates a raster layer from the path to a raster, if the path exists and if the raster is valid

    Param_in:
        raster (string) -- The path to the raster to be transformed into a layer
        log_file (string) -- The path to the log file to write in

    Param_out:
        list: 
            [0] = full path to the raster 
            [1] = raster layer

    """

    if os.path.exists(raster):
        fileName = raster
        fileInfo = QFileInfo(fileName)
        baseName = fileInfo.baseName()
        rlayer = QgsRasterLayer(fileName, baseName)
        if rlayer.isValid():
            return [raster, rlayer]
    else:
        return False

def study_raster(rlayer, log_file):
    """Returns properties of a raster, if this one exists and is valid

    Param_in:
        rlayer (bin) -- A raster layer
        log_file (string) -- The path to the log file to write in

    """

    infos = {}

    if rlayer:
        infos['a - Width'] = rlayer.width()
        infos['b - Height'] = rlayer.height()
        infos['c - Extent'] = rlayer.extent().toString()
        infos['d - # bands'] = rlayer.bandCount()
        infos['e - X resolution'] = rlayer.rasterUnitsPerPixelX()
        infos['f - Y resolution'] = rlayer.rasterUnitsPerPixelY()
        return infos
    else:
        return False


def project_raster(raster, to_crs, log_file):
    """Projects a raster into another crs

    Param_in:
        raster (string) -- The path to the raster to be transformed into a layer
        to_crs (string) -- The coordinate reference system to which the layer must be projected
        log_file (string) -- The path to the log file to write in

    """

    img_out_name = os.path.splitext(os.path.basename(raster))[0] + '_proj' + os.path.splitext(os.path.basename(raster))[1]
    img_out = os.path.join(output_folder, img_out_name)
    #processing.runalg("gdalwarp -overwrite -s_srs EPSG:32642 -t_srs " + to_crs + " " + rlayer[0] + " " + img_out)

    msg = img_out    
    disp.display_msg(log_file, msg, 'a')

    return img_out_name

if __name__ == "__main__":
    t_start_script = time.localtime()
    t_start_script_clean = time.strftime("%Y%m%d-%H%M", t_start_script)

    # Checking the folders
    if not os.path.exists(input_folder_path):
        os.makedirs(input_folder_path)
    if not os.path.exists(temp_folder_path):
        os.makedirs(temp_folder_path)
    if not os.path.exists(output_folder_path):
        os.makedirs(output_folder_path)

    # Creating the output and temp folders
    output_folder = os.path.join(output_folder_path, t_start_script_clean + '-UAVanalyse')
    temp_folder = os.path.join(temp_folder_path, t_start_script_clean + '-UAVanalyse')

    if not os.path.exists(output_folder):
        os.makedirs(output_folder)
    if not os.path.exists(temp_folder):
        os.makedirs(temp_folder)

    # Creating the log file
    log_file_name = t_start_script_clean + '-UAVanalyse.log'
    log_file = os.path.join(output_folder, log_file_name)

    # Heading of the log file
    msg = "Performs a set of analyses on UAV imagery" + os.linesep
    msg += "Input folder: " + input_folder_path
    msg += "\n RGB image: " + img_rgb_name
    msg += "\n NIR image: " + img_nir_name
    msg += "\n RGBIR image: " + img_rgbir_name
    msg += "\n DSM file: " + img_dsm_name
    disp.display_msg(log_file, msg, 'w')

    #msg = "Script started on " + cl.clean_time(t_start_script)
    #disp.display_msg(log_file, msg, 'a')


    # Initialize Qgis (source: http://docs.qgis.org/testing/en/docs/pyqgis_developer_cookbook/intro.html)
    msg = 'Initialize Qgis'
    disp.display_msg(log_file, msg, 'a')
    # supply path to qgis install location
    QgsApplication.setPrefixPath("/usr", True)

    # create a reference to the QgsApplication, setting the
    # second argument to False disables the GUI
    qgs = QgsApplication([], False)

    # load providers
    qgs.initQgis()


    # Write your code here to load some layers, use processing algorithms, etc.

    # Make raster layers
    rlayer_rgb = make_raster_layer(img_rgb, log_file)
    rlayer_nir = make_raster_layer(img_nir, log_file)
    rlayer_rgbir = make_raster_layer(img_rgbir, log_file)
    rlayer_dsm = make_raster_layer(img_dsm, log_file)

    all_valid_layers = []
    if rlayer_rgb: all_valid_layers.append(rlayer_rgb)
    if rlayer_nir: all_valid_layers.append(rlayer_nir)
    if rlayer_rgbir: all_valid_layers.append(rlayer_rgbir)
    if rlayer_dsm: all_valid_layers.append(rlayer_dsm)




    # (I) Infos about the layers
    msg = os.linesep + frm_separator + os.linesep + '(I) Infos about the layers' + os.linesep + frm_separator + os.linesep
    disp.display_msg(log_file, msg, 'a')

    i = 1
    for layer in all_valid_layers:
        infos = study_raster(layer[1], log_file)
        msg = '\n (' + str(i) + ') ' + layer[0] + os.linesep
        for item in sorted(infos):
            msg += '\n ' + str(item) + ': ' + str(infos[item]) + os.linesep

        i+=1
        disp.display_msg(log_file, msg, 'a')

    msg = 'List of valid layers:' + os.linesep
    for layer in all_valid_layers:
        msg += layer[0]+ os.linesep
    disp.display_msg(log_file, msg, 'a')


    # (II) Projects the layers into the national coordinate system or any desired system
    msg = os.linesep + frm_separator + os.linesep + '(II) Projecting of the layers' + os.linesep + frm_separator + os.linesep
    disp.display_msg(log_file, msg, 'a')

    i = 1
    for layer in all_valid_layers:
        project_raster(layer[0], to_crs, log_file)




    # When script is complete, call exitQgis() to remove the provider and
    # layer registries from memory
    qgs.exitQgis()
    msg = 'Qgis has been closed'
    disp.display_msg(log_file, msg, 'a')

    #raw_input("Press Enter to continue...")
  • In your second link, did you use the code provided by @GermánCarrillo? His code is what I also use to run standalone scripts (I edit my paths slightly as I use Windows). – Joseph Aug 11 '16 at 8:58
  • What is the task you are trying to perform? :). Could you include the code in your question please? This could help others identify what is wrong. – Joseph Aug 11 '16 at 9:37
  • I am trying to script a few functions to process UAV images. The first thing I need to do is to reproject, which is why I need the processing module. Let me add my full script, although I'm a bit ashamed: I'm not a "real" developer and I'm sure my code is not the cleanest or the most straight-forward you have ever seen :-) – Stéphane Henriod Aug 11 '16 at 10:11
  • Don't be ashamed! Nothing I post is the most cleanest or straight-forward =) – Joseph Aug 11 '16 at 10:15
  • So I added my whole script. Feel free to have a look :-) – Stéphane Henriod Aug 11 '16 at 10:34
6

Linux QGIS 2.18.1

With this code a got it, run Processing from standalone script:

#!/usr/bin/env python
import qgis
from qgis.core import *
import sys

app = QgsApplication([],True, None)
app.setPrefixPath("/usr", True)
app.initQgis()
sys.path.append('/usr/share/qgis/python/plugins')
from processing.core.Processing import Processing
Processing.initialize()
from processing.tools import *

print Processing.getAlgorithm("qgis:creategrid")
  • This is the only combination that worked for the environment I am working in (Pycharm C.Ed on an Ubuntu 14.04 machine with Python 2.7). Before, I tried the combinations of gis.stackexchange.com/questions/129513/…, and gis.stackexchange.com/questions/176821/… and sadly, none of them imported "processing.core.Processing". I dont know why importing a module is that confusing. For the record: there is a pure Python package also called "processing" that shadows the QGIS one. – Irene Jun 14 '17 at 14:35
3

So i managed to get it working, thanks @Joseph for your hints:

# Import required modules

# Python modules
import sys
import time
import datetime
import os
from getpass import getuser

# Qgis modules and environment
from qgis.core import *
import qgis.utils
from PyQt4.QtCore import QFileInfo, QSettings

from PyQt4.QtGui import QApplication
app = QApplication([])

QgsApplication.setPrefixPath("/usr", True)
QgsApplication.initQgis()

# Prepare processing framework
sys.path.append('/home/' + getuser() + '/.qgis2/python/plugins')
from processing.core.Processing import Processing

Processing.initialize()

And I coud test it with

print Processing.getAlgorithm("qgis:creategrid")

My issue, I guess, comes from the fact that I imported the modules at the beginning of my script and then tried create the Qgi obkects from within a function. I guess it would be possible to do so as well but there is probably a flaw in my Python skills.

Now I'll try to actually ust the processing module :-)

  • Something has changed ? I try this code (from Joseph) to use processing into a standalone script and I get: ImportError: No module named processing.core.Processing Using linux QGIS 2.18.1 – Juanma Font Dec 1 '16 at 14:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.