Writing automated tests for QGIS plugins

Question

I'm looking for advice on writing automatic tests for QGIS plugins written in Python.

I have written tests for Python scripts in the past using PyUnit (the unittest module), but have never done so for an application with a GUI. I've found a page describing how to use PyQt4.QTest to do unit tests on Qt widgets (http://www.voom.net/pyqt-qtest-example), but I'm struggling to see how I can use this with a widget that has been designed to run from within QGIS.

The section on "Testing" in the PyQGIS documentation is notably absent.

What I have so far is:

• Keep the actual processing of data in isolated modules or functions, and write unit tests for those;
• Perform basic testing of the UI using QTest;
• Pray that it all holds together when using the plugin from within QGIS.

Is there a better way?

Answer 1

The capabilities for testing QGIS plugins (particularly the question of integration testing, within a QGIS environment, as the OP highlights) has improved a great deal recently. I therefore hope this update will help contemporary readers, as well as the OP.

Boundless published a must-read article in July 2016 for anyone serious about automating the testing of QGIS plugins entitled; QGIS Continuous Integration Testing Environment for Python Plugins. It describes the approach and tools they use - all of which are open source.

Key aspects are:

• Their special QGIS plugin tester which can automate tests inside the QGIS environment
• The use of docker QGIS images, allowing testing against various QGIS versions/configurations in a container-base environment
• A special docker QGIS image, which is used for testing QGIS itself, but which - by invoking qgis_testrunner.sh can be used to run unit tests on a plugin
• The use of Travis CI for continuous integration - i.e. full test suite is run with every new code commit

If you are familiar with Travis CI/docker it ought to be relatively easy to set up.

1. Pull the Docker image with the QGIS testing environment and run it
2. Run qgis_setup.sh NameOfYourPlugin to install the plugin and prepare QGIS for the test runner
3. Optionally perform all operations needed to build your plugin
4. Run the test runner inside the Docker invoking the qgis_testrunner.sh

You asked for best practice & as of today I'd certainly consider this it. QGIS docs still haven't a dedicated section on plugin testing (I expect this will change shortly) but the "Pray that it all holds together" approach is certainly no longer the only option.

UPDATE (April-2020): Some of the mentioned links no longer work, but a small example of how to run the tests using Travis CI is provided here along with the qgis-testing environment Dockerfile by Planet, formerly known as Boundless.

Answer 2

It looks like this is possible to use unittest to test Python plugins loaded into a standalone Python application.

qgis.core.iface isn't available from standalone applications, so I've written a dummy instance that returns a function which will accept any arguments given to it and do nothing else. This means that calls like self.iface.addToolBarIcon(self.action) don't throw errors.

The example below loads a plugin myplugin, which has some drop down menus with layer names taken from the map layer registry. The tests check to see if the menus have been populated correctly, and can be interacted with. I'm not sure if this is the best way to load the plugin, but it seems to work.

#!/usr/bin/env python

import unittest

import os
import sys

# configure python to play nicely with qgis
osgeo4w_root = r'C:/OSGeo4W'
os.environ['PATH'] = '{}/bin{}{}'.format(osgeo4w_root, os.pathsep, os.environ['PATH'])
sys.path.insert(0, '{}/apps/qgis/python'.format(osgeo4w_root))
sys.path.insert(1, '{}/apps/python27/lib/site-packages'.format(osgeo4w_root))

# import Qt
from PyQt4 import QtCore, QtGui, QtTest
from PyQt4.QtCore import Qt

# import PyQGIS
from qgis.core import *
from qgis.gui import *

# disable debug messages
os.environ['QGIS_DEBUG'] = '-1'

def setUpModule():
    # load qgis providers
    QgsApplication.setPrefixPath('{}/apps/qgis'.format(osgeo4w_root), True)
    QgsApplication.initQgis()

    globals()['shapefile_path'] = 'D:/MasterMap.shp'

# FIXME: this seems to throw errors
#def tearDownModule():
#    QgsApplication.exitQgis()

# dummy instance to replace qgis.utils.iface
class QgisInterfaceDummy(object):
    def __getattr__(self, name):
        # return an function that accepts any arguments and does nothing
        def dummy(*args, **kwargs):
            return None
        return dummy

class ExamplePluginTest(unittest.TestCase):
    def setUp(self):
        # create a new application instance
        self.app = app = QtGui.QApplication(sys.argv)

        # create a map canvas widget
        self.canvas = canvas = QgsMapCanvas()
        canvas.setCanvasColor(QtGui.QColor('white'))
        canvas.enableAntiAliasing(True)

        # load a shapefile
        layer = QgsVectorLayer(shapefile_path, 'MasterMap', 'ogr')

        # add the layer to the canvas and zoom to it
        QgsMapLayerRegistry.instance().addMapLayer(layer)
        canvas.setLayerSet([QgsMapCanvasLayer(layer)])
        canvas.setExtent(layer.extent())

        # display the map canvas widget
        #canvas.show()

        iface = QgisInterfaceDummy()

        # import the plugin to be tested
        import myplugin
        self.plugin = myplugin.classFactory(iface)
        self.plugin.initGui()
        self.dlg = self.plugin.dlg
        #self.dlg.show()

    def test_populated(self):
        '''Are the combo boxes populated correctly?'''
        self.assertEqual(self.dlg.ui.comboBox_raster.currentText(), '')
        self.assertEqual(self.dlg.ui.comboBox_vector.currentText(), 'MasterMap')
        self.assertEqual(self.dlg.ui.comboBox_all1.currentText(), '')
        self.dlg.ui.comboBox_all1.setCurrentIndex(1)
        self.assertEqual(self.dlg.ui.comboBox_all1.currentText(), 'MasterMap')

    def test_dlg_name(self):
        self.assertEqual(self.dlg.windowTitle(), 'Testing')

    def test_click_widget(self):
        '''The OK button should close the dialog'''
        self.dlg.show()
        self.assertEqual(self.dlg.isVisible(), True)
        okWidget = self.dlg.ui.buttonBox.button(self.dlg.ui.buttonBox.Ok)
        QtTest.QTest.mouseClick(okWidget, Qt.LeftButton)
        self.assertEqual(self.dlg.isVisible(), False)

    def tearDown(self):
        self.plugin.unload()
        del(self.plugin)
        del(self.app) # do not forget this

if __name__ == "__main__":
    unittest.main()

Answer 3

The accepted answer is outdated now. In my opinion, getting a docker image running for testing purposes is the preferable way to go. Instructions at the official QGIS repo (QGIS's docker image) is self-sufficient, and explain in detail how to get local testing and automated testing running. The fine people at OPENGIS.ch too do a great job providing a guide on how to perform unit testing using the official docker image.

What I personally was stuck in was the directory structure that is required for CI/unit-testing. The instructions talk about the directory structure of the tests directory. I've added the rest of the project structure that is also needed (which was not obvious to me, and I hope it helps others who dare to tread these treacherous paths):

/Project/
├── PluginDirectory/ (This directory gets deployed as your actual plugin)
|   ├── __init__.py
|   └──"Rest of your Code"
└── tests/
    ├── test_thatthing.py
    └── test_thatotherthing.py

Now this tests directory can be mounted as /tests_directory/ while running the docker image, as such:

$ docker run -d --name qgis -v /tmp/.X11-unix:/tmp/.X11-unix \
    -v /tests/:/tests_directory \
    -e DISPLAY=:99 \
    qgis/qgis:latest

Answer 4

I've also put a DummyInterface together, which enables you to test QGIS plugins standalone. After reading Snorfalorpagus blog, check out my answer here.

To find a real-life example, on how I test(ed) QGIS-plugins visit this github project at https://github.com/UdK-VPT/Open_eQuarter/tree/master/mole and have a look into the tests-package.

Answer 5

This could help: Test PyQt GUIs with QTest and unittest http://www.voom.net/pyqt-qtest-example