Unable to compile QGIS 3.x (master) on Ubuntu 16.04: PyQt5.pyqtconfig module does not exist

Question

From: http://pyqt.sourceforge.net/Docs/PyQt5/pyqt4_differences.html there seems to be no more pyqtconfig module in PyQt5:

pyqtconfig Module

PyQt4’s pyqtconfig module is not supported. The section The PyQt5 Extension API describes the support that PyQt5 provides to third-party packages (e.g. QScintilla) that want to build on top of PyQt5.

So it's not possible to compile QGIS 3.2 from sources on Ubuntu (16.04) because it's used by the file FindPyQt5.py in the cmake folder:

https://github.com/qgis/QGIS/blob/master/cmake/FindPyQt5.py

cd /opt/QGIS/build3_2/
cat ../cmake/FindPyQt5.py | grep -in pyqtconfig
    34:    import PyQt5.pyqtconfig
    35:    pyqtcfg = PyQt5.pyqtconfig.Configuration()

The compilation process ends on:

Traceback (most recent call last):
  File "/opt/QGIS/cmake/FindPyQt5.py", line 34, in <module>
    import PyQt5.pyqtconfig
ImportError: No module named 'PyQt5.pyqtconfig'

My PyQt5 version is 5.11.2.

Is there any solution or workaround?

Answer 1

The FindPyQt5.py error:

By looking in more detail the FindPyQt5.py file, one can see at lines 33-41:

try:
    import PyQt5.pyqtconfig
    pyqtcfg = PyQt5.pyqtconfig.Configuration()
except ImportError:
    import PyQt5.QtCore
    import sipconfig # won't work for SIP v5
    import os.path
    import sys
cfg = sipconfig.Configuration()

So, the pyqtconfig error is related to the first part of the 'try' statement.

Then, in the 'except' part of the code, what I didn't mentionned is that I encountered some error as well which I thought was not important. But it actually was! QtCore module cannot load. It says (from a Python3.5 console):

from PyQt5 import QtCore
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: /usr/local/lib/python3.5/dist-packages/PyQt5/QtCore.so: undefined symbol: PySlice_AdjustIndices

The issue:

The problem is that with the Python3.5 version shipped with Ubuntu 16.04 --- which subversion is Python 3.5.2 --- this is a known bug in PyQt5 version 5.11.2 (latest at that time).

For compatibility reasons I did not want to try upgrading the system Python 3.5.2 to 3.5.4 where this issue seems to have been fixed.

The solution:

One must downgrade PyQt5 from 5.11.2 to 5.10.1.

After several tries, I found that this involves to downgrade as well theses two packages:
- PyQtChart from 5.11.2 to 5.10.1
- and qscintilla from 2.10.7 to 2.10.4

At this point, the QtCore module loads correctly!

But then, some other errors appears during the QGIS first during the configuration step with cmake and then during the compilation;

1. If there is some "SIP not found" during the configuration process (even if it is installed!) or many errors about a sip_corepart3.cpp during the compilation, one can upgrade it (manually, because the version downloaded by pip is an older version (sip 4.19.8)).
   To do that, one must download the tarball from here:
   https://www.riverbankcomputing.com/software/sip/download (sip version 4.19.12) and extract it to some folder.
   From this folder, run python3.5 configure.py, make and make install (Detailed explanations here: http://pyqt.sourceforge.net/Docs/sip4/installation.html ).
   Only then the SIP error disappears and the compilation finally goes to the end!

2. An issue with Qt WebKit:
   /opt/QGIS/src/core/qgswebpage.h:26:20: fatal error: QWebPage: No such file or directory
   => one has to disable the WITH_QTWEBKIT flag in cmake.
   QtWebKit is deprecated in Qt >= 5.5. It has been replaced by QtWebEngineCore. (https://doc.qt.io/qt-5.11/whatsnew55.html#deprecated-functionality More infos: https://forum.qt.io/topic/55504/anyone-knows-the-future-of-qt-webkit & http://doc.qt.io/qt-5/qtwebenginewidgets-qtwebkitportingguide.html ).

To summarize, this is the output log from cmake with all the changes. It doesn't show erros anymore:

QGIS version: 3.3.0 Master (30300)
Could not find GRASS 7
Found Proj: /usr/local/lib/libproj.so (5.2.0)
Found GEOS: /usr/lib/x86_64-linux-gnu/libgeos_c.so (3.5.0)
Found GDAL: /usr/local/lib/libgdal.so (2.3.1)
Found Expat: /usr/lib/x86_64-linux-gnu/libexpat.so
Found Spatialindex: /usr/lib/x86_64-linux-gnu/libspatialindex.so
Found Qwt: /usr/lib/libqwt-qt5.so (6.1.2)
Found libzip: /usr/lib/x86_64-linux-gnu/libzip.so
Found Sqlite3: /usr/lib/x86_64-linux-gnu/libsqlite3.so
Found PostgreSQL: /usr/lib/x86_64-linux-gnu/libpq.so
Found SpatiaLite: /usr/lib/x86_64-linux-gnu/libspatialite.so
Qt WebKit support DISABLED.
Found Qt version: 5.5.1
Found QScintilla2: /usr/lib/libqt5scintilla2.so (2.9.1)
Found QtKeychain: /usr/lib/x86_64-linux-gnu/libqt5keychain.so
Found QCA: /usr/lib/x86_64-linux-gnu/libqca-qt5.so (2.1.1)
Found QCA OpenSSL plugin
Pedantic compiler settings enabled
Found PythonInterp: /usr/bin/python3.5 (found suitable version "3.5.2", minimum required is "3") 
Found Python executable: /usr/bin/python3.5
Found Python version: 3.5.2
Found Python library: /usr/lib/x86_64-linux-gnu/libpython3.5m.so
Found Python site-packages: /usr/lib/python3/dist-packages
Found PyQt5 version: 5.10.1
Found SIP version: 4.19.12
Found QScintilla2 PyQt module: 2.10.4
Found GSL: -L/usr/lib/x86_64-linux-gnu -lgsl -lgslcblas -lm
Ctest Binary Directory set to: /opt/QGIS/build_master/output/bin
Configuring done
Generating done

And the compilation goes to the end, but QGIS itself refuses to load the Python console at startup*!

*It's probably better to separate this issue into a new thread:
Compiling QGIS 3.x (master) on Ubuntu 16.04 with Python3.6 and the latest Qt 5 and PyQt5 familiy modules (PyQt5, sip, qscintilla, pyqtchart)