Trying to use GDAL in Windows Apache WSGI with Python 3.9.5 from OSGeo4W

Question

Not much is missing and I'm going crazy. I'm pulling my hair out to get GDAL running under WSGI in Windows Apache using a Python 3.9.5 Installation from OSGeo4W.

I have installed QGIS 3.24.2 with GDAL 3.4.2 using osgeo4w-setup.exe. When I open the OSGeo4W Shell an run following Python lines of code, everything works as expected.

But when I run the code in my WSGI Python application function, I get an error which tells me DLL load failed while importing _gdal.

  File "C:/OSGeo4W64/apps/qgis-web-client/wsgi/wmservice.py", line 61, in <module>\r
    from osgeo import gdal,osr,ogr\r
  File "C:\\OSGeo4W64\\apps\\Python39\\lib\\site-packages\\GDAL-3.4.2-py3.9-win-amd64.egg\\osgeo\\__init__.py", line 45, in <module>\r
    _gdal = swig_import_helper()\r
  File "C:\\OSGeo4W64\\apps\\Python39\\lib\\site-packages\\GDAL-3.4.2-py3.9-win-amd64.egg\\osgeo\\__init__.py", line 42, in swig_import_helper\r
    return importlib.import_module('_gdal')\r
  File "C:\\OSGeo4W64\\apps\\Python39\\lib\\importlib\\__init__.py", line 127, in import_module\r
    return _bootstrap._gcd_import(name[level:], package, level)\r
ModuleNotFoundError: No module named '_gdal'\r
mod_wsgi (pid=3444): Failed to exec Python script file 'C:/OSGeo4W64/apps/qgis-web-client/wsgi/wmservice.py'.
mod_wsgi (pid=3444): Exception occurred processing WSGI script 'C:/OSGeo4W64/apps/qgis-web-client/wsgi/wmservice.py'.
Traceback (most recent call last):\r
  File "C:\\OSGeo4W64\\apps\\Python39\\lib\\site-packages\\GDAL-3.4.2-py3.9-win-amd64.egg\\osgeo\\__init__.py", line 29, in swig_import_helper\r
    return importlib.import_module(mname)\r
  File "C:\\OSGeo4W64\\apps\\Python39\\lib\\importlib\\__init__.py", line 127, in import_module\r
    return _bootstrap._gcd_import(name[level:], package, level)\r
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import\r
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load\r
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked\r
  File "<frozen importlib._bootstrap>", line 666, in _load_unlocked\r
  File "<frozen importlib._bootstrap>", line 565, in module_from_spec\r
  File "<frozen importlib._bootstrap_external>", line 1173, in create_module\r
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed\r
ImportError: DLL load failed while importing _gdal: The specified procedure could not be found.\r
\r
During handling of the above exception, another exception occurred:\r
\r
Traceback (most recent call last):\r
  File "C:/OSGeo4W64/apps/qgis-web-client/wsgi/wmservice.py", line 61, in <module>\r
    from osgeo import gdal,osr,ogr\r
  File "C:\\OSGeo4W64\\apps\\Python39\\lib\\site-packages\\GDAL-3.4.2-py3.9-win-amd64.egg\\osgeo\\__init__.py", line 45, in <module>\r
    _gdal = swig_import_helper()\r
  File "C:\\OSGeo4W64\\apps\\Python39\\lib\\site-packages\\GDAL-3.4.2-py3.9-win-amd64.egg\\osgeo\\__init__.py", line 42, in swig_import_helper\r
    return importlib.import_module('_gdal')\r
  File "C:\\OSGeo4W64\\apps\\Python39\\lib\\importlib\\__init__.py", line 127, in import_module\r
    return _bootstrap._gcd_import(name[level:], package, level)\r
ModuleNotFoundError: No module named '_gdal'\r

Next you see an extract from my wmservice.py file (until gdal will be imported).

# -*- coding: utf-8 -*-

# set Environment variables
import os,sys
os.environ['PATH'] = 'C:\\OSGEO4W64\\bin;C:\\OSGEO4W64\\apps\\qgis\\bin;C:\\OSGEO4~1\\apps\\qt5\\bin;C:\\OSGEO4~1\\apps\\Python39\\Scripts;C:\\Windows\\system32;C:\\Windows;C:\\Windows\\system32\\WBem'
os.environ['PYTHONHOME'] = 'C:\\OSGeo4W64\\apps\\Python39'
os.environ['PYTHONPATH'] = 'C:\\OSGeo4W64\\bin;C:\\OSGeo4W64\\apps\\qgis\\python;C:\\OSGeo4W64\\apps\\Python39;C:\\OSGeo4W64\\apps\\Python39\\Scripts;'
os.environ['GDAL_FILENAME_IS_UTF8'] = 'YES'
os.add_dll_directory(r'C:\OSGeo4W64\bin')
os.environ['GDAL_DRIVER_PATH'] = r'C:\OSGEO4~1\bin\gdalplugins'
os.environ['PYTHON_EGG_CACHE'] = r'D:\Python-Egg-Cache'
os.environ['USE_PATH_FOR_GDAL_PYTHON'] = 'YES'

sys.path.insert(0,'C:/OSGeo4W64/bin')
sys.path.append('C:/OSGeo4W64/apps/Python39')
sys.path.append('C:/OSGEO4W64/apps/qgis/bin')
sys.path.append('C:/OSGeo4W64/apps/Python39/Scripts')
sys.path.append('C:/OSGeo4W64/apps/qgis/python')
sys.path.append('C:/Windows/system32')
sys.path.append('C:/Windows')
sys.path.append('C:/Windows/system32/WBem')

os.environ['GDAL_LIBRARY_PATH'] = r'C:\OSGeo4W64\bin\gdal304.dll'
os.environ['GEOS_LIBRARY_PATH'] = r'C:\OSGeo4W64\bin\geos_c.dll'

## Important !! set GDAL_DATA in global Windows Environment
## Unfortunately, setting it here does not help !!
#os.environ['GDAL_DATA'] = r'C:\OSGEO4W64\share\gdal'
os.environ['PROJ_LIB'] = r'C:\OSGEO4W64\share\proj'


from osgeo import gdal,osr,ogr

P.S.: I used os.environ and sys.path to set Windows PATH, because I didn't know which one will be used.

I have absolutely no idea what's missing here. I checked all file permissions and even installed GDAL from Christoph Gohlke (https://www.lfd.uci.edu/~gohlke/pythonlibs), unfortunately without luck. Using Gohlkes installation, Python GDAL claims to have no PDF driver support, but gdalinfo.exe works without any problem.

Maybe someone can shade some light on this?

Answer 1

After many hours of trial and error, before giving up, I made one last attempt that finally solved the problem.

I removed the OSGeo4W GDAL installation from C:\OSGeo4W64\apps\Python39\Lib\site-packages and installed GDAL from Christoph Gohlke's site (GDAL-3.4.2-cp39-cp39-win_amd64.whl). With this installation the GDAL DLL was found and loaded, but PDF support was missing.

In the next step I copied gdal304.dll and all other DLLs from C:\OSGeo4W64\binto C:\OSGeo4W64\apps\Python39\Lib\site-packages\osgeo and the problem was solved.