Python GDAL package missing header file when installing via pip

Question

I asked this question on Stack Overflow before I realised this site existed. As this site it more appropriate, I'll ask it here too.

I'm trying to install gdal from pip pip install gdal inside a virtual environment (Ubuntu). It fails because it cannot find cpl_port.h

extensions/gdal_wrap.cpp:2853:22: fatal error: cpl_port.h: No such file or directory
compilation terminated

However GDAL is installed correctly and the header file is located at /usr/include/gdal/cpl_port.h. Is there some environment variable for GDAL that needs to be set in order for pip to find the header files?

Answer 1

selimnairb's answer is close but you wont have the headers unless you've installed libgdal-dev:

sudo apt-get install libgdal-dev

with that done,

export CPLUS_INCLUDE_PATH=/usr/include/gdal

export C_INCLUDE_PATH=/usr/include/gdal

and then

pip install GDAL

The compilation ran to completion and I have GDAL in my virtual env. Phew!

(edit 2018) Note: To be sure that you install the correct version and avoid any problem. retrieve the version with gdal-config --version. and then:

pip install GDAL==version

Answer 2

Τhe header files cannot be found for some reason. Maybe you you are operating inside a Virtual Enviroment or they are not where they should be for some reason. In any case you can specify the include dirs when installing gdal via pip.

first download python's gdal :

pip install --no-install GDAL

in later versions of pip (>= 9.0.0) pip install --no-install does not exist:

pip download GDAL

then specify where the headers are:

python setup.py build_ext --include-dirs=/usr/include/gdal/

then install it:

pip install --no-download GDAL

in later versions of pip (>= 9.0.0) pip install --no-download does not exist:

sudo python setup.py install --include-dirs=/usr/include/gdal

Here's another way to install gdal python:

$ sudo apt-add-repository ppa:ubuntugis/ubuntugis-unstable
$ sudo apt-get update
$ sudo apt-get install python-gdal

after that open IDLE:

from osgeo import gdal

and you're good to go!

Answer 3

After following a subset of this advice, this is how I got the Python GDAL 1.11.0 (the solution should be version-independent, see below) install to work on Ubuntu 14.04 with pip:

Install dependencies:

sudo apt-get install libgdal-dev libgdal1h

pip install by passing along the include path (prefix with sudo for system-wide install) and instructing pip to install the version matching the system installed GDAL version:

pip install --global-option=build_ext --global-option="-I/usr/include/gdal" GDAL==\`gdal-config --version\`

Answer 4

Installing Python package gdal into virtualenv on Linux

GDAL provides nice toolkit for GEO related operations. However, installing it to virtualenv on Linux is not a trivial task.

This recipe describes, how to do that.

note

here I use lowercase gdal for Python package and upper case GDAL for general system wide library.

Requirements

• allow using osgeo libraries (installed via gdal Python package) into virtualenv
• allow installing on Linux Ubuntu

Installation methods

There are multiple methods for installation. One requires compilation and takes few minutes more.

The other is using wheel package of pygdal package and is very quick. Anyway, to create the wheel package one needs to create it once and the creation includes the compilation step anyway.

About GDAL packages and versions

GDAL is a general C(++) based library for GEO related calculations.

GDAL utilities can be installed system-wide what makes shared libraries available, but does not install Python package itself.

GDAL comes in different versions and each Linux distribution may by default install different version.

Python package gdal requires compilation and is not trivial to install on Linux based systems as it expects few environmental variables to be set. This makes installation into virtualenv more difficult.

Each gdal version might assume different version of GDAL and will fail installing if expected version is not present in the system.

Python package pygdal is an alternative to gdal, which installs exactly the same stuff as gdal, but does it in a much more virtualenv friendly manner.

pygdal comes in versions reflecting related GDAL version. So having GDAL version 1.10.1 in the system you shall install pygdal version 1.10.1.

Python package gdal (as well as pygdal) uses root python package named osgeo and has a set of submodules, one being osgeo.gdal.

If needed, other than default versions of GDAL can be installed and used. This is out of scope of this description.

Wheel packages can be cross-compiled, this is also out of scope.

Installing GDAL into system

As pygdal requires GDAL shared libraries to be present, we must install them first.

Assuming GDAL is not yet installed, calling gdal-config will complain and give you a hint how to follow up:

$ gdal-config --version
The program 'gdal-config' is currently not installed. You can install it by typing:
sudo apt-get install libgdal-dev

Follow the hint and install it:

$ sudo apt-get install libgdal-dev

Each distribution may use different version of GDAL. To find out which we use:

$ gdal-config --version
1.10.1

Now you know, GDAL is installed and the version is 1.10.1 (the version can vary).

Install pygdal from source package (requires compilation)

Currently, pygdal is provided only in tar.gz package, which contains package sources and requires compilation.

Assuming, the version of GDAL is 1.10.1 and that our virtualenv is already activated:

$ pip install pygdal==1.10.1

It may take a while to complete, is it needs numpy, which may also require some compilation. Just wait.

Check, it is installed:

$ pip freeze|grep pygdal
pygdal==1.10.1.0

From now on, you may use osgeo package in your Python code as you like in exactly the same manner as if you would install it by gdal Python package.

Creating wheel package for pygdal

Note, that wheel packages must be created for exactly the same architecture, namely, must match:

• CPU architecture
• OS (Linux/Windows)

In our case, it must also match the version of GDAL installed.

Following steps can be done in virtualenv or not, as you like.

First, make sure, wheel package is installed:

$ pip install wheel

Assuming, you have GDAL installed and it has version 1.10.1:

$ pip wheel pygdal==1.10.1.0

and wait, until it completes.

After this, you shall find subdirectory wheelhouse and it shall contain packages with extension `whl`:

$ ls wheelhouse
numpy-1.9.1-cp27-none-linux_x86_64.whl
pygdal-1.10.1.0-cp27-none-linux_x86_64.whl

Install pygdal from wheel packages

Installation from wheel formatted packages is much faster (a second compared to minutes), as it does not require compilation.

Note, that directory with wheel packages can have any name, we will use just the name wheelhouse.

Activate virtualenv first.

Ensure, you have in wheelhouse directory both required wheel packages (for pygdal and numpy).

Ensure, GDAL is installed and the version matches version of pygdal.

Install pygdal from wheel package:

$ pip install pygdal==1.10.1.0 -f wheelhouse

The -f wheelhouse shall point to the directory with whl files.

There is no need to install numpy, it gets installed automatically.

Answer 5

Yes, doing the following before running PIP appears to work:

export CPLUS_INCLUDE_PATH=/usr/include/gdal

export C_INCLUDE_PATH=/usr/include/gdal

Answer 6

The problem with the pip installing gdal is that it only gets the bindings, not the entire library, so it can get tricky. One way to solve it is to use the pip command to download - but not install. Then you tweak the header location from the config file. Then you pip install that. meh.

I was having the same problem but realized that writing a fabric script to recompile gdal and generate the python bindings was going to take less time. You even get the benefit of filegdb with that. Go ahead and use tha gist I wrote or tweak it to your hearts content.

Answer 7

After looking right and left for a solution, here is something that works for me on Ubuntu 14.04, even from within a virtualenv with no access to the system packages :

• Install dependencies :

  sudo apt-get install libgdal-dev libgdal1h libgdal1-dev
  

• Set the compiler flags :

  export CFLAGS=$(gdal-config --cflags)
  

• Install the version corresponding to the system libraries (at the time of writing Ubuntu Trusty is using the 1.10 headers) :

  pip install GDAL==1.10.0
  

Answer 8

On Fedora 24 which has GDAL 2.0.2 in its repositories, I had to install the Python package like this:

pip install \
  --global-option=build_ext \
  --global-option="-I/usr/include/gdal/" \
  GDAL==2.0.1

Answer 9

While a while later, this provides the include path without having to bail out of pip installation: One can set the include path using an environment variable.

Assuming the headers are in /usr/include/gdal, issue an

export CPLUS_INCLUDE_PATH=/usr/include/gdal

before running pip.

Answer 10

Now you can use virtualenv and setuptools friendly version of standard GDAL python bindings pygdal.

Answer 11

Installing via Pip with Single Command

Assuming the GDAL develop package is installed and the header file versions are correct, the only command needed to install GDAL from PyPI repos is as follows:

pip install --global-option=build_ext --global-option="-I/usr/include/gdal" gdal

This obviously assumes the location of the header files is /usr/include/gdal.

Install Up-to-date GDAL

In order to install GDAL with the pip command above, the version of the header files need to be similar to the version that pip will do the build. GDAL has an updated binary reference at: https://trac.osgeo.org/gdal/wiki/DownloadingGdalBinaries.

How to Install on OpenSUSE

Following the link above, there is a URL embedded further down point to an up-to-date set of GIS packages for OpenSUSE 42.1 http://download.opensuse.org/repositories/Application:/Geo/openSUSE_Leap_42.1/Application:Geo.repo

zypper ar -f <URL>

Replace with the appropriate package. And if not using zypper there is more info at https://trac.osgeo.org/gdal/wiki/DownloadingGdalBinaries

Answer 12

I was having similar problems on a Mac. This is how I resolved it:

Firstly, I set up a virtual Python 2.7 environment using virtualenv. The Python distribution was installed in a directory called 'env'.

I then used fink to install gdal

fink selfupdate
fink update-all
fink install gdal

I also installed gdal-dev but this may not have been required since it might have already been installed with gdal.

I checked the version installed using:

gdal-config --version

On my installation, it produced the result 1.11.1

The fink installation of gdal installed the cpl_port.h header file in /sw/include/gdal1. Check your own installation. I then entered:

export CPLUS_INCLUDE_PATH=/sw/include/gdal1
export C_INCLUDE_PATH=/sw/include/gdal1
env/bin/pip install pygdal==1.11.1

That seemed to work for me but I haven't tested installation yet.

Answer 13

I was getting a similar error while trying to install the python GDAL bindings on a mac (OS 10.10.5). I installed the base GDAL software from http://www.kyngchaos.com/software/frameworks using the "Complete" download. I had to set three environment variables.

export C_INCLUDE_PATH=/Library/Frameworks/GDAL.framework/Headers
export CPLUS_INCLUDE_PATH=/Library/Frameworks/GDAL.framework/Headers
export LIBRARY_PATH=/Library/Frameworks/GDAL.framework/Versions/Current/unix/lib

The final piece was to add /Library/Frameworks/GDAL.framework/Programs to my PATH.

echo 'export PATH=/Library/Frameworks/GDAL.framework/Programs:$PATH' >> ~/.bash_profile

After that pip was able to install GDAL for python. Hope this helps.

Answer 14

These gdal Packages 0.10.1 work well for Ubuntu 12.04 : https://launchpad.net/~ubuntugis/+archive/ubuntu/ubuntugis-unstable/+sourcepub/4353415/+listing-archive-extra

gdal Packages 0.10.1 for other ubuntu version : https://launchpad.net/~ubuntugis/+archive/ubuntu/ubuntugis-unstable/+packages?field.name_filter=gdal&field.status_filter=published&field.series_filter=

Answer 15

To answer the virtualenv specific aspect of the question:

pip3 search gdal

GDAL                      - GDAL: Geospatial Data Abstraction Library
pygdal                    - Virtualenv and setuptools friendly version of
                            standard GDAL python bindings

Beware that pygdal may require a different version of GDAL, compared to what the GDAL package of python bindings requires.

---

Below is what I used to get it on recent versions of Fedora (20 and 23).

CPPFLAGS=-I/usr/include/gdal pip install gdal

Answer 16

I've had same problem on Windows 10. After some experiments I came with this solution.

1. Download and install Python 3.6 (if not installed) after installation alter environment variables PYTHONPATH=c:\python36 PATH=C:\python36\Scripts\;C:\python36\;%PATH%
2. Download *.whl for correct python version from https://www.lfd.uci.edu/~gohlke/pythonlibs/#gdal
3. Download and install http://landinghub.visualstudio.com/visual-cpp-build-tools
4. pip install *.whl (*.whl from step2)

Answer 17

If you're using Docker we open sourced our container, that simplifies using GDAL and Python 3. The container captures the steps outlined above to allow you to quickly use GDAL with your apps.

thinkwhere/gdal-python

Answer 18

This approach worked for me:

sudo apt-get install libgdal-dev
export CPLUS_INCLUDE_PATH=/usr/include/gdal
export C_INCLUDE_PATH=/usr/include/gdal
pip3 install gdal==$(gdal-config --version)

Or as a part of a Dockerfile:

RUN apt-get update && \
DEBIAN_FRONTEND=noninteractive apt-get install -y \
                libgdal-dev \
                python3-pip
ARG CPLUS_INCLUDE_PATH=/usr/include/gdal
ARG C_INCLUDE_PATH=/usr/include/gdal
RUN pip3 install gdal==$(gdal-config --version)

Answer 19

If you are using Travis CI and need a recent version of GDAL for Python, here is an example of what part of a .travis.yml file would look like:

language: python

env:
  global:
    - CPLUS_INCLUDE_PATH=/usr/include/gdal
    - C_INCLUDE_PATH=/usr/include/gdal

before_install:
  - sudo add-apt-repository -y ppa:ubuntugis/ppa
  - sudo apt-get -qq update
  - sudo apt-get -qq install libgdal-dev gdal-bin

install:
  - pip install gdal==`gdal-config --version`

Answer 20

If you're running a Debian-based distro, the GDAL python libraries are available via your package manager and can be simply installed with

sudo apt install python-gdal or sudo apt install python3-gdal

Answer 21

On Ubuntu 16.04 with conda in docker container (jupyter stack)

apt-get update
apt-get install libgdal1-dev -y
gdal-config --version
export CFLAGS=$(gdal-config --cflags)
pip install GDAL==1.11.2

Answer 22

In ubuntu, a simpler solution to install the latest gdal for python3: install library files via libgdal-dev, and python wrapper via python-gdal

sudo apt-get install libgdal-dev
sudo apt-get install python3-gdal

for python2:

sudo apt-get install python-gdal

Answer 23

You may also encounter problem whith memory usage.

When running pip install gdal==2.2.3 gcc is launch to compile something and it raises memory usage.

If you don't have enough memory, the compiler fails, with a message like

'x86_64-linux-gnu-gcc' failed with exit status 4

To fix it, you need to add more ram or free some.

Answer 24

This was how I installed gdal 2.x (NOT 3.x) on Ubuntu 18.04 (I assume 16.04 would be the same) inside a virtualenv for python3:

# add the apt repo 
sudo add-apt-repository -y ppa:ubuntugis/ubuntugis-unstable
sudo apt update
sudo apt upgrade -y

# install gdal==2.2.3 on system
# NOTE: these may change in the future, you can check versions with: 
# apt-cache policy <package name>
sudo apt install -y libgdal20=2.2.3+dfsg-2 libgdal-dev=2.2.3+dfsg-2
sudo apt install -y gdal-bin=2.2.3+dfsg-2 python3-gdal=2.2.3+dfsg-2

# ensure that it worked
gdalinfo --version  # should print 'GDAL 2.2.3, released 2017/11/20'
# start a python3 interpreter NOT inside your virtualenv
>>> from osgeo import gdal
>>> gdal.__version__  # should print '2.2.3'

# activate virtualenv 
source .venv/bin/activate

# install inside virtualenv
export CPLUS_INCLUDE_PATH=/usr/include/gdal
export C_INCLUDE_PATH=/usr/include/gdal
pip3 install gdal==2.2.3

# ensure that it worked
# start a python3 interpreter INSIDE your virtualenv
>>> from osgeo import gdal
>>> gdal.__version__  # should print '2.2.3'

Hope this helps!

Answer 25

On MacOS, this solution worked for me.

Summary: instead of pip, just use brew:

brew install gdal

Answer 26

My context

Linux Mint 19, 32 bit, python 3.6

I did this:

pip install GDAL

I get these errors (just part of it):

i686-linux-gnu-gcc: error trying to exec 'cc1plus': execvp: No such file or directory
error: command 'i686-linux-gnu-gcc' failed with exit status 1
Command errored out with exit status 1 ... 

And this works for me. I followed the instructions in this page (read carefully):

sudo apt-get update
sudo apt-get install gdal-bin 
sudo apt-get install libgdal-dev 
export CPLUS_INCLUDE_PATH=/usr/include/gdal 
export C_INCLUDE_PATH=/usr/include/gdal

sudo apt install g++ 

Then:

pip install GDAL==<ogrinfo version >

Answer 27

Create a new python environment and install gdal first using the following command:

mamba create -n new-env gdal -c conda-forge or conda create -n new-env gdal -c conda-forge

Check gdal installation: gdalinfo --version If you get an output like this GDAL 3.5.3, released 2022/10/21

You are all set!