5

I am trying my hardest to utilize the work of the good folks at the GeoDa center, but I cannot seem to get all of the components to load. GDAL, in particular, has been giving me some difficulty. The biggest issue arises because I am operating within the IPython Notebook. If you are unfamiliar with the Notebook, to say it's worth a look is the understatement of the century.

In any event, after some difficulty and some assistance, I was able to install the GDAL module. It was even importable at the Python prompt:

Python 2.7.5 |Anaconda 1.6.1 (64-bit)| (default, Jun 28 2013, 22:10:09) 
[GCC 4.1.2 20080704 (Red Hat 4.1.2-54)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import gdal
>>> 

However, after my difficulties in the Notebook, I went back to the IPython prompt to discover the following difficulty:

Python 2.7.5 |Anaconda 1.6.1 (64-bit)| (default, Jun 28 2013, 22:10:09) 
Type "copyright", "credits" or "license" for more information.

IPython 1.0.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import gdal
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-27bf4694dd2b> in <module>()
----> 1 import gdal

/home/choct155/analysis/Anaconda/lib/python2.7/site-packages/gdal.py in <module>()
      1 # import osgeo.gdal as a convenience
----> 2 from osgeo.gdal import deprecation_warn
      3 deprecation_warn('gdal')
      4 
      5 from osgeo.gdal import *

/home/choct155/analysis/Anaconda/lib/python2.7/site-packages/osgeo/__init__.py in <module>()
     19                 fp.close()
     20             return _mod
---> 21     _gdal = swig_import_helper()
     22     del swig_import_helper
     23 else:

/home/choct155/analysis/Anaconda/lib/python2.7/site-packages/osgeo/__init__.py in swig_import_helper()
     15         if fp is not None:
     16             try:
---> 17                 _mod = imp.load_module('_gdal', fp, pathname, description)
     18             finally:
     19                 fp.close()

ImportError: /usr/lib/libgdal.so.1: undefined symbol: sqlite3_column_table_name

Has anyone else encountered this issue? I know that sqlite3 is an installed module already, so if I can't see it being a dependency issue. That being said, I am not sure where to start beyond poking around the osgeo source. Any thoughts would be greatly appreciated. (I am using Ubuntu 13.04, by the way.)

I first discovered this problem when having trouble with the R package, rgdal from within Notebook (but curiously not from within RStudio). Upon discovering deficiencies on the Python side as well, I thought it might be useful to talk to a more spatially-focused crowd.

UPDATE: Thank you for the prompt responses. With respect to the first comment (SQLite), how would I verify that one was using a different version from the other? I checked the paths in os.environment, and they are the same. Thoughts?

With respect to the latter, sys.executable was identical (and I initiated both from the same working directory):

>>> import sys, os
>>> sys.executable
'/home/choct155/analysis/Anaconda/bin/python'

In [2]: import sys, os

In [3]: sys.executable
Out[3]: '/home/choct155/analysis/Anaconda/bin/python'

However, I did find differences in os.environ:

print set(python_environ.items())-set(ipython_environ.items())

print set(ipython_environ.items())-set(python_environ.items())
*****************************
set([('OLDPWD', '/home/choct155/analysis/fwtools/FWTools-linux-x86_64-3.0.6'), ('WINDOWID', '54529262'), ('_', '/home/choct155/analysis/Anaconda/bin/python'), ('C_INCLUDE_PATH', '/usr/include/gdal'), ('CPLUS_INCLUDE_PATH', '/usr/include/gdal')])
set([('WINDOWID', '54526233'), ('_', '/home/choct155/analysis/Anaconda/bin/ipython'), ('OLDPWD', '/tmp/pip-build-root/GDAL')])

To my eye, the environmental variables that I set in .bashrc (the include paths) are the elephant in the room. In fact, without these set, I couldn't get GDAL to install at all. What I am not clear on is why they would apply only to Python when IPython is using the same Python executable as the back end.

Sidenote: To those that are working with these tools, in general it's probably a good idea to check out minrk's github page and GeospatialPython.org.

UPDATE: Admittedly, I have very little theory behind this attempted fix, but I tried to harmonize the two environments by explicitly adding the missing environmental variables with a startup script in the default_profile for IPython. This, probably predictably, did not work. I am guessing that these variables are utilized only during installation. Since I already installed gdal...

!sudo pip install gdal
*********************
Requirement already satisfied (use --upgrade to upgrade): gdal in /home/choct155/analysis/Anaconda/lib/python2.7/site-packages
Cleaning up...

...I am still not sure how to remedy this. Further, I am still unclear on how and why my Python and IPython platforms would be looking at different versions of SQLite when they have the same PATH. Any help would be greatly appreciated!

  • Can you humor me and show that sys.executable is identical between the two cases? Also compare os.environ and os.getcwd(). – minrk Sep 25 '13 at 23:43
  • You may have an SQLite version conflict. Everybody uses SQLite so most systems have multiple copies of the library lying around. My guess is GDAL is finding a version that appears in the system path before the version it needs and it's incompatible. – GeospatialPython.com Sep 26 '13 at 3:19
  • In general I think that it is better to use the gdal from the ubuntugis ppa than trying to pip install it by hand. – RemcoGerlich Nov 28 '13 at 14:17
  • Afraid that was my first route. When that didn't work I moved to pip. – Marvin Ward Jr Dec 1 '13 at 22:37
0

So it turned out to be a SQLite version conflict (hat tip GP.com). It seems as though IPython loaded a version of SQLite upon initializing the session, and GDAL requires another version. The solution was to interrupt the IPython default importation (thank you IPython help chat). This was done by creating a new script titled sqlite3.py, and populating it with raise ImportError. The script was placed in the first directory in the IPython sys.path.

Caveat: SQLite is used under the hood in IPython for various actions, and thus a version switch may cause some issues. At the current time, this is the cost of doing business for using these (awesome) tools together.

  • 1
    Ho-hum. I use iPython and gdal together all the time. This seems quite a hackish solution. I'd not consider this anywhere near final, even if it rigs it up for you so that it works. Is iPython's SQLite version newer or older than the one you need to import? – chryss Sep 27 '13 at 4:14
  • Agreed, it is hackish. I may have been overzealous in implying this is the only way to do this (I was just really happy to have a solution, and grateful for the assistance from IPython developers.) To answer your question, IPython 1.7.1 appears to load SQLite3 2.6.0 by default, while according to gdal.org, OGR appears to require SQLite 3.x or better. (I obtained the IPython info in session, but I wasn't sure how to check the GDAL module other than by reference.) Do you have a better solution? I am open to something more seamless. – Marvin Ward Jr Sep 27 '13 at 4:46
  • 2.6 is the version of the Python interface to sqlite - to see the sqlite version, you need to do sqlite3.sqlite_version. It will be > 3, that's why it's called sqlite3. I suspect the proper solution is to ensure that Python and GDAL are compiled against the same version of sqlite. – Thomas K Sep 27 '13 at 17:44
  • Yes, that would make sense wouldn't it (the version)? I am afraid I don't know how to specify the proper version of dependency. Do I achieve this by way of an pip option? Or does this something I would have to compile from source to do? – Marvin Ward Jr Sep 27 '13 at 21:31
  • 1
    I thought I should circle back. I ended up using the conda installer provided in the Anaconda python distro to install GDAL (after uninstalling the pip and the ubuntugis ppa installs). It forced the restriction specified by Thomas K, and GDAL imports in IPython just fine. That being said, a more general solution that does not rely on the Anaconda distro would still be a useful add here. – Marvin Ward Jr Dec 1 '13 at 23:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.