4

Is there a way to tweak the fiona/gdal/osgeo packages installed through Anaconda so that they can be used to read/write GDB files using the ESRI FileGDB driver? The main thing I'm actually trying to achieve is writing to GDB a file.

I have found multiple answers online, but all of them involve re-installing OSGeo or GDAL from the python wheel files. These methods do not work for me because I installed fiona, OSGeo and GDAL using Anaconda. So none the solutions I found online (listed below) seem to work.

I have already downloaded the DLL file from ESRI, placed it in my C:\...\Lib\site-packages\osgeo folder and added a new environment variable called GDAL_DRIVER_PATH indicating the appropriate path. However, contrary to the suggestions in the posts listed below, the __init__.py inside the osgeo folder does not have a commented line that reads os.environ['GDAL_DRIVER_PATH'] = os.path.join(os.path.dirname(__file__), 'gdalplugins'). Therefore, I am not sure where to include this line in my own file system.

Similar (unresolved) discussions can be found on the GeoPandas and on the Fiona Github pages.

Also, here is a list of related posts/questions/sites with relevant info but which ultimately did not solve my problem.

How I installed Fiona and other packages:

I'm currently using Windows 10. After installing Anaconda, I created a new environment called myenv and installed the main libraries I was going to use:

conda create -n myenv pandas geopandas fiona spyder numpy scipy

GDAL was automatically installed as one of the associated/required packages for fiona and geopandas.

In my command prompt, when I activate myenv and type gdalinfo --version, I get the following response:

GDAL 3.0.2, released 2019/10/28

When I use the python-based route, I get the same answer.

import osgeo.gdal
print(osgeo.gdal.__version__)
>3.0.2

Additional Notes:

If the answer involves something like "you need to compile/build GDAL from source files", please include a step-by-step on how to do so, including what to do after "building" (for example, where to put the files and how to get Python to use the newly-built version of GDAL within a specific conda environment). I know this is a lot to ask, and I apologize for doing so. It's just that I have tried looking into this process and it is not very newbie-friendly.

5
  • Can you please update your question to include how you installed gdal using conda and also which version of gdal you have installed on your conda environment? – Aaron Nov 10 '20 at 21:20
  • There! added both pieces of information. I hope it helps! =) – Felipe D. Nov 10 '20 at 21:46
  • 1
    FileGDB support requires two things, 1. the Esri FileGDB API library (stored in a directory in your PATH) and 2. the GDAL FileGDB driver (either compiled into the GDAL library itself or compiled separately as a driver plugin and stored in the GDAL_DRIVER_PATH). You only have 1. (the API library) and I don't think Anaconda or conda-forge provide a filegdb enabled GDAL or a driver plugin so you're out of luck there. You may be able to get it to work if you can find and download a version of the driver as a plugin compiled for the same version of gdal that your conda env provides – user2856 Nov 10 '20 at 22:56
  • You should probably add windows as a tag, it actually makes your problem harder than usual. Also, you can always install pip packages over the ones in the Anaconda environment. This is the go-to source for people who know what packages they need but can't or don't want to bother with the hassle of building them on Windows: lfd.uci.edu/~gohlke/pythonlibs – prusswan Mar 19 at 18:54
  • This is still your best bet probably: gis.stackexchange.com/questions/193288/…. Then again, writing to FileGDB may not be necessary, GeoPackage works just as well (and far easier to work with) – prusswan Mar 19 at 18:57
0

You could actually also use gdal to open the dataset and iterate through its features, which is slow but should work. OGR/GDAL cookbook https://pcjericks.github.io/py-gdalogr-cookbook/vector_layers.html#load-data-to-memory <-Does that help you to get started with gdal for example? That is where I started to work with gdal and where I come back to when analysing the sources of fiona and pyogrio. GDAL is slow but should be working, if you know how to get FileGDB working within gdal that is. Much easier though than to build the python libraries from skratch. Chance that you find a non conda repo with built in FileGDB is bigger. Than you can pip install that one into your conda environment? I would start with that and with GDAL. I will crosscheck if FileGDB works with my installed GDAL which is somehow supplied by osgeo

4
  • Hi there! Thanks for the extra info! The problem I'm trying to solve isn't reading data using the FileGDB driver. To read GDB files, I can use the OpenFileGDB driver that already comes pre-packaged with OSGEO/GDAL. The problem I'm really trying to solve is writing to a GDB file. Sadly, that's something that I can't do using pyogrio or the OpenFileGDB driver. – Felipe D. Mar 17 at 15:31
  • Ah ok yeah so then you cant really use that above stuff. I guess without that FileGDB you will be more or less stuck Obivously because it's propriatery stuff trac.osgeo.org/gdal/wiki/FileGDB – Sönke Schmachtel Mar 19 at 7:36
  • But yeah I havent build yet succesfully GDAL on windows. You can use the instructions but then I got stuck with proj. Anyway one thing to peak into could be this script here: github.com/srenoes/Fiona/blob/master/scripts/… Make and make install is typically for unix though, but who knows if this isnt also usable in windows. Otherwise I scan repos I know using the libraries of interest to find building scripts and usually there is some for the platforms Appveyor and/or Travis – Sönke Schmachtel Mar 19 at 7:48
  • Thanks for the info! But like I said in my request for the bounty, if adding support to the FileGDB driver requires building GDAL from scratch, I'm going to need a bit more detail on how to do so because, like you, I also got lost trying to understand what needs to be done. Also, consider that I'm someone who isn't super familiar with compiling C codes from sourcefiles either. I understand the concept and have even done so myself for veeeeeery simple programs many years ago. But for something like this, I'm going to need a bit more clarity. Thanks again, btw!!! – Felipe D. Mar 19 at 17:42
0

There is also the dirty little trick to do pip install although conda is used in principle. That usually works. With that it could be easier to get fiona compiled. I tested the build instructions for windows from the fiona help page and they actually worked where I used OSGeo4W as the precompiled gdal source . I guess you have to fork fiona, clone it to local with git or download it and within the source you might want search for allowed drivers and at least add FileGDB there if it isnt allowed yet. Otherwise in the gitter channel of geopandas there was some hints about how to compile pyogrio in the right way. But actually in pyogrio code you really have to extend that allowed driver list. Because it actually also prevented me to use openapi, although it worked fine on gdal to open an url in a dataset. Nope isnt there for my gdal install https://pcjericks.github.io/py-gdalogr-cookbook/vector_layers.html#get-list-of-ogr-drivers-alphabetically-a-z And I cannot make sense of this here either: https://trac.osgeo.org/gdal/ticket/4420 from https://gdal.org/drivers/vector/filegdb.html#vector-filegdb tricky stuff

2
  • In your answer, you say "add FileGDB there." Can you be a bit more specific about how someone would do that? That's the heart of my question and I can't find a good answer for it. – Felipe D. Mar 17 at 17:17
  • I also forgot to mention: your answer contains instructions about how to install Fiona on a fresh new Python environment (i.e., an environment in which Fiona & GDAL haven't yet been installed). But my original question is: consider that Fiona and GDAL have already been installed using Anaconda. In that case, is it possible to tweak the libraries in Python to get the FileGDB driver to work? Thanks again for your patience. – Felipe D. Mar 19 at 18:35
0

Are you sure that you cant do it? I think if you go through our discussion there Brendan confirmed that this driver list where you find FileGDB with only r option is not really in use. Or did you try and it didn't work/ and I understood something wrong or missread the code in pyogrio.

Anyway if you managed to find gdal drivers with FileGDB support you should be definitely able to use GDAL. Or did you use the OpenFileGDB driver with only read support? If you have the FileGDB somehow compiled into GDAL (or is it just an extension, I don't know?) then I would try this here (adapted from read to memory script in cookbook):

That's basically copying layer by layer with the appropriate layer/file or dir names.

from osgeo import ogr

#open an input datasource
indriver=ogr.GetDriverByName('FileGDB')
srcdb = indriver.Open('File1.gdb',0)

#create an output datasource in memory
outdriver=ogr.GetDriverByName('FileGDB')
source=outdriver.CreateDataSource('File2.gdb')

#open the memory datasource with write access
tmp=outdriver.Open('File2.gdb',1)

#copy a layer to memory
pipes_mem=source.CopyLayer(srcdb.GetLayer('layer_you_have'),'layer_you_have',['OVERWRITE=YES'])
2
  • Thanks for the clarificaiton. So, the main problem here is that I don't have FileGDB compiled into GDAL. The GDAL distribution that I have was installed automatically through Anaconda, which does not contain the FileGDB. And that's what I'm asking: how can I make GDAL use the FileGDB driver after GDAL was compiled. – Felipe D. Mar 19 at 17:14
  • And to answer your question: yes, I did try that! When I type in those initial commands, the indriver variable just contains None, because GDAL cannot find the FileGDB driver. – Felipe D. Mar 19 at 17:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.