gdal/geopandas data object compatibility in python

Question

Is it possible to convert an ogr datasource into a geopandas GeoDataFrame without writing the datasource first as a physical file and reading it back? I am referring to something like

import ogr, geopandas

drv = ogr.GetDriverByName('MEMORY')
source = drv.CreateDataSource('test')
gdf = geopandas.GeoDataFrame(source)

The above throws an error:

Traceback (most recent call last): File "", line 1, in File "C:\ProgramData\Anaconda2\envs\wps_env36\lib\site-packages\geopandas\geodataframe.py", line 47, in init super(GeoDataFrame, self).init(*args, **kwargs) File "C:\ProgramData\Anaconda2\envs\wps_env36\lib\site-packages\pandas\core\frame.py", line 422, in init raise ValueError('DataFrame constructor not properly called!') ValueError: DataFrame constructor not properly called!

Which I get, no problem. However I find it hard to believe that you can't transfer data objects from ogr to geopandas and back without writing to physical files. What if you don't want to write a file because it is some part of an intermediate process you are trying to run?

Short of recreating the datasource feature by feature, geometry by geometry, field by field and, worst of all, transferring all the spatial reference data by writing custom wrapper functions, is there a way for ogr and geopandas to communicate?

Answer 1

The GDAL memory driver is written for GDAL and obviously exactly for improving the performance https://www.gdal.org/drv_memory.html as you wrote. The documentation seems to suggest that other processes can't open a memory datastore.

Memory This driver implements read and write access layers of features contained entirely in memory. This is primarily useful as a high performance, and highly malleable working data store. All update options, geometry types, and field types are supported.

There is no way to open an existing Memory datastore. It must be created with CreateDataSource() and populated and used from that handle. When the datastore is closed all contents are freed and destroyed.

GDAL memory rasters https://www.gdal.org/frmt_mem.html can be opened. The difference is probably that for memory raster it is possible to define the offset where the data begins in the memory:

It is possible to open an existing array in memory. To do so, construct a dataset name with the following format:

MEM:::option=value[,option=value...]

For example:

MEM:::DATAPOINTER=342343408,PIXELS=100,LINES=100,BANDS=3,DATATYPE=Byte, PIXELOFFSET=3,LINEOFFSET=300,BANDOFFSET=1

or

MEM:::DATAPOINTER=0x1467BEF0,PIXELS=100,LINES=100,BANDS=3,DATATYPE=Byte, PIXELOFFSET=3,LINEOFFSET=300,BANDOFFSET=1

DATAPOINTER: address of the first pixel of the first band. The address can be represented as a hexadecimal or decimal value. Hexadecimal values must be prefixed with '0x'. Some implementations (notably Windows) doesn't print hexadecimal pointer values with a leading '0x', so the prefix must be added. You can use CPLPrintPointer to create a string with format suitable for use as a DATAPOINTER.