5

I would like to write a geopandas.Dataframe directly to (Google) cloud storage, from the documentation fiona enables this, however I am not sure how to supply the correct **kwargs for the geopandas.Dataframe.to_file function. any examples?

import geopandas
gdf = geopandas.read_file("my-file.shp")
gdf.to_file("gs://my-bucket/my-file.shp", driver='ESRI Shapefile')

1 Answer 1

1

GeoPandas relies on Fiona/GDAL which has trouble writing to cloud storage. Try this instead: If you have a service account that can access the bucket with Storage Admin (or at least Storage Create and Storage Read) Create a key for the service account and store it locally. Set the environment variable:

export GOOGLE_APPLICATION_CREDENTIALS="path/to/key_file.json"

Then the following should work.

from rasterio.io import MemoryFile
from gcsfs import GCSFileSystem
import geopandas

with MemoryFile() as mem_dst:
    gdf = geopandas.read_file("my-file.shp")
    gdf.to_file(mem_dst.name, driver='ESRI Shapefile')
    #"UPLOAD TO CLOUD"
    fs=GCSFileSystem()
    with fs.open("gs://mybucketname/objectname.shp", 'wb') as file:
        result = file.write(mem_dst.read())

You could use a local disk file instead of a memory file.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.