5

I need to have a layer time-enabled for Esri application-making. Coming from the source, it has an int type column called year. I need to find a way to automate its conversion to date.

I first tried with Geopandas, but it gives a ValueError if I try and save to Shapefile with a datetime[ms] dtype:

ValueError: Invalid field type <class 'pandas.tslib.Timestamp'>

I know Fiona can handle date, time, and datetime field types though:

>>> fiona.FIELD_TYPES_MAP

{'date': fiona.rfc3339.FionaDateType,
 'datetime': fiona.rfc3339.FionaDateTimeType,
 'float': float,
 'int': int,
 'str': str,
 'time': fiona.rfc3339.FionaTimeType}

When I load it with Fiona I get:

{'geometry': 'Point',
 'properties': OrderedDict([('npri_id', 'int:9'),
              ('facility', 'str:80'),
              ('year', 'int:9'),
              ...])}

How would I go about changing it to date?

1
  • What is the date format in your file? 05.04.2018 or 05.04.18 etc. May 4, 2018 at 21:20

3 Answers 3

3

You can automate it with geopandas, but there seems to be an issue in automatically converting the pandas datetime objects to the right properties schema. Fortunately, as geopandas is built directly on top of fiona for reading and writing you can specify a schema for writing output, e.g.:

schema = {
    'geometry': 'Point',
    'properties': {
        'npri_id': 'int',
        'facility': 'str',
        'year': 'datetime',
}}

geodataframe.to_file('output.shp', schema=schema)

Note that the number of fields in the schema must match the number of fields in the geodataframe to export (though you can of course subset the fields to export).

2
  • Hi. In the schema of my geodataframe, the time field has already 'datetime' as datatype. But still when I am trying to export the geodataframe to ESRI shapefile it is throwing the error "DriverSupportError: ESRI Shapefile does not support datetime fields".
    – RRSC NGP
    Feb 5 at 16:02
  • 1
    @RRSCNGP Yeah, shapefile isn't a good format for many reasons. One of those is the lack of datetime support. Typically you have to do the classic excel trick of splitting into year, month, day, hour, minute, second columns to deal with it numerically (still not great), or to export the datetime as an ISO format string.
    – om_henners
    Feb 16 at 4:47
2

Thanks again to om_henners for the answer! I'll go through my own steps through the problem so that it can be used for others with this problem. Maybe eventually find its way in geopandas proper? 😀

import geopandas as gpd
import pandas as pd

gdf = gpd.read_postgis(query, con)  # These are defined elsewhere

# Make sure 'year' is type 'datetime'
gdf['year'] = pd.to_datetime(gdf['year'], format='%Y')

schema = gpd.io.file.infer_schema(gdf)
print(schema)

>>> {'geometry': 'Point',
     'properties': OrderedDict([('npri_id', 'int'),
                   ('facility', 'str'),
                   ('year', 'int'),
                  ...])}


In my example, I want to change year's datatype to datetime.

schema['properties']['year'] = 'datetime'
gdf.to_file('test.shp', schema=schema)

And this indeed becomes a Shapefile that is "time-enabled" in Esri applications.

1

The existing answers did not work for me (fiona 1.8.13, geopandas 0.8.0). The ESRI shapefile format supports python's date class, but not the datetime class https://github.com/Toblerity/Fiona/issues/572.

Instead of this:

gdf['datetime'] = pd.to_datetime(x)  # datetime

Use this:

gdf['date'] = pd.to_datetime(x).date()  # date
gdf['time'] = pd.to_datetime(x).time()  # time

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.