I am initiating python with little knowledge on the theme.

Recently, I have been trying to manipulate a data-point Shapefile. Each point has attributes as date acquired and remote sensor applied. Now I have to filter this data according to these two attributes, but I am having real trouble.

I have been able to filter the sensor, but not the date. The date parameter is in string format, as the following format: "yyyy/mm/dd hh:mm:ss".

My first attempt was to split the date attribute into separate attributes like: year, month, day and so on. So later, I could filter each point through GDAL library.

Here is the code which I have have extracted from the this site and tried to apply without success:

   # pay note that elem.GetFieldAsString(7)[5:7] is supposedly the function to return the month from the date string, once it is the eighth attribute from my shapefile.

import sys
import os

    from osgeo import gdal,osr,ogr
    sys.exit("ERROR: GDAl library !")

import fiona
# schema: it is a simple dictionary with geometry and properties as keys
schema = {'geometry': 'LineString','properties': {'test': 'int'}}
# for defining the geometry, you need Shapely
from shapely.geometry import LineString, mapping
# two simples geometries
lines = [LineString([(272830.63,155125.73),(273770.32,155467.75)]),LineString([(273536.47,155914.07),(272033.12,152265.71)])]
with fiona.open('myshp.shp', 'w', 'ESRI Shapefile', schema) as layer:
    for line in lines:
        # filling schema
        elem = {}
        # geometry with mapping function of shapely
        elem['geometry'] = mapping(line) 
        # attribute value (the same here)
        elem['properties'] = {'test': 145}
        # writing element in the file

# 1) Open the original shapefile:
vector_file = "original_path_file.shp"
shapefile_exit = "new_path_file.shp"    

shapefile =fiona.open(vector_file)

#read the schema
schema2 = shapefile.schema
print schema2

# 2) As it is a dictionary, it is easy to add new fields/keys in the properties:


# 3) Now we create a new shapefile copying myshp.shp with the new schema :

with fiona.open(shapefile, 'r') as input:
    schema = schema2
    # writing the new shapefile
    with fiona.open(shapefile_exit, 'w', 'ESRI Shapefile', schema) as output:
        for elem in input:
            # add the new attribute value

            geometry = elem.GetGeometryRef() 
            elem['properties']['string']= elem.GetFieldAsString(7)[5:7]
            output.write({'properties': elem['properties'],'geometry': mapping(geometry(elem['geometry']))})
  • Are you willing and/or able to use another library besides shapely and fiona? If so, filter operations are pretty easy with GeoPandas (geopandas.org/indexing.html). – neighdough Oct 10 '17 at 19:58
  • I can use other libraries without worry. Nevertheless, my problem resides in the date string. I simply can't get around it – Philipe Riskalla Leal Oct 10 '17 at 20:03

Without knowing the exact structure of the data, this is a guess.

import geopandas as gpd
import pandas as pd
import datetime

shp = gpd.read_file('myshp.shp')
#replace date column with a formatted date
#this assumes the format of the date is year/month/day
shp['date'] = pd.to_datetime(shp['date'], format='%Y/%m/%d')
#select rows from your data that are after January 1, 2015 and are 
#equal to specified sensor id
filtered = shp[(shp.issued > datetime.date(2015,1,1)) & 
    (shp['sensor'] == 'sensorid')]
  • Dear neighdough, thank you for your help, but the function doesn't work. It states: "invalid syntax". Here is how I have used the filtered function: "filtered = shp[(shp.issued == datetime.date(:,"01",:)) & (shp['sensor'] == 'TERRA_M-M')]" – Philipe Riskalla Leal Oct 10 '17 at 20:36
  • Check the parameters that you're using for datetime, they should be integer values indicating the date range that you're comparing. For example, if the date I'm trying to filter is January 21, 2015, I would enter datetime.date(2015,1,21). For reference, see docs.python.org/2/library/datetime.html#available-types. – neighdough Oct 10 '17 at 21:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.