Using Fiona to write a new shapefile from scratch

Question

I am new to Fiona, and trying to wrap my head around reading a csv file and writing out an ESRI shapefile. I have most of it down, but am missing two things; (1) defining the projection/spatial reference and (2) the attributes actually being populated with the values from the csv file.

So my questions are: 1. How do I define the spatial reference, either by defining an EPSG or a custom projection 2. Why is my data not writing as attributes?

Thanks in advance for any guidance.

import csv
from shapely.geometry import Point, mapping
from fiona import collection
from fiona.crs import from_epsg


schema = { 'geometry': 'Point', 
 'properties': {'dx': 'float:13.3',
            'dy': 'float:13.3',
            'dline': 'str',
            'dtrace': 'int',
            'ddepth': 'float:9.4',
            'dtrash': 'float:9.4',               
            'sx': 'float:13.3',
            'sy': 'float:13.3',
            'sline': 'str',
            'strace': 'int',
            'sdepth': 'float:9.4',
            'strash': 'float:9.4',                 
            'bx': 'float:13.3',
            'by': 'float:13.3',
            'bline': 'str',
            'btrace': 'int',
            'bdepth': 'float:9.4',
            'btrash': 'float:9.4',}}

with collection(
shapeout, "w", "ESRI Shapefile", schema) as output:
with open(finalout, 'rb') as f:
    reader = csv.DictReader(f)
    for row in reader:
        point = Point(float(row['dx']), float(row['dy']))
        output.write({
            'properties': {'dx': 'float:13.3',
            'dy': 'float:13.3',
            'dline': 'str',
            'dtrace': 'int',
            'ddepth': 'float:9.4',
            'dtrash': 'float:9.4',               
            'sx': 'float:13.3',
            'sy': 'float:13.3',
            'sline': 'str',
            'strace': 'int',
            'sdepth': 'float:9.4',
            'strash': 'float:9.4',                 
            'bx': 'float:13.3',
            'by': 'float:13.3',
            'bline': 'str',
            'btrace': 'int',
            'bdepth': 'float:9.4',
            'btrash': 'float:9.4',},
            'geometry': mapping(point)
        })

With the code updated to the "newer" (aka, not outdated) process:

with fiona.open(shapeout, 'w', crs=from_epsg(3996), driver='ESRI Shapefile', 
            schema=myschema) as output:
with open(finalout, 'rb') as f:
    reader = csv.DictReader(f)
    for row in reader:
        #geometry
        point = Point(float(row['dx']), float(row['dy']))
        #attributes
        prop = {'dx': float(row['dx']),
                'dy': float(row['dy']),
                'dline': str(row['dline']),
                'dtrace': float(row['dtrace']),                    
                'ddepth': float(row['ddepth']),                    
                'dtrash': float(row['dtrash'])}, 
        print{'geometry': mapping(point), 'properties':prop}

Gives me this:

{'geometry': {'type': 'Point', 'coordinates': (-245215.3333, -975226.3333)}, 'properties': ({'dtrace': 24387.0, 'ddepth': 1433.2458, 'dtrash': 1433.25, 'dx': -245215.3333, 'dy': -975226.3333, 'dline': '      lsl0904.201302'},)}
{'geometry': {'type': 'Point', 'coordinates': (-245215.0, -975251.0)}, 'properties': ({'dtrace': 24386.0, 'ddepth': 1432.7268, 'dtrash': 1432.73, 'dx': -245215.0, 'dy': -975251.0, 'dline': '      lsl0904.201302'},)}

But as soon as I change to:

output.write({'geometry': mapping(point), 'properties':prop})

I get the error message: AttributeError: 'tuple' object has no attribute 'keys'

Answer 1

Your solution is a little out of date (look at Fiona - Preffered method for defining a schema). For a better solution look at that proposed by Sean Gillies in gistfile1.py to parse a delimited text file data and create a new shapefile

1) How do I define the spatial reference: Fiona crs Module

from fiona.crs import from_epsg
from_epsg(3857)
{'init': 'epsg:3857', 'no_defs': True}

So, to write a new shapefile (not need of collection):

with fiona.open('your_shapefile.shp', 'w',crs=from_epsg(3857),driver='ESRI Shapefile', schema=yourschema) as output:

C2) Why is my data not writing as attributes?

In you script, you are confusing the schema of the shapefile and the attributes:

• Definition of the geometry type and the fields of a shapefile = schema of the layer

 yourschema =  {'geometry': 'Point',
               'properties': {'bx': 'float:13.3',
                'by': 'float:13.3',
                'bline': 'str',
                'btrace': 'int',
                'bdepth': 'float:9.4',
                'btrash': 'float:9.4'}}

• values from the dx field in the table:

for row in reader:
     # the value is
        dx = float(row['dx']
        ....

You script becomes: (you must use the mappingfunction of shapely and not shape (opposite)):

with fiona.open(shapeout, 'w',crs=from_epsg(3857),driver='ESRI Shapefile', schema=yourschema) as output:
    reader = csv.DictReader(finalout)
    for row in reader:
          # geometry
          point = Point(float(row['dx']), float(row['dy']))
          # attributes
          prop = {'dx': float(row['dx']),'dy': float(row['dy']),....,'btrash': float(row['btrash')}
          # write the row (geometry + attributes in GeoJSON format)
          output.write({'geometry': mapping(point), 'properties':prop})

explication of shapeand mapping:

shape(point)
<shapely.geometry.point.Point object at 0x105ebb450>
# GeoJSON format
mapping(point)
{'type': 'Point', 'coordinates': (..., ...)}

New

You can test the solution: I use here one of my files as an example:

from csv import DictReader
reader = DictReader(open("test.csv"), delimiter=',')
elem = reader.next()
print elem.keys()
['Y', 'X', 'Type', 'dip', 'dip_direct']
print elem.values()
['93729.059941647559754', '255598.788501376315253', 'N', '37', '340']

So a simple schema is (without OrderedDict):

schema={
  'geometry': 'Point', 
  'properties': {
      'X':'float',
      'Y': 'float',
      'Type': 'str:10', 
      'dip': 'int', 
      'dip_direct': 'int'}}

and for testing only the GeoJSON result (without creating the shapefile):

from shapely.geometry import Point, mapping
reader = DictReader(open("test.csv"), delimiter=',')
for row in reader:
     point = Point(float(row['X']), float(row['Y']))
     prop = {'X': float(row['X']), 'Y' :float(row['Y']),'Type':row['Type'],'dip':int(row['dip']),'dip_direct':int(row['dip_direct'])}
     print {'geometry':mapping(point),'properties': prop}
 {'geometry': {'type': 'Point', 'coordinates': (255598.78850137632, 93729.05994164756)}, 'properties': {'Y': 93729.05994164756, 'X': 255598.78850137632, 'Type': 'N', 'dip': 37, 'dip_direct': 340}}
 {'geometry': {'type': 'Point', 'coordinates': (255493.10748570191, 93839.054468165821)}, 'properties': {'Y': 93839.054468165821, 'X': 255493.10748570191, 'Type': 'N', 'dip': 20, 'dip_direct': 330}}

New

First, I don't know if that was a mistake but the indentation is not correct:

with fiona.open(shapeout...:
    with open(finalout, 'rb') as f:
       reader = csv.DictReader(f)

If you use csv.DictReader, you don't need with open(finalout, 'rb') as f:.

 with fiona.open(shapeout...:
    reader = csv.DictReader(finalout)

With my example:

with fiona.open('my_shapefile.shp', 'w',crs=from_epsg(3857),driver='ESRI Shapefile', schema=schema) as output:
    reader = DictReader(open("test.csv"), delimiter=',')
    for row in reader:
         point = Point(float(row['X']), float(row['Y']))
         prop = {'X': float(row['X']), 'Y' :float(row['Y']),'Type':row['Type'],'dip':int(row['dip']),'dip_direct':int(row['dip_direct'])}
         output.write({'geometry':mapping(point),'properties': prop})

Test:

c = fiona.open('my_shapefile.shp')
# schema of the shapefile
c.schema
{'geometry': 'Point', 'properties': OrderedDict([(u'Y', 'float'), (u'X', 'float'), (u'Type', 'str:10'), (u'dip', 'int:10'), (u'dip_direct', 'int:10')])}
# crs of the shapefile
c.crs
{u'a': 6378137, u'lon_0': 0, u'no_defs': True, u'y_0': 0, u'b': 6378137, u'proj': u'merc', u'x_0': 0, u'units': u'm', u'lat_ts': 0}
# first row of the shapefile
c.next()
{'geometry': {'type': 'Point', 'coordinates': (255598.78850137632, 93729.05994164756)}, 'type': 'Feature', 'id': '0', 'properties': OrderedDict([(u'Y', 93729.05994164756), (u'X', 255598.78850137632), (u'Type', u'N'), (u'dip', 37), (u'dip_direct', 340)])}