10

I am trying to understand how I can use the csv module in python to open a csv file in the same folder as the python script, and then create a shapefile using the shapefile module pyshp.

The csv file looks like this, but can have a couple of thousand rows of records:

id_nr;date;target;start_lat;start_lon
1;2012-05-21;navpoint 25x;55.123654;13.456954
1;2012-05-23;navpoint 11f;55.143654;12.456954
14

The pyshp module is a bit tricky to get the hang of, but really useful once you get it going. I've written a script that reads in a csv of the example data and writes out a shapefile with the data stored as attributes of the correct datatypes. The pyshp/xbase datatyping has always been tricky for me until I found this user guide for the xbase format and as a result of this question I have written a small note on my blog regarding the relevant pyshp datatypes, part of which I have pasted below:

  • C is ASCII characters
  • N is a double precision integer limited to around 18 characters in length
  • D is for dates in the YYYYMMDD format, with no spaces or hyphens between the sections.
  • F is for floating point numbers with the same length limits as N
  • L is for logical data which is stored in the shapefile's attribute table as a short integer as a 1 (true) or a 0 (false). The values it can receive are 1, 0, y, n, Y, N, T, F or the python builtins True and False

The full listing is as follows:

import shapefile as shp
import csv

out_file = 'GPS_Pts.shp'

#Set up blank lists for data
x,y,id_no,date,target=[],[],[],[],[]

#read data from csv file and store in lists
with open('input.csv', 'rb') as csvfile:
    r = csv.reader(csvfile, delimiter=';')
    for i,row in enumerate(r):
        if i > 0: #skip header
            x.append(float(row[3]))
            y.append(float(row[4]))
            id_no.append(row[0])
            date.append(''.join(row[1].split('-')))#formats the date correctly
            target.append(row[2])

#Set up shapefile writer and create empty fields
w = shp.Writer(shp.POINT)
w.autoBalance = 1 #ensures gemoetry and attributes match
w.field('X','F',10,8)
w.field('Y','F',10,8)
w.field('Date','D')
w.field('Target','C',50)
w.field('ID','N')

#loop through the data and write the shapefile
for j,k in enumerate(x):
    w.point(k,y[j]) #write the geometry
    w.record(k,y[j],date[j], target[j], id_no[j]) #write the attributes

#Save shapefile
w.save(out_file)

I hope this helps.

  • Very nice script. I got an error as it didn't read it as text so I changed this line: with open('input.csv', 'rt') as csvfile: – againstflow Aug 29 '14 at 12:45
  • 1
    I think you can improve the performance by using next(r) before the for loop to skip the header instead of checking using an if statement. – dexgecko Nov 15 '17 at 18:18
  • @sgrieve - this script converts a csv with specific pre-determined fields. I'd like a generic script to convert any csv into a feature class. Perhaps there are useful arcpy functions to achieve this? – Waterman May 3 '18 at 0:23
2

As an alternative you do not need to hold the data in lists.

# import libraries
import shapefile, csv

# create a point shapefile
output_shp = shapefile.Writer(shapefile.POINT)
# for every record there must be a corresponding geometry.
output_shp.autoBalance = 1
# create the field names and data type for each.
# you can insert or omit lat-long here
output_shp('Date','D')
output_shp('Target','C',50)
output_shp('ID','N')
# count the features
counter = 1
# access the CSV file
with open('input.csv', 'rb') as csvfile:
    reader = csv.reader(csvfile, delimiter=',')
    # skip the header
    next(reader, None)
    #loop through each of the rows and assign the attributes to variables
    for row in reader:
        id= row[0]
        target= row[1]
        date = row[2]
        # create the point geometry
        output_shp.point(float(longitude),float(latitude))
        # add attribute data
        output_shp.record(id, target, date)
        print "Feature " + str(counter) + " added to Shapefile."
        counter = counter + 1
# save the Shapefile
output_shp.save("output.shp")

You can find a working example of this implementation here.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.