Take the 2-minute tour ×
Geographic Information Systems Stack Exchange is a question and answer site for cartographers, geographers and GIS professionals. It's 100% free, no registration required.

I have a Python script that creates a point shapefile from a csv file within a Python script. This script gives me a ValueError for the longitude field (LON) if there isn't an extra nonsensical field to the right of it. Below is the error and how the csv file has been formatted to remove this error:

    lonValueIndex = valueList.index("LON")
    ValueError: list.index(x): x not in list

DATE       LAT      LON          TEST
2/19/2011  34.27531  -118.21071    a
2/19/2011  34.18069  -118.34079    b
2/19/2011  34.0215   -118.21857    c

Do you see that Test field with values of a, b, and c? If I don't put a field to the right of the LON field I get the ValueError. Below is part of my script. Does anyone know why that extra field is necessary? Thank you.

import arcpy, csv
arcpy.env.overwriteOutput = True

#Set variables
arcpy.env.workspace = "C:\\GIS\\StackEx\\"
outFolder = arcpy.env.workspace
pointFC = "art2.shp"
coordSys = "C:\\Program Files\\ArcGIS\\Desktop10.0\\Coordinate Systems" + \
           "\\Geographic Coordinate Systems\\World\\WGS 1984.prj"
csvFile = "C:\\GIS\\StackEx\\chicken.csv"
fieldName = "DATE1"

#Create shapefile and add field
arcpy.CreateFeatureclass_management(outFolder, pointFC, "POINT", "", "", "", coordSys)
arcpy.AddField_management(pointFC, fieldName, "TEXT","","", 10)

gpsTrack = open(csvFile, "r")

headerLine = gpsTrack.readline()
#print headerLine
#I updated valueList to remove the '\n'
valueList = headerLine.strip().split(",")
print valueList
latValueIndex = valueList.index("LAT")
lonValueIndex = valueList.index("LON")
dateValueIndex = valueList.index("DATE")

# Read each line in csv file
cursor = arcpy.InsertCursor(pointFC)
for point in gpsTrack.readlines():

   segmentedPoint = point.split(",")
   # Get the lat/lon values of the current reading                    
   latValue = segmentedPoint[latValueIndex]
   lonValue = segmentedPoint[lonValueIndex]
   dateValue = segmentedPoint[dateValueIndex]
   vertex = arcpy.CreateObject("Point")
   vertex.X = lonValue
   vertex.Y = latValue
   feature = cursor.newRow()
   feature.shape = vertex
   feature.DATE1 = dateValue
   cursor.insertRow(feature)

del cursor
share|improve this question
3  
Post the entire script for us to be able to give better answers. You might also consider using the csv module to read the file. –  blah238 Dec 6 '11 at 3:24
    
Hello, I added the entire script, as you can see I imported the csv module but I haven't used it, yet. Thank you. –  Patty Jula Dec 6 '11 at 4:04
    
Can you post a few lines from the CSV file including the header as plain text? Also the line gpsTrack = (csvFile, "r") appears to be an editing typo, as that would not have the desired result :) –  blah238 Dec 6 '11 at 4:48
    
Plain text of CSV file added. Thank you. –  Patty Jula Dec 6 '11 at 4:59
2  
Well, without having the file in hand, and based on your workaround of adding a dummy field onto the end, I suspect the culprit is a newline character stuck on the end of the last value in the list returned by split(). This is one of the things the csv module was designed to account for. So when you use index() to look for LON, it's not finding it because what you should really be looking for is LON\r\n or similar, which is of course unintuitive. –  blah238 Dec 6 '11 at 5:26
show 1 more comment

3 Answers

up vote 9 down vote accepted

When you split headerLine by the commas, the last entry "LON" contains a newline character:

print headerLine.split(",")
['DATE', 'LAT', 'LON\n']

This is why adding a new column after LON causes the script to work, since the newline character is applied to the TEST field.

Try using STRIP to remove the newline character:

print headerLine.strip().split(",")
['DATE', 'LAT', 'LON']

Steve

share|improve this answer
    
Exactly! 10char –  blah238 Dec 6 '11 at 5:34
1  
@blah238 I just learned something new (not about GIS - the 10char thing). Thanks! –  Stephen Lead Dec 6 '11 at 5:37
    
Ah, that was it. I changed the valueList definition to remove the '\n'. My script has been updated above. Thank you! –  Patty Jula Dec 6 '11 at 16:19
add comment

Here is an alternative workflow using geoprocessing tools that may be easier:

  1. Use Make XY Event Layer (Data Management) to create a temporary point layer from your CSV file

  2. Use Copy Features (Data Management) or Feature Class To Feature Class (Conversion) to make a permanent feature class or shapefile from the temporary point layer.

share|improve this answer
    
That looks like an excellent solution... probably a method I would switch to in the future. But I'm really after finding out why the extra field is necessary. –  Patty Jula Dec 6 '11 at 4:51
add comment

If you are willing to use OGR/GDAL, your problems can be solved really easily. If you can export your data to be comma separated (or tab separated), and put it into a file called test.csv. Then you can create an XML file that will define where the lat/long columns are to be read from:

<OGRVRTDataSource>
  <OGRVRTLayer name="test">
    <SrcDataSource>test.csv</SrcDataSource>
    <GeometryType>wkbPoint</GeometryType>
    <LayerSRS>WGS84</LayerSRS>
    <GeometryField encoding="PointFromColumns" x="LAT" y="LON"/>
  </OGRVRTLayer>
</OGRVRTDataSource>

Then, to get your shapefile, use ogr2ogr:

$ ogr2ogr -f "ESRI Shapefile"  /tmp/stuff.shp /tmp/test.vrt 
$ ogrinfo -al -so stuff.shp
INFO: Open of `stuff.shp'
using driver `ESRI Shapefile' successful.

Layer name: stuff
Geometry: Point
Feature Count: 3
Extent: (34.021500, -118.340790) - (34.275310, -118.210710)
Layer SRS WKT:
GEOGCS["GCS_WGS_1984",
   DATUM["WGS_1984",
     SPHEROID["WGS_1984",6378137,298.257223563]],
    PRIMEM["Greenwich",0],
UNIT["Degree",0.017453292519943295]]
DATE: String (80.0)
LAT: String (80.0)
LON: String (80.0)
TEST: String (80.0)

There's more documentation here.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.