3

There are three methods to convert the output of a Make XY Event Layer to a Feature Class:

  • Copy Features
  • Feature to Point
  • Feature Class to Feature Class

Which one is the fastest method? I need to convert a huge amount of points stored in a csv table (1,5 GB file) and I wonder what's the best way to do it.

I will use Background geoprocessing on a 64-bit machine.

  • Background geoprocessing is slower. – GISI Aug 28 '15 at 9:50
  • Really? Recently I've run into problems with foreground processing (out of memory). I understood Background processing (64-bit) is the right solution when processing huge amount of data. – Manuel Frias Aug 28 '15 at 11:05
  • 2
    Hi Manuel. If you have a CSV table with LatLon coordinates and want to create a new feature class out of it - and know a little Python - the quickest way would be to use the csv module to read each line, then use an insert cursor to write the geometries to a blank feature class. I've used this technique and it's much faster than any ArcGIS out-of-the-box geoprocessing tool, especially for huge CSV files. – John Aug 28 '15 at 18:32
  • I explain the same in chat . Your csv is reconize as table by arcgis. You can use a couple of da.searchCursor and da.insertCursor – GeoStoneMarten Nov 27 '15 at 14:06
2

Here's an example of technique I mentioned in the my comment to your question. Reading a CSV and populating a feature class with its values can be easy. The script assumes the Latitude field is the 1st column and Longitude field is the 2nd column. You can tweak the coordinate system in the code to something other than WGS84, tweak the field types, etc.

# Author: John K. Tran
# Contact: jtran20@masonlive.gmu.edu

import arcpy
import os
import csv
print "arcpy imported"

"""
Input CSV can look like:

Lat,Lon,First Name,Last Name
12.34,56.78,Joe,Smith
0.98,7.65,Jane,Doe
65.43,43.21,Bob,Sagat

Make sure 'Lat' and 'Lon' fields are the 1st and 2nd column in the CSV respectively.

"""

incsv = r"C:\Users\MyName\Desktop\My Docs\Sample Data\SampleCSV.csv" # Change this to the path of your CSV file.
outfc = r"C:\Users\MyName\Desktop\My Docs\Sample Data\SampleGDB.gdb\SampleFC" # Change this to the path of your output FC.

spatialref = arcpy.SpatialReference(4326) # Create the spatial reference object as WGS84. Can modify if desired.

if not arcpy.Exists(outfc): # Create the output feature class if needed.
    arcpy.CreateFeatureclass_management(os.path.dirname(outfc), os.path.basename(outfc), "POINT", None, None, None, spatialref)

csv.register_dialect("xls", delimiter=",", lineterminator="\n") # Register the dialect for native CSV syntax in Microsoft Excel.
f = open(incsv, "r") 
reader = csv.reader(f, dialect = "xls")

headers = reader.next() # Read the first line as the header names.

for header in headers[2:]: # Add fields for remaining columns if needed. Default is TEXT field.
    arcpy.AddField_management(outfc, header, "TEXT")

cursor = arcpy.da.InsertCursor(outfc, ['SHAPE@XY'] + headers[2:]) # Create InsertCursor.
count = 0
for row in reader:
    if count % 1000 == 0:
        print "processing row {0}".format(count)
    Ycoord = row[0] # Make sure 'Lat' is in the 1st column.
    Xcoord = row[1] # Make sure 'Lon' is in the 2nd column.
    newrow = [(float(Xcoord), float(Ycoord))] + row[2:]
    cursor.insertRow(newrow) # Insert point in FC for each row in CSV.
    count += 1

del cursor
f.close()

And example of a CSV that can be read is:

Lat,Lon,FirstName,LastName,Tone,Magnitude
26.61,-81.49,John,Smith,Dark,6.0
26.68,-81.47,Jim,Jones,Light,9.0
20.01,-155.87,Steve,Bruin,Medium,3.9
25.43,-80.45,Bob,Johnson,Dark,2.75
26.05,-80.11,Bill,Mayer,Light,26.6

100,000 CSV records took about 3-5 seconds for me to run.

  • 1
    Absolutely amazing. 18 minutes to process 1,7 GB csv file. It took hours with the ArcGIS tools. Thanks for sharing this gem! – Manuel Frias Aug 29 '15 at 13:08
  • @John yeah today i can see original script and it create all fields in text... Perfect and sure it work. Look at based question – GeoStoneMarten Nov 27 '15 at 14:20
3

I've found a model builder model to be fastest with: "Make X Y Event Layer" followed by "Feature class to Feature Class"

model example

  • 2
    BTW: I've seen more than a few issues with "make x y event layer" over the years, if you have issues export the csv to a geodatabase table (as an intermediate) then use that as the input. – GISI Aug 28 '15 at 11:23
  • if you have time could you visit this question? gis.stackexchange.com/questions/283659/… – Waterman May 23 '18 at 21:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.