I am trying to convert ~270k csv-files with lat/long-data to layerfiles. The files are split into folders by year, i.e. ~30 folders with ~9k csv-files in each.
I have written a script to iterate through the files and it works well for test-samples of 100+ split into a couple of different folders. However, when I try to run it on the full set of files, it will either:
work well for the first folder (~9k files) and then when moving on to the second folder it will give the following error for all subsequent files:
arcgisscripting.ExecuteError: ERROR 000223: Cannot extract event table properties
Failed to execute (MakeXYEventLayer).on my other computer it will usually stop after an arbitrary amount of file-conversions before it finishes the first folder (typically at +-8.5k of the conversions) and then give the same error for all subsequent files
The error seems to be similar to this: http://forums.arcgis.com/threads/80161-Make-XY-Event-Layer-Error ,but my error pops up after an arbitrary amount of files has been processed.
I've included my code for reference below. Note that I have also tried to split the list of files per folder into chunks of 100 and defined the layercreation as a function that is being called for each chunk, but the error still persisted in the same way. I suspect that it might be some setting in ArcGIS or on my computers that causes it, but I don't know what. Does anybody have any idea where I might start to solve this?
#setting paths for the project
path="C:/project"
savepath = path + "/outputlayers"
csvpath = path+ "/csv"
#lists the directories in the csv directory
yearlist = os.listdir(csvpath)
#checks the directory of each year:
for year in yearlist:
indir = csvpath + "/" + year
hourlist = os.listdir(indir)
#runs through each file and converts it to a .lyr file in the outputdirectory
for infile in hourlist:
try:
# Set the local variables
in_Table = indir + "/" + infile
x_coords = "longitudex"
y_coords = "latitudey"
z_coords = ""
out_Layer = infile[:-4]
saved_Layer = savepath + "/" + year + "/" + infile[:-4] +'.lyr'
# Set the spatial reference
spRef = arcpy.SpatialReference(4301)
# Make the XY event layer...
arcpy.MakeXYEventLayer_management(in_Table, x_coords, y_coords, out_Layer, spRef, z_coords)
# Print the total rows
print arcpy.GetCount_management(out_Layer)
# Save to a layer file
arcpy.SaveToLayerFile_management(out_Layer, saved_Layer)
except:
# If an error occurred print the message to the screen
print arcpy.GetMessages()