Convert multiple CSV files to shapefiles using Python?

Question

At my internship, I have been asked to learn Python on the job. It's been a rough road never having coded before so please forgive me if my problem is elementary.

The Lightning CSV files I am trying to convert have over 400,000 rows.
The headers go as follows: mon, day, year, hr, min, sec, lat, lon, ht, type, Ip, Id

I found the answers to this question as potential solutions. I struggle to modify this code in order to convert the CSVs into shapefiles.

The path for the CSVs is C:\Users\zherran\Desktop\shp\enwest2015*.csv

The path where I want the shapefile C:\Users\zherran\Desktop\shp\shapefiles

import shapefile as shp
import csv

out_file = 'enwest3days.shp'

#Set up blank lists for data
mon,day,year,hr,min,sec,lat,lon,ht,type,Ip,Id=[],[],[],[],[],[],[],[],[],[],[],[]

# sample row mon,day,year,hr,min,sec,lat,lon,ht,type,Ip,Id
#            6,2,2015,7,27,16.6,41.5,-101.4,13789.5,1,-7900,26abb8a


#read data from csv file and store in lists
with open('enwest*.csv', 'rb') as csvfile:
    r = csv.reader(csvfile, delimiter=';')
    for i,row in enumerate(r):
        if i > 0: #skip header
            mon.append(int(row[0]))
            day.append(int(row[1]))
            year.append(int(row[2]))
            hr.append(int(row[3]))
            min.append(int(row[4]))
            sec.append(float(row[5]))
            lat.append(float(row[6]))
            lon.append(float(row[7]))
            ht.append(float(row[8]))
            type.append(int(row[9]))
            Ip.append(float(row[10]))
            Id.append(complex(row[11]))

#Set up shapefile writer and create empty fields
w = shp.Writer(shp.POINT)
w.autoBalance = 1 #ensures gemoetry and attributes match
w.field('X','F',10,8)
w.field('Y','F',10,8)
w.field('Date','D')
w.field('Target','C',50)
w.field('ID','N')
w.field('ID','N')
w.field('ID','N')
w.field('ID','N')
w.field('ID','N')
w.field('ID','N')
w.field('ID','N')

#loop through the data and write the shapefile
for j,k in enumerate(x):
    w.point(k,y[j]) #write the geometry
    w.record(k,y[j],date[j], target[j], id_no[j]) #write the attributes

#Save shapefile
w.save(out_file)

Answer 1

a) With the solution you use (Pyshp (shapefile), you need

1) to extract the fields of the csv file
2) to define the fields of the shapefile (w.field('Target','C',50))
3) to construct the geometry from the lon et lat fields ( w.point(float(i['x']),float(i['y'])) (see CSV to Shapefile )

b) With Fiona, it is easier but you have the same problem of field definition.
c) osgeo/ogr complicates the things...
d) Therefore, be modern, to convert directly csv files to shapefiles, the best solution is now Pandas, GeoPandas(uses Fiona) and Shapely. You do not have to worry about the fields and the fields definitions (pandas.DataFrame.from_csv).

from pandas import DataFrame
from geopandas import GeoDataFrame
from shapely.geometry import Point
# convert the csv file to a DataFrame
data = DataFrame.from_csv('enwest2015_1.csv', index_col=False)
# extract the geometry from the DataFrame
points = [Point(row['lon'], row['lat']) for key, row in data.iterrows()]
#convert the DataFrame to a GeoDataFrame 
geo_df = GeoDataFrame(data,geometry=points)
# save the resulting shapefile
geo_df.to_file('enwest2015_1.shp', driver='ESRI Shapefile') 

For the paths, it is a pure Python problem