This is why I am using Pandas, to do a job quickly:
Output:
Script:
import arcpy
from arcpy import env
inTable=arcpy.GetParameterAsText(0)
caseField=arcpy.GetParameterAsText(1)
theFields=arcpy.GetParameterAsText(2).split(";")
outTable=arcpy.GetParameterAsText(3)
usePandas=arcpy.GetParameterAsText(4)
env.overwriteOutput = True
aTypes=["SUM"]
if usePandas=='false':
arcpy.AddMessage("Using ArcGIS")
if len(theFields[0]):
stats=[[field, "SUM"] for field in theFields]
arcpy.analysis.Statistics(inTable,outTable,stats,caseField)
else:
arcpy.analysis.Frequency(inTable,outTable,caseField)
else:
arcpy.AddMessage("Using Pandas")
fldSource=arcpy.ListFields(inTable)
sourceNames=[row.name for row in fldSource]
i=sourceNames.index(caseField)
fld=fldSource[i]
theType = fld.type
import pandas as pd
import string, random,os
csv=''.join(random.choice(string.ascii_uppercase + string.digits) for _ in range(10))
outCSV='F:/scratch/%s.csv'%csv
if len(theFields[0]):
tbl=arcpy.da.TableToNumPyArray(inTable,[caseField]+theFields)
df=pd.DataFrame(tbl);del tbl
df["Frequency"]=1
fList=["Frequency"]+theFields
else:
tbl=arcpy.da.TableToNumPyArray(inTable,caseField)
df=pd.DataFrame(tbl);del tbl
df["Frequency"]=1
fList=["Frequency"]
sums=df.groupby([caseField])[fList].sum()
del df
if theType=='String':
outINI='F:/scratch/schema.ini'
f = open(outINI, "w")
f.write('[%s.csv]\n' %csv)
f.write('Col1=%s Text' %caseField)
f.close()
arcpy.AddMessage(sums.head())
sums.to_csv (outCSV,header=True)
GDB,tbl=os.path.split(outTable)
arcpy.conversion.TableToTable(outCSV, GDB, tbl)
arcpy.management.Delete(outCSV)
The code could be much-much shorter, but ArcGIS has a mysterious ways of dealing with CSV files.
For a set of about 800Krecords, Pandas is 5.5 times faster, but slightly slower for tiny datasets.
field1,field2
first? In a PostgreSQL database, a GROUP BY sum query with an index on 7M rows would take just seconds. Then there's thememory
workspace option. Lots of ways to do this., even a DA SearchCursor with a dictionary.