I'm trying to cut down on arcpy processing time by using the python multiprocessing toolbox. Using the example from http://wltrimbl.github.io/2014-06-10-spelman/intermediate/python/04-multiprocessing.html as a guideline.
Basically, I'm trying to select all tracks (point features) with a specific trackID (in field 'track'), give it a buffer and extract all road features within that buffer as preprocessing for another script. But the data set is so large it would take me about a week to run it.
I now have the following code, but I'm having a problem with the program executing normally, but not outputting the files in Tracks.gdb. Does anyone have advice?
import arcpy from arcpy import env import multiprocessing def MyFunction(i): TrackFeatureOut = "C:\tryingOut\Tracks.gdb\Track_" + str(i) RoadFeatureOut = "C:\tryingOut\Tracks.gdb\Road_" + str(i) if not (RoadFeatureOut in featureclasses): BufferFeatureOut = "C:\tryingOut\Temporary.gdb\Buffer_" + str(i) Query = "track =" + str(i) arcpy.Select_analysis("C:\tryingOut\Input.gdb\Tracks", TrackFeatureOut, Query) arcpy.Buffer_analysis(TrackFeatureOut, BufferFeatureOut, "100 Meters") arcpy.Clip_analysis("C:\tryingOut\Input.gdb\Roads", BufferFeatureOut, RoadFeatureOut) if __name__ == '__main__': global featureclasses env.workspace = "C:\tryingOut\Tracks.gdb" featureclasses = arcpy.ListFeatureClasses() # Getting all unique track numbers Track = set() cursor = arcpy.da.SearchCursor("C:\tryingOut\Input.gdb\Tracks",["track"],sql_clause=(None, 'ORDER BY track' )) for row in cursor: Track.add(row) # Trying to apply multiprocessing pool = multiprocessing.Pool(multiprocessing.cpu_count()) for i in Track: if Counter % 1000 == 0: print "Counter = ", Counter, "Current Track = ",i Counter += 1 pool.apply_async(MyFunction, i)