I am attempting to convert 15 feature classes containing roughly 70,000 contours each into polygons record by record. Using my current code, each feature class takes roughly 16 hours on my current computer using a Python script (i.e. not executed from within ArcGIS, only using the ArcPy library). My computer has a 3.2 GHz W3670 processer, 12 GB RAM, and an SSD. So, I think it should be able to process the data faster.
I have had no exposure to multiprocessing until I started looking around yesterday trying to find a method to speed up this process. I found several articles (see ESRI blog post and user post; earlier posts on GIS stack exchange 1 and 2)
Using the above examples, I attempted to mash-up what I saw there with my existing code and came up with:
import multiprocessing
import arcpy
arcpy.env.overwriteOutput = True
def contour2polygon(oid):
contour = arcpy.management.MakeFeatureLayer('[some_path]\\test1.gdb\\contours','layer{0}'.format(oid[0]),'"OBJECTID" = {}'.format(oid[0]))
out_name = "[some_path]\\test2.gdb\\feat{}".format(str(oid[0]))
polygons = arcpy.FeatureToPolygon_management(contour, memory_name)
return polygons
def main():
contours = '[some_path]\\test1.gdb\\contours'
oids = [row[0] for row in arcpy.da.SearchCursor(contours, 'OBJECTID')]
cpuNum = multiprocessing.cpu_count() - 1
pool = multiprocessing.Pool(processes = cpuNum)
polygons = pool.map(contour2polygon, oids)
pool.close()
pool.join()
if __name__ == '__main__':
main()
However, when I execute the script, I just see a blank box and my Task Manager does not indicate any change in activity. Is there something my code is missing? I understand from the above, linked examples that I should have a worker function, but I get lost in the rest of the details especially with how I could convert my old scripts to using multiprocessing.