I'm executing a python script which does these steps:

  1. iterates through 19 features (makes pairs "each one with each other", order doesn't matter: A-B = B-A)
  2. Creates Closest Facility Layer
  3. Adds location points
  4. Solves Closest Facility route
  5. Exports into single .shp for each pair

I need to get a polyline for each of the pairs of 19 combinations. This will result (if I count correctly) in 19*18/2 = 174 shapefiles. When I run the script, it works fine until about 40th pair. Then it gets slower and slower. I let it run for about 3 hours and I got 142th pair...then ArcMap crashed with an error screen saying all my work is lost. If it ended alright I would then merge all the shapefiles into a single one with 174 features in it.

Is there any way to make the script more efficient or a way how to solve this problem?

This is the script:

 from collections import defaultdict
 import itertools
 import pprint

 import arcpy
 from arcpy import env
 import os

 ddict = defaultdict(list)
 for start,end in itertools.combinations(range(19), 2):

env.workspace = r"D:/Dokumenty/GISdata/PID"
env.overwriteOutput = True

inNetworkDataset = r"D:/Dokumenty/GISdata/PID/New/silnice_useky_krnap_ND.nd"
impedanceAttribute = "Length"
accumulateAttributeName = ["Length"]

for key,value in ddict.items():
    for v in value:
        inFacilities = r"PID/New/Obce_body_luzka" # melo by byt key
        inIncidents = r"PID/New/Obce_body_luzka" # melo by byt v
        outNALayerName = "Dvojice_bodu_{}_{}".format(key, v)
        outLayerFile = os.path.join(r"D:/Dokumenty/GISdata/output2", outNALayerName + ".lyr")
        NAResultObject = arcpy.na.MakeClosestFacilityLayer(inNetworkDataset,outNALayerName,impedanceAttribute,"TRAVEL_TO","",1, accumulateAttributeName,"NO_UTURNS")

        outNALayer = NAResultObject.getOutput(0)

        subLayerNames = arcpy.na.GetNAClassNames(outNALayer)
        #Stores the layer names that we will use later
        facilitiesLayerName = subLayerNames["Facilities"]
        incidentsLayerName = subLayerNames["Incidents"]

        import arcpy

        intable = "D:\Dokumenty\GISdata\PID\New\Obce_body_luzka.dbf"

        fields = arcpy.ListFields(intable)

        fieldinfo = arcpy.FieldInfo()

        for field in fields:
            fieldinfo.addField(field.name, field.name, "VISIBLE", "")
        arcpy.MakeTableView_management(intable, "temp_table", "FID = {}".format (key), "", fieldinfo)
        arcpy.MakeTableView_management(intable, "temp_table2", "FID = {}".format (v), "", fieldinfo)
        fieldMappings = arcpy.na.NAClassFieldMappings(outNALayer, facilitiesLayerName, False, fields) 
        arcpy.na.AddLocations(outNALayer, facilitiesLayerName, "temp_table", fieldMappings, "")
        arcpy.na.AddLocations(outNALayer, incidentsLayerName, "temp_table2", "", "")


        routesLayer = arcpy.mapping.ListLayers(outNALayer, "routes")[0]
        arcpy.FeatureClassToShapefile_conversion(routesLayer, r"D:/dokumenty/gisdata/output4")

print "Script completed successfully"

except Exception as e:

import traceback, sys
tb = sys.exc_info()[2]
print "An error occured on line %i" % tb.tb_lineno
print str(e)
  • 2
    If you want efficiency code in C# or VB.net for ArcObjects. You could consider 'pairing' out the processes by writing a batch file or python script then using subprocess.Popen create new threads, if one fails for any reason you've still got the instructions to retry. – Michael Stimson Nov 17 '17 at 2:05
  • 1
    What precisely is the wording of "an error screen saying all my work is lost"? – PolyGeo Nov 17 '17 at 2:10
  • Thank you for the reply. I'm sorry but I can't code in anything but python and only basics. In fact, my friend helped me to write this script. So I don't really understand what you did suggest, I apologize. – M. Tomas Nov 17 '17 at 2:11
  • 1
    The subprocess.Popen coding pattern that @MichaelStimson mentions is for Python and used at gis.stackexchange.com/a/231120/115 (and in other Q&As here and at Stack Overflow). I used it last week for the first time when tracing a Geometric Network repeatedly was bailing. – PolyGeo Nov 17 '17 at 2:19
  • 1
    A method I have used with great success is to create a script that works for one instance, remove the top few lines, save, then in the 'master' script write to a new python script the missing few lines (InFeatures = , OutFeatures = etc..) then copy in the remaining lines from the working script, close and Popen... with some careful looping I am able to keep the number of processes down to int(os.environ.get('NUMBER_OF_PROCESSORS')) - 1. That is fairly advanced concepts though, perhaps lean on your friend and ask nicely with a gift for these modifications. – Michael Stimson Nov 17 '17 at 2:46