2

I have a point layer and a table with approximately 100 fields. I need to join the table to the point layer based on a key field and create a new shapefile / feature class.

This script has been painfully slow (40+ minutes for 1 layer w/100k records) to run. The run time for this part of the overall script makes it unstable due to memory issues. Is there a better way to do this? The frustrating thing is that the same function takes less than 2 minutes when performed through the ArcMap user interface.

    #Join CLS shp to off system table based on KEYFIELD1
    arcpy.AddMessage("running ELSE subscript for: "+fc_name)
    arcpy.FeatureClassToGeodatabase_conversion(CLS_file,output)
    arcpy.AddMessage("finish import cls to gdb")
    CLS_name = os.path.basename(CLS_file)
    CLS_filename = CLS_name.split('.')
    CLS_fc = output+"\\"+CLS_filename[0]
    CLS_fc_lyr = CLS_fc+"_lyr"
    input_dbf_view = input_dbf+"_view"
    arcpy.MakeFeatureLayer_management(CLS_fc,CLS_fc_lyr)
    arcpy.MakeTableView_management(input_dbf,input_dbf_view,"","","")
    arcpy.AddJoin_management(CLS_fc_lyr,"KEYFIELD_1",input_dbf_view,"KEYFIELD1","KEEP_ALL")
    input = fc_name+"_2013_join"
    arcpy.AddMessage("finish create view and join")

    #Report the % of records that failed to join + create new shp
    arcpy.CopyFeatures_management(CLS_fc_lyr,output+"\\"+input,"#","0","0","0")
    ##arcpy.TableSelect_analysis(output+input,output+input+"_NoCoords", "ADD SQL STATEMENT")
    ##error_cnt = int(arcpy.GetCount_management(output+fc_name+"_NoCoords").getOutput(0))
    ##total_cnt = int(arcpy.GetCount_management(output+fc_name).getOutput(0))
    ##print "The error % for "+fc_name+ " is "+str((float(error_cnt)/float(total_cnt))*100)+'%'
    arcpy.AddMessage("finish copy features")

    #Calculate Lat / Long Cooridnates        
    arcpy.AddXY_management(output+"\\"+input)
    arcpy.CalculateField_management(output+"\\"+input,"LONGITUDE","!POINT_X!","PYTHON")
    arcpy.CalculateField_management(output+"\\"+input,"LATITUDE","!POINT_Y!","PYTHON")
    arcpy.DeleteField_management(output+"\\"+input,["POINT_X","POINT_Y"])
    arcpy.AddMessage("finish add lat/long")
4

Have a search in help for the subject "Performance tips for joining data" it offers advice on improving join performance. Your code does not indicate you have added an attribute index which can often improve performance.

  • Adding an attribute index to the table and shapefile did the trick. It only takes a minute to process now. Thanks! – fathom analytics Oct 21 '14 at 19:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.