Why is ArcPy script slow?

Question

I have a simple arcpy script to update a field in a point shapefile with info from the polygon feature that it is within. It takes 9 minutes to do 100 points in arcpy but a spatial join in arcmap is instantaneous. I'm sure there is a quick established way to solve this problem. Can someone point me in the right direction?

import took 0:00:07.085000
extent took 0:00:05.991000
one pt loop took 0:00:03.780000
one pt loop took 0:00:03.850000
one pt loop took 0:00:03.791000


import datetime
t1 = datetime.datetime.now()
import arcpy
t2 = datetime.datetime.now()
print "import took %s" %  ( t2-t1)
#set up environment
arcpy.env.workspace = "data\\"
arcpy.env.overwriteOutput = True

desc = arcpy.Describe("parcels.shp")
ext = desc.Extent
extent = (ext.XMin,ext.XMax,ext.YMin,ext.YMax)
t3 = datetime.datetime.now()
print "extent took %s" %  (t3 -t2)
fc = arcpy.CreateRandomPoints_management("", "malls.shp", "", ext, 100, "", "POINT", "")
arcpy.AddField_management("malls.shp", 'ParcelID', 'LONG')

rows = arcpy.UpdateCursor('malls.shp',"","",'ParcelID')
for row in rows:
    t4 = datetime.datetime.now()
    pt = row.Shape.getPart()
    for polyrow in arcpy.SearchCursor('parcels.shp'):
        t6 = datetime.datetime.now()
        poly = polyrow.getValue('Shape')
        if extent[0]<pt.X<extent[1] and extent[2]<pt.Y<extent[3]:
            if poly.contains(pt):
                print "works"
                row.ParcelID = polyrow.Parcels_ID
                rows.updateRow(row)
                break #we can stop looking for matches since
        t7 = datetime.datetime.now()
        "a full poly loop took %s" % (t7-t6)
    t5 = datetime.datetime.now()
    print "one pt loop took %s" % (t5-t4)


print datetime.datetime.now() -t1

Answer 1

If you need to create a second cursor for parcels.shp, do so outside of the loop for your first cursor. As it stands, your script is creating a new cursor object for each row in malls.shp which is what's costing you all that processing time.

...
rows = arcpy.UpdateCursor('malls.shp',"","",'ParcelID')
polyrows = arcpy.SearchCursor('parcels.shp')
for row in rows:
    t4 = datetime.datetime.now()
    pt = row.Shape.getPart()
    for polyrow in polyrows:
...

Answer 2

The problem with @Jason's answer (and your original approach) is that it does not take advantage of the spatial index and requires a nested, two-cursor loop which is going to become exponentially slower as the number of points increases.

An alternative workflow that may be faster while still letting you update the point feature class in-place (Spatial Join only outputs a new feature class, not updates an existing one) might be to:

1. Use Spatial Join to create an intermediate (perhaps in-memory) feature class
2. Use Add Join to join the intermediate feature class to your existing point feature class
3. Use Calculate Field or an UpdateCursor to copy over the values in the joined field to the field in the existing point feature class.