Creating a value-counting table with search- and update cursor

Question

I'm a python beginner.

I`m working with ArcGIS 10.2.2 and IDLE Python 2.7.5

I have a shapefile with around 20 fields, each attribute has a value between -3 and 3.

This would look like this:

I need to count the number of occurrences of each value for every object. For this I created a new shapefile with fields where the values get counted:

The actual counting is my problem. My approach is to browse with search and update cursor through the files. My script does work, however there are two problems:

1.It is really slow (for a shapefile with 300 rows the program ran over 9 minutes). I tried using the arcpy.da.searchcursor but I couldn't bring it to work. 2. The way I built the script seems a little odd. I tried a lot to make it easier, but I cannot really figure out which way to go. I believe there is an easier way to do it.

try:
#getting the field names and creating a search cursor
    fieldnames = arcpy.ListFields("Eignung")
    srows=arcpy.SearchCursor("Eignung",fieldnames[2:18])


    print "operating"
#creating a while loop to work row by row
    srow = srows.next()
    while srow:
# for every field in the row I get the Value
        for fieldname in fieldnames[0:18]:
            TheFieldName=fieldname.name
            TheValue=srow.getValue(TheFieldName)

#if the Value is 0 I add "1" to the "zerofield" in the counting table 
            if TheValue == 0:
                obj_id = srow.getValue("OBJECTID")
                urowquery = '"OBJECTID" ='+ str(obj_id)
                urows = arcpy.UpdateCursor("Eignung_1",urowquery)
                for urow in urows:
                    urow.setValue("Null_",urow.getValue("Null_")+1)
                    urows.updateRow(urow)
                obj_id +=1
                del urow, urows
                print "check0"
# and do the same thing for 1 and 2 and 3 and so on
            elif TheValue == 1:
                obj_id = srow.getValue("OBJECTID")
                urowquery = '"OBJECTID" ='+ str(obj_id)
                urows = arcpy.UpdateCursor("Eignung_1",urowquery)
                for urow in urows:
                    urow.setValue("Eins",urow.getValue("Eins")+1)
                    urows.updateRow(urow)
                obj_id +=1
                del urow, urows
                print "check1"
            elif TheValue == 2:
                obj_id = srow.getValue("OBJECTID")
                urowquery = '"OBJECTID" ='+ str(obj_id)
                urows = arcpy.UpdateCursor("Eignung_1",urowquery)
                for urow in urows:
                    urow.setValue("Zwei",urow.getValue("Zwei")+1)
                    urows.updateRow(urow)
                obj_id +=1
                del urow, urows
                print "check2"

            elif TheValue == 3:
                obj_id = srow.getValue("OBJECTID")
                urowquery = '"OBJECTID" ='+ str(obj_id)
                urows = arcpy.UpdateCursor("Eignung_1",urowquery)
                for urow in urows:
                    urow.setValue("Drei",urow.getValue("Drei")+1)
                    urows.updateRow(urow)
                obj_id +=1
                del urow, urows
                print "check3"

            elif TheValue == -1:
                obj_id = srow.getValue("OBJECTID")
                urowquery = '"OBJECTID" ='+ str(obj_id)
                urows = arcpy.UpdateCursor("Eignung_1",urowquery)
                for urow in urows:
                    urow.setValue("Eins_minus",urow.getValue("Eins_minus")+1)
                    urows.updateRow(urow)
                obj_id +=1
                del urow, urows
                print "check-1"

            elif TheValue == -2:
                obj_id = srow.getValue("OBJECTID")
                urowquery = '"OBJECTID" ='+ str(obj_id)
                urows = arcpy.UpdateCursor("Eignung_1",urowquery)
                for urow in urows:
                    urow.setValue("Zwei_minus",urow.getValue("Zwei_minus")+1)
                    urows.updateRow(urow)
                obj_id +=1
                del urow, urows
                print "check-2"

            elif TheValue == -3:
                obj_id = srow.getValue("OBJECTID")
                urowquery = '"OBJECTID" ='+ str(obj_id)
                urows = arcpy.UpdateCursor("Eignung_1",urowquery)
                for urow in urows:
                    urow.setValue("Drei_minus",urow.getValue("Drei_minus")+1)
                    urows.updateRow(urow)
                obj_id +=1
                del urow, urows
                print "check-3"

        srow = srows.next()
    print done

As I said, it does work, but it doesn't work well.

Answer 1

I would re-engineer your code to use the da search and update cursors as these are significantly faster. Have a look at the help file and practise, it will be worth it in the long run.

A performance improving step is to search in a single sweep through the data, collating all your numbers then write this out into a single update step. Currently you are calling multiple updates on every step through the search cursor. This would be killing your performance.

So how do you keep a count whilst stepping through with a search cursor? Well I would suggest using dictionaries. These are in-memory data structures which are very fast to read/write from.

Another off the top of my head approach is to run the summary tool grouping by objectID and field and doing a count, then repeat for every field then do a big join and export, the sort of thing you can knock together in model builder but I would imagine python approach is quicker?

Answer 2

Thanks for the sugestions. I used da.searchcursor and da.update cursor and I avoided calling the updates in every step. My script is now less complex and performance is good.

import arcpy, sys, traceback
from arcpy import env

env.workspace = r"C:\GIS\data\KUP\Eingangsdaten.gdb"
#Need to be translated to data infrastructure via "arcpy.GetParameterAsText()"


inFC = "Eignung"
outpFC = "Eignung_1"


try:
 # creating update cursor for all fields I need
    urows = arcpy.da.UpdateCursor(outpFC,["Null_","Eins","Zwei","Drei","Drei_minus","Zwei_minus","Eins_minus"])
#creating a search cursor and a list including the tuples from 
# the search cursor)
    lstall = [srow[2:18] for srow in arcpy.da.SearchCursor(inFC,"*")]
    for lstrow in lstall:
  # using the count function for every value in every raw and writing it to the output feature class
        urow = urows.next()
        urow[0] = lstrow.count(0)
        urow[1] = lstrow.count(1)
        urow[2] = lstrow.count(2)
        urow[3] = lstrow.count(3)
        urow[4] = lstrow.count(-3)
        urow[5] = lstrow.count(-2)
        urow[6] = lstrow.count(-1)

        urows.updateRow(urow)

    del lstall, lstrow, urow, urows
    print "done"



except:
    print "error"