Take the 2-minute tour ×
Geographic Information Systems Stack Exchange is a question and answer site for cartographers, geographers and GIS professionals. It's 100% free, no registration required.

With the introduction of the Data Access module in arcpy (30x faster search cursors), I want to know if counting features matching sql criteria is faster than the traditional MakeTableView + GetCount methodology?

share|improve this question
4  
How stupid is it that the feature count isn't just a property of an arcpy.Describe object –  Grant Humphries Sep 4 '13 at 20:24
    
This was pretty easy with ogrinfo with some OGR SQL. The dataset has something like 170000 records, and this wildcard search on an unindexed VARCHAR field came back in just a few seconds. ogrinfo "C:\xGIS\Vector\parcels\parcels_20140829_pmerc.ovf -sql "SELECT count(*) FROM parcels_20140829_pmerc WHERE tms like 'R39200-02-%'" –  elrobis Sep 26 at 13:54

1 Answer 1

up vote 24 down vote accepted

I am using an example with 1 million randomly generated points inside of a filegeodatabase. Attached here.

Here is some code to get us started:

import time
import arcpy

arcpy.env.workspace = "C:\CountTest.gdb"

time.sleep(5) # Let the cpu/ram calm before proceeding!

"""Method 1"""
StartTime = time.clock()
with arcpy.da.SearchCursor("RandomPoints", ["OBJECTID"]) as cursor:
    rows = {row[0] for row in cursor}

count = 0
for row in rows:
    count += 1

EndTime = time.clock()
print "Finished in %s seconds" % (EndTime - StartTime)
print "%s features" % count

time.sleep(5) # Let the cpu/ram calm before proceeding!

"""Method 2"""
StartTime2 = time.clock()
arcpy.MakeTableView_management("RandomPoints", "myTableView")
count = int(arcpy.GetCount_management("myTableView").getOutput(0))

EndTime2 = time.clock()
print "Finished in %s seconds" % (EndTime2 - StartTime2)
print "%s features" % count

And some initial results:

>>> 
Finished in 6.75540050237 seconds
1000000 features
Finished in 0.801474780332 seconds
1000000 features
>>> =============================== RESTART ===============================
>>> 
Finished in 6.56968596918 seconds
1000000 features
Finished in 0.812731769756 seconds
1000000 features
>>> =============================== RESTART ===============================
>>> 
Finished in 6.58207512487 seconds
1000000 features
Finished in 0.841122157314 seconds
1000000 features

Imagine larger, more complex datasets. The SearchCursor will indefinitely crawl.

I am not at all dissatisfied with the results, however, the DataAccess module is being used extensively in our GIS development circle. I am looking to rebuild some of our function definitions with this module as it is more flexible than a MakeTableView + GetCount methodology.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.