I have data from 2007 to 2015 of various animals that I'm trying to break up by year and name. I wrote a little for loop that goes through all the items in my attribute table (gps_points).

The problem is, it will search for items that don't exist. For example lets say I have animal1 for the years 2007 and 2008 only. The script will make a shapefile for animal1 for year 2007, then 2008, then it will run for 2009 where there is no data and crash.

I want to find a way to skip these periods of no data so after 2008, the code will skip 2009, 2010, ect then move onto the next animal. Is that possible? I can't seem to wrap my head around how an if statement would work since I just want to know if the SQL function will return a result or not before running the select line or skipping. A mixture of using searchcursor and boolean?

for byear in xrange(2007:2016): 
    for bname in animal_names: 
       sav_loc = "C:\\Desktop\\temp.gdb\\GB_"+str(bname)+"_"+str(byear) 
       arcpy.Select_analysis(gps_points, sav_loc, "name={} AND loc_year={} AND Age >= 5".format(str(bname),str(byear)) 

I'm using Python 2.7 and ArcGIS 10.3.1

extra code:

def unique_values(table, field):
    data = arcpy.da.TableToNumPyArray(table, [field])
    return numpy.unique(data[field])

animal_names = unique_values(gps_points,"name")
  • Please mark as answered – Ben S Nadler May 1 '16 at 19:17

Look at creating a set of valid animal/year values . feed that into the loop. Using arcpy, run frequency analysis on the data, using the two fields as the frequency fields. The resulting rows will be the valid combinations. With a cursor on the table, read the animal/year into your query.

freqFields = ['ANIMAL', 'YEAR']
freq = arcpy.Frequency_analysis(data, frequency,freqFields)
with arcpy.da.SearchCursor(frequency,freqFields) as cur:
    expression =  "name={} AND loc_year={} AND Age >= 5".format(cur[0],cur[1])
    #This expression is valid since the combination of variables came directly from the data

    sav_loc = "C:\\Desktop\\temp.gdb\\GB_"+cur[0]+"_"+str(cur[1]) 
    arcpy.Select_analysis(gps_points, sav_loc, expression)

if you do not have an advanced license you can create your own frequency (You already had this in your code). This ends up being faster than the Frequency tool

vals = [val for val in arcpy.da.TableToNumPyArray(data, freqFields).tolist()]
#Remove duplicates
uniqueVals = set(vals)

Procedurally you have a couple options:

use a try / except block to catch the error and ignore, so when you encounter the error it will still function.

    arcpy.Select_analysis(gps_points, sav_loc, expression) 

Depending on when in the process you encounter the failure and how many failures you have, this could be no big deal or a huge waste of time.

Or, before running the select analysis, make feature layer with the same query and check if any rows are returned

layer = arcpy.MakeFeatureLayer_management(data,r'in_memory\layer',expression)
if int(arcpy.GetCount_management(layer).getOutput(0)) >0:
    Do things

This check is expensive, the first two option's are probably quicker.

  • That's it, use getCount to decide if there will be an output before exporting. Beware that getCount doesn't return a number, have a read of the docs on how to get a number from this method resources.arcgis.com/en/help/main/10.1/index.html#//… I wouldn't consider this method as being 'expensive', I would think it would be far slower to cursor through all the records building a list . – Michael Stimson Apr 28 '16 at 1:57
  • Try it and find out. Frequency on the data is much faster. You only have to do it once. To get count, you first need a layer, so that adds 2 steps per query. It IS expensive when you consider counting thousands or millions of points just to verify you have >0 – Ben S Nadler Apr 28 '16 at 2:07
  • I need to step away from my computer screen but that sounds like a really cool idea. – JDEMAN Apr 28 '16 at 2:12
  • Absolutely Ben, Frequency (or even better Summary Statistics) is much faster than cursoring in arcpy, it just depends on how much the summary is condensed by the summarizing... i.e. how many rows need to be stepped through. In the end there's not much real time in it and I'd (personally) go with the method that uses the least lines of code, it is after all arcpy and not ArcObjects so there's not a real expectation of lightning speed. – Michael Stimson Apr 28 '16 at 2:43
  • There were only 2 fields of interest: Animal and year. I ran a test on a table with 10,000 rows and timed execution to either create or validate the case. Using layer+getCount took 6.0 seconds, Pre-Processing first with a frequency table took 2.2 seconds. A third test for generating the frequency without Frequency Analysis tool (which requires Advanced License) took 0.25 seconds. – Ben S Nadler May 1 '16 at 18:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.