I am new to Python and have started to create scripts for ArcGIS workflows. I am wondering how I can speed up my code to generate an "Hours" double numeric field from a timestamp field. I start with a track point log (breadcrumb trail) shapefile generated by DNR Garmin, with an LTIME timestamp field (a text field, length 20) for when each trackpoint record was taken. The script calculates the difference in Hours between each successive timestamp ("LTIME"), and puts that into a new field ("Hours").

That way I can go back and sum up how much time I spent in a particular area/polygon. The main part is after the print "Executing getnextLTIME.py script..." Here is the code:

# ---------------------------------------------------------------------------
# Created on: Sept 9, 2010
# Created by: The Nature Conservancy
# Calculates delta time (hours) between successive rows based on timestamp field
# Credit should go to Richard Crissup, ESRI DTC, Washington DC for his
# 6-27-2008 date_diff.py posted as an ArcScript
    This script assumes the format "month/day/year hours:minutes:seconds".
    The hour needs to be in military time. 
    If you are using another format please alter the script accordingly. 
    I do a little checking to see if the input string is in the format
    "month/day/year hours:minutes:seconds" as this is a common date time
    format. Also the hours:minute:seconds is included, otherwise we could 
    be off by almost a day.

    I am not sure if the time functions do any conversion to GMT, 
    so if the times passed in are in another time zone than the computer
    running the script, you will need to pad the time given back in 
    seconds by the difference in time from where the computer is in relation
    to where they were collected.

# ---------------------------------------------------------------------------
import arcgisscripting, sys, os, re
import time, calendar, string, decimal
def func_check_format(time_string):
    if time_string.find("/") == -1:
        print "Error: time string doesn't contain any '/' expected format \
            is month/day/year hour:minutes:seconds"
    elif time_string.find(":") == -1:
        print "Error: time string doesn't contain any ':' expected format \
            is month/day/year hour:minutes:seconds"

        list = time_string.split()
        if (len(list)) <> 2:
            print "Error time string doesn't contain and date and time separated \
                by a space. Expected format is 'month/day/year hour:minutes:seconds'"

def func_parse_time(time_string):
    take the time value and make it into a tuple with 9 values
    example = "2004/03/01 23:50:00". If the date values don't look like this
    then the script will fail. 
    time_string = str(time_string)
    if not len(l) == 2:
        gp.AddError("Error: func_parse_time, expected 2 items in list l got" + \
            str(len(l)) + "time field value = " + time_string)
        raise Exception 
    if not len(cal) == 3:
        gp.AddError("Error: func_parse_time, expected 3 items in list cal got " + \
            str(len(cal)) + "time field value = " + time_string)
        raise Exception
    if not len(ti) == 3:
        gp.AddError("Error: func_parse_time, expected 3 items in list ti got " + \
            str(len(ti)) + "time field value = " + time_string)
        raise Exception
    if int(len(cal[0]))== 4:
    # formated tuple to match input for time functions
    return result


def func_time_diff(start_t,end_t):
    Take the two numbers that represent seconds
    since Jan 1 1970 and return the difference of
    those two numbers in hours. There are 3600 seconds
    in an hour. 60 secs * 60 min   '''

    start_secs = calendar.timegm(start_t)
    end_secs = calendar.timegm(end_t)

    x=abs(end_secs - start_secs)
    #diff = number hours difference
    #as ((x/60)/60)
    diff = float(x)/float(3600)   
    return diff


print "Executing getnextLTIME.py script..."

    gp = arcgisscripting.create(9.3)

    # set parameter to what user drags in
    fcdrag = gp.GetParameterAsText(0)
    psplit = os.path.split(fcdrag)

    folder = str(psplit[0]) #containing folder
    fc = str(psplit[1]) #feature class
    fullpath = str(fcdrag)

    gp.Workspace = folder

    fldA = gp.GetParameterAsText(1) # Timestamp field
    fldDiff = gp.GetParameterAsText(2) # Hours field

    # set the toolbox for adding the field to data managment
    gp.Toolbox = "management"
    # add the user named hours field to the feature class
    gp.addfield (fc,fldDiff,"double")
    #gp.addindex(fc,fldA,"indA","NON_UNIQUE", "ASCENDING")

    desc = gp.describe(fullpath)
    updateCursor = gp.UpdateCursor(fullpath, "", desc.SpatialReference, \
        fldA+"; "+ fldDiff, fldA)
    row = updateCursor.Next()
    count = 0
    oldtime = str(row.GetValue(fldA))
    #check datetime to see if parseable
    gp.addmessage("Calculating " + fldDiff + " field...")

    while row <> None:
        if count == 0:
            row.SetValue(fldDiff, 0)
            start_t = func_parse_time(oldtime)
            b = str(row.GetValue(fldA))
            end_t = func_parse_time(b)
            diff_hrs = func_time_diff(start_t, end_t)
            row.SetValue(fldDiff, diff_hrs)
            oldtime = b

        count += 1
        row = updateCursor.Next()

    gp.addmessage("Updated " +str(count+1)+ " rows.")
    del updateCursor
    del row

except Exception, ErrDesc:
    import traceback;traceback.print_exc()

print "Script complete."
  • 1
    nice program! I haven't seen anything to speed up calculation. The field calculator takes forever!!
    – Brad Nesom
    May 27, 2011 at 20:46

3 Answers 3


Cursor are always really slow in the geoprocessing environment. The easiest way around this is to pass a Python code block into the CalculateField geoprocessing tool.

Something like this should work:

import arcgisscripting
gp = arcgisscripting.create(9.3)

# Create a code block to be executed for each row in the table
# The code block is necessary for anything over a one-liner.
codeblock = """
import datetime
class CalcDiff(object):
    # Class attributes are static, that is, only one exists for all 
    # instances, kind of like a global variable for classes.
    Last = None
    def calcDiff(self,timestring):
        # parse the time string according to our format.
        t = datetime.datetime.strptime(timestring, '%m/%d/%Y %H:%M:%S')
        # return the difference from the last date/time
        if CalcDiff.Last:
            diff =  t - CalcDiff.Last
            diff = datetime.timedelta()
        CalcDiff.Last = t
        return float(diff.seconds)/3600.0

expression = """CalcDiff().calcDiff(!timelabel!)"""

gp.CalculateField_management(r'c:\workspace\test.gdb\test','timediff',expression,   "PYTHON", codeblock)

Obviously you'd have to modify it to take fields and such as parameters, but it should be pretty fast.

Note that although your date/time parsing functions are actually a hair faster than the strptime() function, the standard library is almost always more bug-free.

  • Thanks David. I didn't realize that the CalculateField was faster; I will try to test this. The only problem I think there may be is that the dataset may be out of order. On occasion, this happens. Is there a way to Sort Ascending on the LTIME field first and then apply the CalculateField, or to tell the CalculateField to execute in a certain order?
    – Russell
    May 28, 2011 at 2:33
  • Just a note, calling the pre-canned gp functions will be faster most of the time. I explained why in a previous post gis.stackexchange.com/questions/8186/… May 29, 2011 at 1:12
  • +1 for using the datetime built-in package, as it offers great functionality and almost replaces time/calendar packages
    – Mike T
    May 30, 2011 at 4:04
  • 1
    that was incredible! I tried your code, and integrated it with @OptimizePrime 's "in memory" suggestion and it took the script's average running time from 55 seconds to 2 seconds (810 records). This is exactly the kind of thing I was looking for. Thank you so much. I learned a lot.
    – Russell
    Jun 1, 2011 at 2:25

@David has given you a very clean solution. +1 for using the strengths of the arcgisscripting code base.

Another option, is to copy the dataset to memory using:

  • gp.CopyFeatureclass("path to your source", "in_memory\copied feature name") - for a Geodatabase Feature Class, shapefile or,
  • gp.CopyRows("path to your source", ) - for a Geodatabase table, dbf etc

This removes the overhead incurred when you request a cursor from the ESRI COM code base.

The overhead comes about from the conversion of python data types to C data types and access to the ESRI COM code base.

When you have your data in memory, you are reducing the need to access disk (a high cost process). Also, you reduce the need for the python and C/C++ libraries to transfer data, when you use arcgisscripting.

Hope this helps.


An excellent alternative to using an old style UpdateCursor from arcgisscripting, that has been available has been available since ArcGIS 10.1 for Desktop, is arcpy.da.UpdateCursor.

I have found that these are typically about 10 times faster.

These would/may not have been an option when this question was written but should not be overlooked by anyone reading this Q&A now.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.