Deleting cursor used in SearchCursor within dictionary comprehension?

Question

If it is best to open cursors using a with statement to ensure it is deleted, like so:

with arcpy.da.UpdateCursor(fc,fields) as cursor:

Then, if a cursor is used as the iterable in a comprehension like so:

d = {k:v for (k,v) in arcpy.da.SearchCursor(fc,fields)}

Is it necessary to delete the cursor after using it in the comprehension?

Answer 1

Whether it's absolutely necessary is the wrong question to ask. The question is whether it's a good idea.

As a rule in programming, you should avoid doing weird things and use the best tool for the job. If something has an explicit way of releasing resources, just make the release explicit and be done with it:

with arcpy.da.UpdateCursor(fc,fields) as cursor:
    d = {k: v for (k,v) in cursor}

What you may not be aware of is that the with clause actually invokes additional logic. A with clause requires a context manager, which must have an __enter__ (invoked when the block is entered) and __exit__ (invoked when the block is exited) method. In particular, the __exit__ method is invoked regardless of whether an exception occurred, ensuring the program always releases the resource even on error. This gives your code explicit documentation of when a resource is acquired and when it is released, and it ensures that a resource can be released as soon as possible.

By contrast, you can't actually depend on the runtime to magically close it immediately for you. This is because the way it gets closed is by invoking the object's destructor, which may or may not happen immediately. Python does not make any guarantees about when a destructor is invoked, only that it will be eventually when the object is garbage collected. (See here.) Currently, Python is implemented so that it happens as soon as soon as there's no longer a reference to an object. But it's easy to accidentally propagate references to an object, and Python's runtime might change.

Also consider the long term maintenance. There's no long term reference to it now, but what happens in 6 months when you need to modify the code so that there is a reference? What if someone else does it? The person making the change may not think to switch to a with block since there's not one already there. Make cleaning up your resources a habit, and you'll have far fewer problems with it.

Do you really want to tie your code to implementation details of garbage collection? Do you want to have to constantly think about whether you might be accidentally propagating a reference via an exception? No, you don't. Imagine if that happened when the script was invoked in ArcMap. The user would be forced to close the entire process just to release the file. So don't put yourself in that position. Release the resource explicitly. Saving one line of code is not worth the risks of problems it can cause. Context managers are the standard mechanism for acquiring and releasing resources in Python, and they do it very well.

The bottom line is that not releasing it explicitly is a bad idea.

This, of course, assumes that the code has some possibility of affecting someone else, like putting it in a script that someone else will need to run or maintain or it might delay delivering your work if you have to close ArcMap all the way because you can't save your changes. If you're the only one who will be impacted by a problem, then by all means, fly in the face of good practices all you want.

Answer 2

No, it is not necessary to delete a cursor after using it in a comprehension. A cursor is an instance of a class, which is an object (everything in python is an object). Every python session has a namespace which contains references to all the objects in the session - think of it like a dictionary where the keys are references to each object, and the values are the objects themselves. When the 'reference count' - the number of keys that refer to that object - drops to zero, the object is removed and the memory re-allocated. When you use a cursor in a comprehension, there is no reference to that object in the namespace. After the comprehension is done, the object will be deleted.

There is no entry in the namespace, and therefore no need to delete anything. ESRI also illustrates this syntax in example 2, here.

To further clarify, if you run:

>>> import arcpy
>>> f = r'C:\Workspace\study_area.shp'
>>> a = arcpy.da.SearchCursor(f, ['*'])

You will see a .lock file appear in the directory (check your file explorer). The reference to the cursor is a, which will make the cursor (and therefore the lock) persist until a is deleted. So when you then run:

>>> del(a)

The entry in the namespace will be removed, and the lock will release (.lock file will disappear). If you run:

>>> t = [i for i in arcpy.da.SearchCursor(f, ['*'])]

You either won't see a lock file, or it will disappear when the command completes. Without an entry in the namespace, the cursor is not persistent. t refers to the list you just created, not the cursor used to create it.

To summarize, you need only worry about deleting cursors when they have a reference in the namespace (i.e. when you have assigned them to a variable, like a in the above example).

Answer 3

Update and insert cursors cannot be created for a table or feature class if an exclusive lock exists for that dataset. The UpdateCursor or InsertCursor functions fail because of an exclusive lock on the dataset. If these functions successfully create a cursor, they apply an exclusive lock on the dataset so that two scripts cannot create an update or insert cursor on the same dataset.

In Python, the lock persists until the cursor is released. Otherwise, all other applications or scripts could be unnecessarily prevented from accessing a dataset. A cursor can released by one of the following:

Including the cursor inside a with statement, which will guarantee the release of locks regardless of whether or not the cursor is successfully completed;

Calling reset() on the cursor;

The completion of the cursor;

Explicitly deleting the cursor using Python's del statement - ESRI

Locking with arcpy.da cursors is pretty much the same as locking with the original arcpy cursors.

After testing your code, and as gberard pointed out, there is no reference to the cursor after the comprehension ends.
Also, there are no locks on the feature class after the comprehension ends.