37

I am unclear how best to use the in_memory workspace in ArcGIS. The best resource I have found on the subject is from ESRI on Using in-memory workspace. Unfortunately, the help section is not very informative. Also, there is a post on this site that briefly touches on the subject here and here.

My specific questions:

  • What is the proper usage and syntax for using in_memory workspace in ArcGIS/arcpy based scripts?
  • Is in_memory workspace the same as, for example, creating a layer using arcpy.MakeFeatureLayer_management()?
  • Are there any standards such as deleting in_memory workspace at the end of the script?
Improve this question

3 Answers 3

Reset to default
45

I've been using "in_memory" quite a bit recently. It can be very useful, as it has the potential to dramatically increase processing speeds for certain tasks, however if you are working with very large datasets, it might cause your program to crash.

You can use "in_memory" to define process outputs... often, if I am performing a task on a feature class, I will copy it to the "in_memory" workspace first:

inFeature = r'C:\myDir.gdb\myFeature'
memoryFeature = "in_memory" + "\\" + "myMemoryFeature"
arcpy.CopyFeatures_management(inFeature, memoryFeature)

Note that you don't have to concatenate memoryFeature together like I did, you could write it out as "in_memory\myMemoryFeature", I just like doing it that way to switch back and forth between "in_memory" and a physical directory easily. You can then perform processes on your feature in memory. When you are done, you can reverse the process to save it back to a directory.

I could be wrong, but I believe it is not the same as creating a feature layer. feature layers give you access to selection methods and other layer specific operations. Think of the "in_memory" directory as the vector equivalent to the raster object (raster = arcpy.Raster(myRasterLocation))

To clean up after using "in_memory" simply add the following line of code:

arcpy.Delete_management("in_memory")

Hope that helps.

Improve this answer
1
  • 3
    I agree with @egdetti. It is very useful, very powerful, and significantly speeds up processing. You do not actually need to delete the workspace/layer, however, as it will clean itself up when you close the application. As was mentioned, though, running the arcpy.Delete can be helpful (sometimes necessary) if you find yourself running out of RAM because you have too much stuff in the in_memory workspace.
    – RyanKDalton
    Commented Oct 10, 2012 at 22:02
12

There's not a whole lot to it. There's a magic directory you can access in geoprocessing tools and arcpy cursors named in_memory. You'd do something like arcpy.management.CreateTable('in_memory', 'my_new_table') or CreateFeatureClass to make a new featureclass or table. Then you have a new table in that process with the path in_memory\my_new_table, if you're in arcmap it will add a table view to the table of contents named my_new_table that you can use as a shortcut to get to it. If it's a temporary in memory table, just use arcpy.management.Delete(r'in_memory\my_new_table') when you're done with it at the end of your script.

Improve this answer
11

Slightly off topic, but I think worth pointing out. Firstly I agree with both @JasonScheirer and @egdetti above that in_memory can be very useful. The biggest thing to note is that in_memory workspace does not support raster data. If this becomes an issue I'd reccomend using the Python tempfile module. This won't be as fast, but is more reliable. For example, you could do something like:

import arcpy
import tempfile
import os.path
import shutil

try:
    temp_dir = tempfile.mkdtemp()
    temp_gdb = "temp.gdb"
    arcpy.CreateFileGDB_management(temp_dir, temp_gdb)
    arcpy.env.workspace = os.path.join(temp_dir, temp_gdb)

    #Some processing...

except Exception, e:
    #handle exceptions here
    print e
finally:
    shutil.rmtree(temp_dir, True)
Improve this answer
6
  • 2
    arcpy has a way to deal with rasters built in with the arcpy.Raster() method. By defining a variable such as myRaster = arcpy.Raster("C:\RasteLocation") you create a raster object in memory that can be used in map algebra calculations and various other raster operations easily. Saving such a raster object is as simple as writing in myRaster.save("C:\SaveLocation"). A very helpful feature that arcpy brings to the table.
    – bluefoot
    Commented Oct 11, 2012 at 0:24
  • @egdetti Fair call. I'm curious to know how it handles larger datasets though.
    – om_henners
    Commented Oct 11, 2012 at 0:56
  • 1
    @mbenedetti - the Raster object writes a raster to the arcpy scratch workspace - as late as it possibly can, which allows arcpy to optimize I/O by stacking single-cell processing tools (like "+" for example) into a single execution process, for example. The save() method will force execution of the raster processing.
    – Curtis Price
    Commented Apr 2, 2013 at 0:46
  • 3
    @om_henners in_memory supports rasters in ArcGIS 10.1!
    – Curtis Price
    Commented Apr 2, 2013 at 0:47
  • 2
    @om_henners arcpy provides a tool to create scratch names, which has advantages in that for example a scratch feature class in a folder automatically has ".shp" added to it. The method is arcpy.CreateScratchName().
    – Curtis Price
    Commented Apr 2, 2013 at 0:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.