Create temporary files as intermediate output to be used as input in concatenation

Question

Q_PART_I

I need to merge many (MANY) tables in one unique output table by means of the arcpy.Merge_management. Anyway, I'd like to avoid the creation of tons of intermediate files as my script running, having only my final big table rather than a number of "mini-table" for each file I have. I'm converting raster attribute table to table views (by arcpy.TableToTable_conversion), and then using them to run the Merge. How can I avoid having a table for each raster in the intermediate process? Hope someone could help me. I've been reading something about the creation of an InMemoryWorkspace, but I couln'd manage it so far. Thanks in advance.

Q_PART_II

Ok, I tried both Your suggestions, but so far I couldn't make it. It seems the Merge function cannot recognize the temporary files as input. Infact, I receive this error: ERROR 000735: Input Datasets: Value is required Failed to execute (Merge) Here is my script (the important part is at the end, I can always avoiding this problem by simply having a lot of files, but I'd prefer to be more "clean"):

#import relevant modules, create geoprocessing dispatch object
import win32com.client, sys, string, os, arcpy

gp = win32com.client.Dispatch("esriGeoprocessing.gpDispatch.1")

# Remember to change this to wherever your files are stored
gp.workspace = 'G:\\Chile_2012\\MODIS10A2.V005\\provvisoria\\ATTRIBUTE_TABLE'
# Remember to change the path to point where you want to store the final tables
out_dir = 'G:\\Chile_2012\\MODIS10A2.V005\\provvisoria\\ATTRIBUTE_TABLE'

try:
    rasters = gp.ListRasters("*", "all")
    rasters.reset()
    rst = rasters.Next()

    while rst:
        # Create the new field
        gp.AddField_management (rst, "FILENAME", "text", "", "", "50")

        # Apply the filename to all entries       
        gp.CalculateField_management (rst, "FILENAME", '"' + rst + '"')
        rst = rasters.Next()

except:
    print gp.GetMessages ()

listRaster = arcpy.ListRasters ()

# Create a table view from each raster to be used as input for the next Merge function:
for raster in listRaster:
    arcpy.TableToTable_conversion (raster, "in_memory", 'Tab_'+ raster)

listTable = arcpy.ListTables ()

# Change the name of the output table as needed!!!
arcpy.Merge_management (listTable, 'Cl_Rio_Aconcagua.dbf')

Hope this could help You helping me in turn. Many thanks in advance!

Answer 1

Sadly, the in_memory workspace in ArcGIS is one of the least-known, least used, and least advertised, yet most powerful functions in arcpy scripting. It really needs to be more highly touted than it is...

What you really should consider doing is copying the data into an memory layer using the in_memory workspace. Arc will treat it just like a normal file-based table, but it will automatically clean itself up after you exist your python script.

So your script would be something like:

arcpy.TableToTable_conversion(inTable, "in_memory", outTable, expression)

Answer 2

A few points:

• Table To Table creates a Table, not a Table View. This creates new data, in this case probably unnecessarily.

• Make Table View creates a Table View.

• A Table View is a reference to the original data with a few additional properties such as visible fields and a definition query, as well as selection support. But, much like a Layer, it does not actually create new data, hence why it's called a View.

• If you want to avoid intermediate data, use Make Table View.

Is there some reason you are using Table to Table specifically?

Update:

Here is a code example to hopefully steer you in the right direction. I have not tested it, however.

This picks up after your listRaster = arcpy.ListRasters() line.

tableviews = []
for raster in listRaster:
    tableviews.append(arcpy.MakeTableView_management(raster, 'TabView_' +
        os.path.splitext(os.path.basename(raster))[0]).getOutput(0))
arcpy.Merge_management(tableviews, 'Cl_Rio_Aconcagua.dbf')