Take the 2-minute tour ×
Geographic Information Systems Stack Exchange is a question and answer site for cartographers, geographers and GIS professionals. It's 100% free, no registration required.

I am attempting to find the fastest method for looping over multiple map documents while keeping memory resources down. I'm running this script outside of Arcmap 10.

I have 5 map documents which are about 1mb each.

I have used both return functions and generator functions. After some testing I found the following code below to be the fastest, running in about 20 seconds.

Memory usage balloons to about 220 000 k when running then back down to 150 000 k after the del statements. If I remove the the del statements run in 18 secs but memory after remains at 220 000 k. which is undesirable.

Note: Importing arcpy on its own uses 117 000 k

I am looking for any feedback you have or suggestions to get increase speed, and reduce memory usage for the code below.

import time
import glob
import arcpy

start = time.time()

path = "F:\\Jobs2012\\06292\\ESRI\\"

def getMxdPath(path):
    """ Returns list of mxd filepath
   as strings for a given path."""
    mxdPaths = list()
    for mxd in glob.glob(path + r"*.mxd"):
        mxdPaths.append(mxd)
    return mxdPaths

def getMxdObjects(mxdPaths):
    """ Returns list of mxd objects"""
    mxdObjects = list()
    for mxd in mxdPaths:
        mxdObjects.append(arcpy.mapping.MapDocument(mxd))
    return mxdObjects

def genr(mxdObjects):
    for mxd in mxdObjects:
        yield mxd

def saveMxd(mxdGen):
    for mxd in mxdGen:
        print mxd.filePath
        mxd.save()
        #del mxd

mxdPaths = getMxdPath(path)

mxdObject = getMxdObjects(mxdPaths)
#del mxdPaths

mxdGen = genr(mxdObject)
#del mxdObject

saveMxd(mxdGen)
#del mxdGen

del mxdPaths
del mxdObject
del mxdGen

elapsed = (time.time() - start)

print elapsed
share|improve this question
add comment

1 Answer 1

I don't understand why you're doing this:

def getMxdObjects(mxdPaths):
    """ Returns list of mxd objects"""
    mxdObjects = list()
    for mxd in mxdPaths:
        mxdObjects.append(arcpy.mapping.MapDocument(mxd))
    return mxdObjects

def genr(mxdObjects):
    for mxd in mxdObjects:
        yield mxd

Consolidate the two so you don't have all the MXDs in memory at once (that list is unnecessary if you're using a generator):

def genr(mxdPaths):
    """ Returns list of mxd objects"""
    mxdObjects = list()
    for mxd in mxdPaths:
        yield arcpy.mapping.MapDocument(mxd)

saveMxd(genr(mxdPaths))

Another thing you can do is force a garbage collection:

def genr(mxdPaths):
    """ Returns list of mxd objects"""
    import gc
    for mxd in mxdPaths:
        yield arcpy.mapping.MapDocument(mxd)
        gc.collect()

Also, 220000 k is only about 215 megabytes, this is a very small amount of memory on modern hardware.

share|improve this answer
    
Hey, I ran your code takes 59.7239999771 without garbage collection (GP) and 69.506000042 with. My code runs in 20.9749999046 with GP and 18.6949999332 without GP. As you mention 215 mb is small overhead so I won't worry about it. –  TForward Jul 5 '13 at 23:44
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.