Take the 2-minute tour ×
Geographic Information Systems Stack Exchange is a question and answer site for cartographers, geographers and GIS professionals. It's 100% free, no registration required.

I would like to run a multiprocessing task from a python add-in tool. My issue is that the process keeps failing. Basically crashes ArcMap.

Here is my basic code:

def function(startOID, endOID, fc):

    wrksp = r"c:\temp\mp_addintest\data\test_%s.txt" % (int(startOID) + int(endOID))
    # real logic removed to dumb it down
    with open(wrksp, 'w') as writer:
        writer.write("%s to %s from %s \n" % (startOID, endOID, fc))
    return wrksp
class btnMP(object):
    """Implementation for src_addin.MPButton (Button)"""
    def __init__(self):
        self.enabled = True
        self.checked = False
    def onClick(self):
        pool = None
        try:
            pythonExe = os.path.join(sys.exec_prefix, 'python.exe')
            multiprocessing.set_executable(pythonExe)
            pool = multiprocessing.Pool(4)
            results = []
            for i in xrange(4):
                results.append(pool.apply_async(function, [str(1),
                                      str(i),
                                      str("test")]))
            pool.close()
            pool.join()
            for result in results:
                print result.get()
        except:
            del pool
            print 'error'

If I run the code outside of ArcMap or from a toolbox, it works without a problem, but when I put the logic inside a button, it causes arcmap to crash.

My guess is that ArcMap is running in process for all python add-ins. Is there a work around for this issue?

I've tried adding in the freeze_support() to the code as well, but that did nothing as well.

share|improve this question
1  
If ArcMap is crashing, contact your ESRI support. if they can replicate it they'll acknowledge its a bug (and maybe even fix it one day). –  GIS-Jonathan Oct 31 '12 at 10:40
    
Have you installed all 5 Service Packs for ArcGIS 10.x that are already out? Maybe that'll help –  Sergios Kolios Nov 2 '12 at 16:07
    
The OP is using 10.1 –  Petr Krebs Nov 2 '12 at 16:14
    
Also service packs are cumulative so you only need to install the latest one, not each in succession. –  blah238 Nov 2 '12 at 20:23
    
SP1 for 10.1 was released this week. –  Timothy Michael Nov 2 '12 at 21:58
show 4 more comments

1 Answer 1

up vote 7 down vote accepted
+50

Parallel processing is easier 'shown than done.' In the case of stuffing this all into a button, I'm guessing two issues:

  1. Multiple threads block the ArcMap UI thread, or
  2. ArcMap puts its own schema lock on the data source and doesn't permit the python process access to the data.

Hmm looking further issue has been documented here in an ArcGIS Resources page. Schema lock looks like the culprit.

share|improve this answer
    
Not sure if you meant to link something other than what you did (an ArcGIS forums post, not an official document). –  blah238 Nov 1 '12 at 6:57
    
The forum is the correct link. When someone finds more official documentation they can feel free to post it. –  WolfOdrade Nov 1 '12 at 18:07
    
Thank you for your suggestions. I believe it's actually caused by #1. The thread blocks for ArcMap UI. I am use an SDE database, so schema locks are not my issues here. –  josh1234 Nov 5 '12 at 11:37
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.