I am interested in learning methods to utilize the full extent of multicore processing power available on a desktop computer. Arc states that background geoprocessing allows the user to utilize multiple cores, however, tasks essentially have to wait in line for the previous task to be completed.
Has anyone developed parallel or multithreaded geoprocessing methods in Arc/Python? Are there hardware bottlenecks that prevent multicore processing on individual tasks?
I found an interesting example in Stackoverflow that caught my interest, although it is not a geoprocessing example:
from multiprocessing import Pool import numpy numToFactor = 976 def isFactor(x): result = None div = (numToFactor / x) if div*x == numToFactor: result = (x,div) return result if __name__ == '__main__': pool = Pool(processes=4) possibleFactors = range(1,int(numpy.floor(numpy.sqrt(numToFactor)))+1) print 'Checking ', possibleFactors result = pool.map(isFactor, possibleFactors) cleaned = [x for x in result if not x is None] print 'Factors are', cleaned