Take the 2-minute tour ×
Geographic Information Systems Stack Exchange is a question and answer site for cartographers, geographers and GIS professionals. It's 100% free, no registration required.

I have this error when I try to use Manager() from python multiprocessing library. (Just start QGIS open Python console and execute the two following lines)

>>> import multiprocessing
>>> mgr = multiprocessing.Manager()
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "C:\DEV\osgeo4w_dev\apps\Python27\lib\multiprocessing\__init__.py", line 99, in Manager
    m.start()
  File "C:\DEV\osgeo4w_dev\apps\Python27\lib\multiprocessing\managers.py", line 524, in start
    self._process.start()
  File "C:\DEV\osgeo4w_dev\apps\Python27\lib\multiprocessing\process.py", line 130, in start
    self._popen = Popen(self)
  File "C:\DEV\osgeo4w_dev\apps\Python27\lib\multiprocessing\forking.py", line 252, in __init__
    cmd = get_command_line() + [rhandle]
  File "C:\DEV\osgeo4w_dev\apps\Python27\lib\multiprocessing\forking.py", line 339, in get_command_line
    if process.current_process()._identity==() and is_forking(sys.argv):
AttributeError: 'module' object has no attribute 'argv'
>>> 

It happens in both 1.8 and 1.9 dev version on Windows. Of course everything is ok using QGIS on Linux!... :P

It seems something related to the main Python/QGis process startup and the way the library tries to start a new process.... but I can't figure out any workaround.

Any idea? (already posted this on qgis dev list but had no response...)

share|improve this question
    
Have you found a solution to this issue? I am interested to know are you running this application as a windows service? I am getting the same error and looking for solutions. –  MattWritesCode Jan 4 '13 at 11:12
    
There is a related Q on SO. Although the python path is incorrect with OSGeo4W. Unfortunately it all together still did not help me to go too far either. –  mlt Nov 11 '13 at 19:41
add comment

1 Answer

I was wrong in my comment to original post. Suggested workaround does work. In case someone else has that issue here is what needs to be done in QGIS 2.0 before instantiating Manager().

# OSGeo4W does not bundle python in exec_prefix for python
path = os.path.abspath(os.path.join(sys.exec_prefix, '../../bin/pythonw.exe'))
mp.set_executable(path)
sys.argv = [ None ]

Note that this cannot be tested from python console as Windows lacks fork() and all multiprocessing statements shall be isolated.

If one wants to play around with multiprocessing and embedded python from OSGeo4W bundle outside of QGIS here is the code.

tst.py

import multiprocessing as mp
import sys, os

print("Non-isolated statement")

if __name__ == '__main__':
    print("I'm in main module")
    path = os.path.abspath(os.path.join(sys.exec_prefix, '../../bin/pythonw.exe'))
    mp.set_executable(path)
    print("Setting executable path to {:s}".format(path))
    sys.argv = [ None ]               # '../tst.py' __file__
    mgr = mp.Manager()
    print("I'm past Manager()")

tst.c

#include <Python.h>
#include <stdio.h>

int main(int argc, char * argv[]) {
  char buf[10240] = {0};
  size_t sz, res;
  FILE *f;
  Py_SetProgramName(argv[0]);  /* optional but recommended */
  Py_Initialize();

  f = fopen("../tst.py", "r");
  // obtain file size:
  fseek(f, 0 , SEEK_END);
  sz = ftell(f);
  rewind(f);
  res = fread(buf, 1, sz, f);
  fclose(f);
  PyRun_SimpleString(buf);
  getchar();
  /* PyRun_SimpleString("from time import time,ctime\n" */
  /*                    "print 'Today is',ctime(time())\n"); */
  Py_Finalize();
  return 0;
}

CMakeLists.txt

cmake_minimum_required(VERSION 2.8)
find_package(PythonLibs)
include_directories(${PYTHON_INCLUDE_DIRS})

add_executable(tst tst.c)
target_link_libraries(tst ${PYTHON_LIBRARY})

if(MSVC)
  set_target_properties(tst PROPERTIES LINK_FLAGS -NODEFAULTLIB:python27_d)
endif()
  1. From your OSGeo4W console, initialize your vcvarsall.bat building environment
  2. Create subfolder build (or alike) and cd to it
  3. Use cmake-gui .. to generate jom/nmake makefiles in build folder provided all files are saved in the parent one
  4. Use nmake or jom to build tst.exe
  5. Try to run tst or python ..\tst.py
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.