ArcGIS Geoprocessing Services - Don't work if 'Run python script in process' not checked

Question

I have a standard geoprocessing toolbox that I am exposing as a geoprocessing service. The tools in this toolbox are plain 'ol python scripts. All of this is being done in ArcGIS 9.3.1.

As a standalone toolbox run in the desktop environment, all of the tools execute flawlessly. However, as soon as I try to execute the same 'tools' as exposed through the geoprocessing service, all type of weird stuff happens. Among other things, the script stops executing abruptly, without throwing any type of error. In turn, the client never gets a notification of success or failure...

After many days of head scratching, I have become aware of a property of script tools - the 'Run Python script in process' option. From ESRI's standard web help:

Running in-process requires that all modules loaded with the Python import directive have the necessary logic to enable them to run in-process. All standard Python libraries, such as os, string, and time, have the necessary logic. However, non standard modules obtained from third parties may not have the necessary logic to run in-process. If you are experiencing unexplainable problems when your script runs, try unchecking the in-process option and running your script again. If the problem goes away when running out-of-process, then there is most likely an issue with one of the modules you imported. In this case, leave the option unchecked.

Sure enough, my scripts do make use of 3rd party libraries, so I was thinking I finally figured it out - uncheck that option and all my problems should disappear. Well, if I uncheck the option and expose tool as gp service - it fails immediately with the "Error Executing Task Error Code 500". Instead of making half-way through the script - it just bombs immediately.

Does anyone know if it is possible to run geoprocessing services "out of process"?

Answer 1

I don't think the problem is in the in-process option.

In general in ArcGIS Server the tool must be in-process, because every time a gp tool is called, ArcGIS Server creates a new process for that call and the tool runs inside it, it's not possibile, for server threading reasons, to run the gp tool outside that created process.

on the other side if you plan to run a gp tool on ArcGIS Server you have to think ahead about how to develop the tool because there are some important differences between a desktop tool and a server tool: not all the desktop tool are suitable to run on the server (and this is true noy only for Python gp tools, but also for Model Builder gp tools)

There are differences in what kind of parameters you can use for input and output, you have to take into account the execution folder that ArcGIS Server creates every time a gp tool is called (in which you also have a special folder containing the Scratch geodb), you cannot rely on the file system in the same manner you do on the desktop, and other things like these.

So maybe the problem is somewhere else, and if I have to guess basing on your description this is the case.

Answer 2

I'm running this same stack and there are certain things that it can do orders of magnitude faster than arcpy. My advice is to ensure you're running the absolute latest version of GDAL and bindings - I get mine here: vbkto.dyndns.org/sdk Then comment out all of your GDAL (osr, ogr) calls and uncomment them one at a time. You might discover that its only one call that's killing the script and you can swap it out with a workaround. Persevere; it is possible to make this work.