1

I have a arcpy python script that runs a process that I publish as a geoprocessing service on ArcGIS Server 10.3.1. The python script uses various input feature classes. These feature classes are within geodatabases that are registered on the ArcGIS Server.

My python script also uses the in_memory workspace for storage of some temporary data. The in_memory feature classes are not the input or final output, there are only intermediate data.

When trying to publish the geoprocessing service, the publishing tools want to copy the in_memory feature classes to the server, since the in_memory workspace is not registered with ArcGIS Server. The specific warning I get is Data source used by Script MyToolThatDoesStuff is not registered with the server and will be copied to the server: in_memory\myTempData

I let the publishing tools copy the in_memory data to the server and the my geoprocessing service works as expected. However, I suspect that every time the tool is run, the in_memory data is copied to the arcgisserver\directories\arcgissystem\arcgisinput\ MyToolThatDoesStuff .GPServer\extracted\v101\data1.gdb geodatabase and never removed. Over time this gdb bloats, slows down the geoprocessing service and ultimately fills the disk, creating mass problems.

My questions:

  1. Is there a way to prevent the geoprocessing tools from publishing the in_memory workspace to the server?
  2. Is there a way to register the in_memory workspace with ArcGIS Server?
  3. Is using the in_memory workspace in scripts that are published as geoprocessing services not “best practice”. If that's the case, how should temporary, intermediate data be processed in arcpy scripts?
Improve this question

1 Answer 1

Reset to default
3

In_memory layers are not written to the disk; that is the whole point with the in_memory geodatabase workspace.

  1. The in_memory workspace is not published to the server.
  2. Registering in_memory on the server wouldn't make sense.
  3. Using in_memory is best practices for handling the intermediate data both for desktop and server geoprocessing workflows.

For the publishing purposes, I usually suggest do a dry run of the tool, when no code is executed and no data is created. Publish the tool results and then either uncomment the code in the published file (or maybe you have a parameter value that is used for defining the dry run).

Even better, keep the copied script with ArcGIS bits published (parameters etc) separated from the business code. Please refer to this post for best practices I found very useful when working with GP services.

Improve this answer
2
  • Your post looks very interesting. I like how your "tricking" it so that python scripts act more like they did in 10.0 - that way you can just modify the script without having to republish everything. The other comment I have is that I understand how in_memory should work, but what I don't understand is why the publishing tools is specifically trying to copy the in_memory data to the server.
    – dvdhns
    Commented Apr 21, 2016 at 15:30
  • I finally marked this as Answered because it helped me get over the hurdle of what ArcGIS Server does with you publish a geoprocessing service. What I did find is that most of the in_memory layers are kept as in_memory layers when published. However one layer is being saved in \extracted\v101\data1.gdb. I have no idea why that is happening. The data1.gdb is also bloating unless you compact it regularly. However I as you suggest I can go in to the published py files and fix it.
    – dvdhns
    Commented Sep 9, 2016 at 22:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.