1

I have a GP script and I'm wondering if I should use Scratch GDB at the processes or just regular GDB, The problem is that this is a read only environment and my temp layers need additional processes and editing

  1. What are the advantages using Scratch GDB for saving temporal data of geoprocessing script ?

  2. Whats the difference between those options in terms of performance ?

1 Answer 1

5

In the first place, you need a certain place, or a container, for your temporal data produced by your scripts so you can store it somewhere before it will be fed to the second process. If there will be no need to store such temporal aka intermediate data, there is no need for using any workspace for storing the data.

If you do need to store your data somewhere to be able to supply input for the upcoming GP tools or processes, you can surely use any file, personal, or ArcSDE geodatabase for storing these data. I would go with a file geodatabase wherever possible since it is faster comparing to personal and ArcSDE ones. It can be any file geodatabase that can be pre-created and stored locally on the computer you run the script on. Or it can be just one file geodatabase for instance on a network shared disk (however, this will work slower since writing locally is generally faster than writing over the network). You can of course create a file geodatabase as a part of the script logic to make sure you work with a new fresh geodatabase at a certain location.

Answering the 1).

Your other option is to use a scratch geodatabase which will be specific to each computer where you have ArcGIS installed. This geodatabase is stored at C:\Users\%usernama%\AppData\Local\Temp\scratch.gdb and you can refer to it as arcpy.env.scratchGDB[1] from your scripts. So, when you define the temp outputs, you use this notation:

out_dissolved = os.path.join(arcpy.env.scratchGDB,"PointsDissolved")

This will make sure the output will be written to a scratch geodatabase of the user's computer where the script is running. This is convenient because you don't have to define a hard-coded path to a geodatabase and neither do you have to precreate one before running the tool, since the scratch geodatabase will always exist.

Answering the 2).

There will be no performance difference between writing the temp data to a scratch file geodatabase or a regular one. The only situation I can think of where the performance may slow down is when there will be many concurrent processes running the script tool on the same PC and this may result in I/O competition for the geodatabase. Another thing is that theoretically the bigger the file geodatabase is, the slower it will work, but unless you don't work with terabytes, you will not notice any difference.

Deleting temporal geodatabase data after you don't need is a good practice. If you reuse the same geodatabase again and again and there are hundreds of processes writing there all the time, it might be a good idea to compact the geodatabase to tidy it up a bit, but again I doubt you will notice any significant performance difference.

What you can do in terms of performance, is to use in_memory workspace for writing your temp outputs to. This will be faster in most situations, and I highly recommend using it wherever you can. It might be more convenient however to write and keep temp outputs on disk first so you can explore them and make sure the script works as expected. After you are done with testing and troubleshooting, just replace the temp geodatabase workspace of your choice in the script with in_memory and your temp data will be written to the memory.

Esri Help pages:

Managing intermediate (scratch) data in shared model and script tools

Scratch GDB

2
  • Thanks Alex for your detailed response, I have 3 more Questions regrading to this ? 1. What about self schema lock if i'm working in memory at temp GDB ? 2. is scratch GDB is read only or I can process it ? 3. What is "I/O competition for the geodatabase" ? Thanks agin - Geog Commented Dec 26, 2013 at 9:16
  • 1
    1. You get the same schema locks for the in_memory workspace as for a normal file gdb. 2. You can write to scratch gdb and delete datasets already written to it. 3. It is when multiple processes compete with each other to obtain access to file gdb to write objects to it. resources.arcgis.com/en/help/main/10.1/index.html#//… Commented Dec 26, 2013 at 9:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.