As is not possible formulate, or add a solution to question:
Delete temporary files from geoprocessing in ArcGIS 10 and above , because is [closed]
I would like to make this question with same problem, I found that geoprocessing tools like arcpy.gp.SegmentMeanShift makes a lot of heavy temporal files (script is shown below) that keeps locked while the script shell still open, so you can't delete them, in the same Python script, also when tool is repeated in loop many times, script also crash.
I would like to ask for a correct way to manage this problem. I'm using Python 2.7, and ArcPy (ArcMap 10.6) package in Windows 10.
This is a part of script code that repeats many times "arcpy.gp.SegmentMeanShift_sa" on same raster (in this sample), so in this case on image shouldn't be the problem of crashing:
import arcpy import os # Check out any necessary licenses arcpy.CheckOutExtension("spatial") # segmentation variables Spectral_Detail = "15" Spatial_Detail = "18" Minimun_Segment = "9" # folders route work_folder = os.getcwd()+"\\work_folder" out = work_folder +"\\out" out_OS = work_folder +"\\out_OS" #geoprocessing order =0 while order < 150: order +=1 print "processing " +str(order) id_item = "31165796" arcpy.gp.SegmentMeanShift_sa(work_folder +"\\out\\"+id_item+".tif", out_OS+"\\" + id_item +"_"+Spectral_Detail+"_"+Spatial_Detail+"_"+Minimun_Segment+"_"+str(order)+".img", Spectral_Detail, Spatial_Detail, Minimun_Segment, "")
When this script is executed, these files are generated in temp directory each iteration (XX = random number):
SMS_Index_XX.til --> 32.768 kb
After more or less a hundred iterations, Python crash, whith this error:
Traceback (most recent call last):
File "D:/TALLER_SSD/20190405_SEGMENTACION/python/test/segmentacion_temporal_problem_2.py", line 212, in <module>
Spectral_Detail, Spatial_Detail, Minimun_Segment, "")
File "C:\Program Files (x86)\ArcGIS\Desktop10.6\ArcPy\arcpy\geoprocessing\_base.py", line 510, in <lambda>
return lambda *args: val(*gp_fixargs(args, True))
ExecuteError: ERROR 999998: Unexpected Error.
Failed to execute (SegmentMeanShift).
This crash is not the problem, errors can be controled, the problem still in main cuestion, temp files can't be deleted while script is running Python's Shell, filling disk space in many iterations. Even when script finish well and quit, temporary files are not deleted.
Any solution, or something missed?
A solution that works for me it's to divide script into two:
- first script iterate on an item list, pass an "id_item" to the second, and erase temporal files
- second script do geoprocess for each item.
Note: in sample code there is a comented sample geoprocess, it's only for take an idea of what could be.
First script (01_call_process.py):
import subprocess import os script = os.getcwd()+"\\02_process.py" # script process script route item_list = ["1","2","3"] for item in item_list: #iterate through items # run script process, passing item value, without open cmd window subprocess.call ("python " + script +" "+item, shell=True) #delete temporal files, by Kevin @ stackoverflow tmp = os.environ.get("TMP") if tmp: for f in os.listdir(tmp): name = os.path.join(tmp, f) try: if os.path.isfile(name): os.remove(name) else: os.removedirs(name) except OSError as e: # Specific exception handling could be done based on the 'errno'. pass # skip files that are in use on Windows or otherwise cannot be removed.d.
Second script, placed in the same folder as first(02_process.py) will do the job:
import sys import os id_item = str(sys.argv) #get passed data raster = os.getcwd()+"\\"+id_item+".tif" #sample code in order to visualice results in log files... file = open(os.getcwd()+"\\Sample_file_"+id_item+".txt", "w") file.write(raster+" Processed") file.close() # do geoprocess....
When second script is finnished, first script keeps running until the end of the loop.
This solution works for me, but make two scripts more complicated to manage workflow. But at less prevent disk to get full and script to crash.
Any other solutión?