I am following the accepted answer for the question: Batch conversion of netCDF to Raster in ArcGIS using Python, obviously, to convert netcdfs. The code taken from the above answer (pasted below) works fine on a single NetCDF. However, if I run the same function on many
.nc files by making
inNetCDF variable a parameter, ArcMap breaks down. It crashes with ArcGIS 10.1 and hangs with ArcGIS 10.2.2 (on two different machines) after generating about 500 or 600 images.
I suspected that the issue is that ArcMap can't handle this many layers (more than 1000 considering the extracted nc layers). So I added:
arcpy.env.addOutputsToMap = False
to forbid adding output to the content index, but the crashes still remain.
Is it possible to run the involved ArcToolbox functions like
CopyRaster_management outside of ArcMap from command line python and, if not, how do I avoid the crash problem?
def extractAllNetCDF(): variable = "RRt_10m" x_dimension = "lon" y_dimension = "lat" band_dimension = "" dimension = "time" valueSelectionMethod = "BY_VALUE" outLoc = "E:/New Folder/" inNetCDF = "E:/netCDFFiles/RRt.nc" nc_FP = arcpy.NetCDFFileProperties(inNetCDF) nc_Dim = nc_FP.getDimensions() for dimension in nc_Dim: top = nc_FP.getDimensionSize(dimension) for i in range(0, top): if dimension == "time": dimension_values = nc_FP.getDimensionValue(dimension, i) nowFile = str(dimension_values) #THIS IS THE NEW CODE HERE dv1 = ["time", dimension_value] dimension_values = [dv1] #END NEW CODE arcpy.MakeNetCDFRasterLayer_md(inNetCDF, variable, x_dimension, y_dimension, nowFile, band_dimension, dimension_values, valueSelectionMethod) arcpy.CopyRaster_management(nowFile, outLoc + nowFile + ".img", "", "", "", "NONE", "NONE", "") print dimension_values, i
Here's the outcome using python from the command line.
The script works well except for needing a few
sys. There was still a crash, but the number of processed images increased 10 fold to about 7000. This time, memory seems to be at issue, with 1.8G consumed before the crash. I am trying the 64 bit background processing as suggested.