1

I am trying to perform some raster math and wanted to try parallel processing in R using rasterEngine.

I am able to run the script for a few minutes and watch my CPU usage increase substantially, but soon an error occurs and processing stops.

Here is my code

library(raster)
library(spatial.tools)
library(doParallel)

cpus = 24 # The number of nodes/cores to use in the cluster
cl <- makeCluster(spec = cpus, type = 'PSOCK', methods = FALSE, outfile="E:/Heatload_10m/cluster.txt")
registerDoParallel(cl)

l = raster('E:/Heatload_10m/Lat_Radian.img')
s = raster('E:/Heatload_10m/R1_10m_WGS84_Slope.img')
f= raster('E:/Heatload_10m/Folded_Aspect.img')

heatload <- function(lat,slope,folded){
    hl= exp(-1.467 + 1.582*cos(lat)*cos(slope)-1*cos(folded)*sin(lat)*sin(folded)-0.262*sin(lat)*sin(slope)+0.607*sin(folded)*sin(slope))
    return (hl)
}

print('Starting Heatload Calc')
final_heatload <- rasterEngine(lat=l,slope=s,folded=f, fun=heatload, overwrite=TRUE, processing_unit = "chunk", chunk_format = "raster" setMinMax =TRUE, )
stopCluster(cl) 
registerDoSEQ()  

Here are the error messages:

The instruction at 0x61b83401 referenced memory at 0xd120c1d0. The memory could not be written.

enter image description here

Error in unserialize(socklist[[n]]) : error reading from connection

Error in serialize(data, node$con) : error writing to connection

enter image description here

1

This is an old question, so hopefully you found the answer. This doesn't address your question but after setMinMax = TRUE you need to delete the comma.

Regarding your question, I think that you should specify an output file by setting filename = yourpath.tif. Then rasterEngine will use that as a temporary file. The alternative is it will likely set a temp file somewhere in your C drive, which can not only lead to read/write issues but also to memory issues (e.g. if your C drive is quite small as is my case, then rasterEngine will go on forever and not crash). I think your writing issue might have to do with that.

In any case, you should be testing your function using debugmode = TRUE to see where exactly your code is crashing. If your function code runs ok, then set verbose = TRUE and have a look at when the function is crashing and what it tried to do beforehand.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.