1

I am relatively new to parallel computing in R. I am using a high performance cluster to perform kriging for soil data points. I found this previous post which I used as a template:

How to achieve parallel Kriging in R to speed up the process?

This is the code I have been using, non-parallel, it works fine a produces kriged rasters. For clarity, I am using Random Forests to initially create the spatial predictions from the point data, then run kirging on the residuals.

qs_s5: spatial points data frame of the point data

p2: spatial pixels data frame of the covariate rasters

p <- as(p, Class= "data.frame")
p1 <- p[complete.cases(p),]
p2 <- na.omit(p1)
colnames(p2)[colnames(p2) == "s1"] <- "lat"
colnames(p2)[colnames(p2) == "s2"] <- "long"
v.rf <- variogram(qs_s5$resid.rf~1, data = qs_s5, width = 1000,cutoff=10000)
vm <- vgm(psill = 10, model = "Sph", range =4000, nugget = 20)
vmf.rf <- fit.variogram(v.rf, model = vm)
rf.s_ppm <- krige(resid.rf ~ 1, locations = qs_s5, newdata = p2, model = vmf.rf, debug.level = -1)
gridded(p2)<- ~lat+long
proj4string(p2) <- CRS(utm14)
rf.s_ppm$trend <- p.rf


rf.s_ppm$trend <- p.rf

rf.s_ppm$pred <- rf.s_ppm$trend + rf.s_ppm$var1.pred

rf.s_ppm$lower <- rf.s_ppm$pred - 1.645 * sqrt(rf.s_ppm$var1.var)
rf.s_ppm$upper <- rf.s_ppm$pred + 1.645 * sqrt(rf.s_ppm$var1.var)

writeGDAL(rf.s_ppm["pred"], fname = "./out10/s_ppm_parallel.tif", drivername = "GTiff", type = "Float32")

enter image description here enter image description here

Here is the code with parallel integrated:

v.rf <- variogram(qs_s5$resid.rf~1, data = qs_s5, width = 1000,cutoff=10000)
vm <- vgm(psill = 10, model = "Sph", range =4000, nugget = 20)
vmf.rf <- fit.variogram(v.rf, model = vm)

gridded(p2)<- ~lat+long
proj4string(p2) <- CRS(utm14)

numCores <- detectCores() -1
numCores 

cl <- makeCluster(numCores)

parts <- split(x = 1:length(p2), f = 1:numCores)

clusterExport(cl = cl, varlist = c("qs_s5", "p2", "vmf.rf", "parts"), envir = .GlobalEnv)
clusterEvalQ(cl = cl, expr = c(library('sp'), library('gstat')))

parallelX <- parLapply(cl = cl, X = 1:numCores, fun = function(x) krige(resid.rf ~ 1, locations = qs_s5, newdata = p2[parts[[x]],], model = vmf.rf, debug.level = -1))

stopCluster(cl)

mergeParallelX <- maptools::spRbind(parallelX[[1]], parallelX[[2]])
for (j in 3:length(parallelX)) {
mergeParallelX <- maptools::spRbind(mergeParallelX, parallelX[[j]])
}

mergeParallelX <- SpatialPixelsDataFrame(points = mergeParallelX, data = mergeParallelX@data)

rf.s_ppm <- mergeParallelX

rf.s_ppm$trend <- p.rf

rf.s_ppm$pred <- rf.s_ppm$trend + rf.s_ppm$var1.pred

rf.s_ppm$lower <- rf.s_ppm$pred - 1.645 * sqrt(rf.s_ppm$var1.var)
rf.s_ppm$upper <- rf.s_ppm$pred + 1.645 * sqrt(rf.s_ppm$var1.var)

writeGDAL(rf.s_ppm["pred"], fname = "./out10/s_ppm_parallel.tif", drivername = "GTiff", type = "Float32")

Sdata<-raster("./out10/s_ppm_parallel.tif")
plot(Sdata)

When the raster is generated, the image is distorted to swirls. I understand that this must be do to the way the analysis is cut, distributed, and merged; but I dont know what in the code is the cause nor the solution.

image of raster generated with parallel

image of raster generated without parallel

2
  • 1
    This question is probably not getting answered because: your example isn't reproducible - we don't have the data to simply copy and paste to run your code; your second plot is tiny and hard to see what it is; I can't see any plotting functions in your code so its not clear what you are actually plotting or how, or what the big black and white raster image is meant to be.
    – Spacedman
    Commented Oct 13, 2019 at 9:42
  • @Spacedman Thanks for the tips. I switched the plots for higher quality ones. I believe the visualization is more clear now. I also added the plotting code.. simple plot function. With the reproducibility, without being able to share the data, it's difficult to make this code more reproducible, however, i'll think about ways to do it.
    – dtg37
    Commented Oct 15, 2019 at 0:50

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.