2

I have a list of 420 rasterLayer .tif files that I need to create a rasterBrick from.

I first create a stack:

library(raster)
rList <- list.files(pattern = ".tif")
rasterStack <- stack(rList)

Which is quick, but when I try make a brick from the stack, it takes really long.

rasterBrick <- brick(rasterStack)

I'm currently working on a cluster. Is this a job for parallel computing? I tried the following:

library(parallel)

numCores <- detectCores()
rList <- list.files(pattern = ".tif")
stacked <- stack(rList)
rasterList <- mclapply(1:length(stacked), mc.cores = numCores, function(x) brick(stacked[[x]])) 

But I get the following error:

Error in seq.default(i, length(X), by = cores) :
  'by' argument is much too small
Calls: mclapply -> lapply -> FUN -> seq -> seq.default
Execution halted

I'm using all the cores available to me. Any suggestions please?

2

From the brick help:

"Yet they are less flexible as they can only point to a single file."

The raster package will not read your raster data into memory until it has to, so creating single-layer rasters and stacks from files can be fast since no large amounts of data are being read in - its just pointers to files plus some small metadata in the R object. Once you make a brick it has to read in the data.

If you really need to build a brick I don't think parallel will help you, since it will be slowed down by disk access, and you can't parallelise that away.

In any case, this code looks like its going to return a number of single-layer bricks, from each layer of stacked:

rasterList <- mclapply(1:length(stacked), mc.cores = numCores, function(x) brick(stacked[[x]])) 

but fails at least because length(stacked) doesn't return the number of layers - you probably want nlayers(stacked).

2
  • Thanks @Spacedman I understand now. So best would be to just read in the files, stack them, and then use them for my next step for which I need an output. I assume then that regardless of whether I create a brick or stack, if I need to save the stack/brick output using writeRaster, there is no real way to quicken the process? – Joanne Bentley Nov 6 '20 at 11:34
  • 1
    If storing the data in memory in a brick is going to speed up the rest of your work, then you take the hit in its construction. Otherwise work with the stack and take the hit at processing time. Can't say which is overall faster - do some tests on data subsets once you've got your processing chain complete. – Spacedman Nov 6 '20 at 11:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.