I have a possibility to do processing on a super computer, where task managing and resource allocation are controlled by SLURM (Simple Linux Utility for Resource Management) batch job system. However, I have not found the right configurations how to utilize the allocated resources with lidR efficiently. I have tried to allocate 20 CPU's to one task in SLURM and specified 20 workers for a multisession with Future-package within R script. After running a process for a short while, using LAScatalog processing engine, CPU efficiency statistics suggested that with these settings only one of the CPU's was used. Slurm bash script presented below

#SBATCH --job-name=pointsToRaster
#SBATCH --account=project_num
#SBATCH --time=00:05:00
#SBATCH --output=output_%j.txt
#SBATCH --error=error_%j.txt
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
#SBATCH --cpus-per-task=20
#SBATCH --mem-per-cpu=15G
#SBATCH --partition=hugemem


#load module
module load r-env-singularity

# Bind threads to individual cores
export OMP_PROC_BIND=true

#Run script
srun --threads-per-core=1 singularity_wrapper exec Rscript --no-save pointClouds.R

This bash script allocates resources and executes script pointClouds.R. Script reads in 30 .las files, containing point clouds produced with SFM-MVS photogrammetric methods. File sizes vary between 1Gt to 5Gt, and they are missing ground classification. First step is to classify groud points. Script content presented below.

#load packages



#Set working directory
setwd(dir = "/scratch/project_num/lasFiles")
filePaths = list.files(pattern = "./*las")

# Parallelization settings:
plan(multisession, workers = 20L)

#Read unclassified point clouds to a LAS-catalog object
pointCat = readLAScatalog(filePaths)

#### CLASSIFY GROUND POINTS ############

#Progressive Morphological Filter-algorithm 
opt_output_files(pointCat) = "./outputs/classified_{ORIGINALFILENAME}" 

ws  = seq(3, 12, 3) 
th  = seq(0.1, 1.5, length.out=length(ws))
groundClassified = lasground(pointCat, algorithm = pmf(ws, th))

Tried changing the setting around determining 20 tasks per node and one CPU per task. This setting raised CPU utilization, but when looking at the "process outputs" -textfile, it shows that every part of the code was executed 20 times (i.e. every package was loaded 20 times). I am not sure are is the problem related to the bash or R-script.

Any help on how to correctly allocate resources with a SLURM bash script to achieve efficient parallel processing with lidR LAScatalog?

  • i know nothing about SLURM, thus your question is unclear to me even if it looks like a good question. Please try to explain with more details what you are trying to achieve. And please provide details on the content of pointClouds.R. My understanding it that you have a big machine with many CPUs and you want to reserve 20 of them for your job. In R script you set workers = 20 to use these 20 CPU. Is that correct?
    – JRR
    Oct 23, 2020 at 10:27
  • To me this looks like a question for stackoverflow or maybe a linux specialized forum. This question is related to SLURM and future and does not seems to be actually related to lidR and GIS. Waiting for clarifications to see.
    – JRR
    Oct 23, 2020 at 10:30
  • You understood the problem correctly. I added information regarding the R script. First step would be to classify ground points in number of .las -files. I am not quite sure should I modify the bash or R script, to be able to efficiently utilize the LAScatalog. Oct 23, 2020 at 13:12

2 Answers 2


Issue was found by HPC service provider. For unknown reason OMP_PLACES=cores variable which should tie threads/processes to specific cores, appeared to bind all processes to a single core only when running multi-core R jobs. Issue has been solved by rebuilding r-environment singularity-container.


Your R script is correct. I think you should better use plan(multicore) since your are running the code on linux but this is not the issue. I think your issue is not related to lidR but to future and SLURM

I don't have any expertise with SLURM and I don't believe you will find anybody to help you on this GIS forum. You should better ask a question decoupled of lidR on stackoverflow where you will find more people with more advanced skills about this kind of stuff. The code that drives the processing of multiple files roughly looks like that. This will help you to make some tests yourself and ask a more focused question to more qualified people.


custom_apply = function(n = 10)
  f = vector("list", n)
  for (i in 1:n)
    f[[i]] = future({
      # do something computationally demanding
      # here I only sleep for a random delay
      d = runif(1, 0, 1)
      return (d)

ti = Sys.time()
Sys.time() - ti
#> Time difference of 6.39 secs

plan(multisession, workers = 4)

ti = Sys.time()
Sys.time() - ti
#> Time difference of 2.51 secs
  • Thank you for your input. I will post the question on stackoverflow page. Oct 26, 2020 at 10:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.