2

Is there a way to find the optimal chunk size using opt_chunk_size() when using a lidR catalog?

I seem to always randomly choose integer values until the program quits throwing errors at me. For example, this is from the catalog_apply() lidR documentation (p.11):

## ===================================================
## Example 2: compute a rumple index on surface points
## ===================================================
rumple_index_surface = function(cluster, res)
{
las = readLAS(cluster)
if (is.empty(las)) return(NULL)
las <- lasfiltersurfacepoints(las, 1)
rumple <- grid_metrics(las, ~rumple_index(X,Y,Z), res)
bbox <- raster::extent(cluster)
rumple <- raster::crop(rumple, bbox)
return(rumple)
}
LASfile <- system.file("extdata", "Megaplot.laz", package="lidR")
project <- readLAScatalog(LASfile)
opt_chunk_buffer(project) <- 1
opt_chunk_size(project) <- 120 # small because this is a dummy example.
opt_select(project) <- "xyz" # read only the coordinates.
opt <- list(raster_alignment = 20) # catalog_apply will adjust the chunks if required
output <- catalog_apply(project, rumple_index_surface, res = 20, .options = opt)
output <- do.call(raster::merge, output)
plot(output, col = height.colors(50))
0

1 Answer 1

2

The control of the chunk size intends mainly to enable the processing of big files on computers with a small configuration. You don't have a lot of RAM but you need to process high density file? You can process quarters of the files.

The default is 0 meaning that you process by file. I rarely change that myself except in the documentation for the need of the examples. You are not expecting to encounter trouble with other values except maybe for very small and irrelevant ones (e.g chunks of 20 m in grid_metrics with a resolution of 20 m).

However there is one case where it can matters: exponentially complex methods. For example the li2012 method is quadratically complex with the number of points meaning you will process faster 4 quarters of a file than the whole file.

3
  • Is there any way to quantitatively determine which size to use in complex methods? For example Total_Size/4? Also, is there any way to know which methods are complex and require chunks?
    – Aaron
    Dec 12, 2019 at 16:06
  • 1
    No. This definitively not trivial. And there is no way to know the complex methods because it is not necessarily obvious even for me. I'd say that li2012() is computed in 0(n²) or more. TIN-based methods are computed in 0(n log n) according to the state of the art. Other methods are computed in 0(n) or something not too bad. li2012() has been added a long time ago but now I would not have added it because of its complexity that makes it hardly computable on broad coverage.
    – JRR
    Dec 12, 2019 at 16:38
  • 1
    In lidRplugins you will find more "problematic" algorithms not added in lidR because of the computation time. And yet I'm sure I could linearize li2012 one day or another.
    – JRR
    Dec 12, 2019 at 16:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.