1

I need to normalize, eliminate outliers and calculate metrics for a big number of .LAS data of a large area by using as much possible the package LidR. The problem comes when I try to run this part of a code I made that works great for a small number of .LAS data:

norm <- readLAScatalog("C:/JAIME/00_tests/norm/") #directory to folder with already normalized .LAS files

#Basic function detect outliers is to measure the 90th percentile of height in 10 x 10-m pixels (area-based approach) and
#then remove the points that are above the 90th percentile in each pixel plus, for example, 20% (sensitivity = 1.2).
lasfilternoise = function(norm, sensitivity)
{
  p90 <- grid_metrics(norm, ~quantile(Z, probs = 0.90), 10)
  norm <- lasmergespatial(norm, p90, "p90")
  norm <- lasfilter(norm, Z < p90*sensitivity)
  norm$p90 <- NULL
  return(norm)
}

#Read each LAS file of the LAScatalog called norm
las_norm <- readLAS(norm)

#Elimination of OUTLIERS in normilized LAS

norm_denoised <- lasfilternoise(las_norm, sensitivity = 1.2)
norm_denoised <- lasfilter(norm_denoised, buffer == 0) # Don't forget to remove the buffer

#Export of normalized LAS witout outliers as an only LAS file
writeLAS(norm_denoised, "C:/JAIME/00_tests/norm_p90/norm_p90.las")

#THEN CALCULATE METRICS FOR norm_p90.las

The problem comes when reading as a LAS the LAScatalog called norm:

#Read each LAS file of the LAScatalog called norm
> las_norm <- readLAS(norm)
Error: cannot allocate vector of size 6.9 Gb

It seems like the .LAS file of the total area is too big for my computer (16Gb of RAM) as I have read in other questions. So I tried to eliminate the outliers from the normalized .LAS files one by one with this code:

norm <- readLAScatalog("C:/JAIME/00_tests/norm/") #directory to folder with already normalized .LAS files

lasfilternoise = function(norm, sensitivity)
{
  p90 <- grid_metrics(norm, ~quantile(Z, probs = 0.90), 10)
  norm <- lasmergespatial(norm, p90, "p90")
  norm <- lasfilter(norm, Z < p90*sensitivity)
  norm$p90 <- NULL
  #Export of normalized LAS witout outliers as a LAS
  writeLAS(norm, "C:/JAIME/00_tests/norm_p90/{ORIGINALFILENAME}_p90.las")

  return(norm)

}

#Elimination of OUTLIERS in normilized LAS
norm_denoised <- lasfilternoise(norm, sensitivity = 1.2) 

The crypt works but the outputs are in GTiffformat and not in .lasas I need for calculating the metrics later. Is there any way to obtain the outputs of a grid_metricscommand in .las format, as I think it is the problem here?

I think it can be done as I found this example at CRAN.R where it is done and from which I based my code:

lasfilternoise.LAS = function(las, sensitivity)
{
  p95 <- grid_metrics(las, ~quantile(Z, probs = 0.95), 10)
  las <- lasmergespatial(las, p95, "p95")
  las <- lasfilter(las, Z < p95*sensitivity)
  las$p95 <- NULL
  return(las)
}
This function is fully functional on a point cloud loaded in memory

las <- readLAS("file.las")
las <- lasfilternoise(las, sensitivity = 1.2)
writeLAS(las, "denoised-file.las")

------------------------ EDIT ------------------------

After studying the documentation facilatated by @JRR in the comments to this questions, I finally managed to write a scrypt that eliminates outliers but do not works fine:

#Function to detect and eliminate outliers (>p90)
lasfilternoise = function(las, sensitivity)
{
  p90 <- grid_metrics(las, ~quantile(Z, probs = 0.90), 10)
  las <- lasmergespatial(las, p90, "p90")
  las <- lasfilter(las, Z < p90*sensitivity)
  las$p90 <- NULL

  return(las)
}

#Run the previous function for the catalog clusters
lasfilternoise.LAScluster = function(las, sensitivity)
{
  # The function is automatically fed with LAScluster objects
  # Here the input 'las' will a LAScluster

  las <- readLAS(las)                          # Read the LAScluster
  if (is.empty(las)) return(NULL)              # Exit early (see documentation)

  las <- lasfilternoise(las, sensitivity)      # Filter the noise

  las <- lasfilter(las, buffer == 0)           # Don't forget to remove the buffer

  if(is.empty(las)) return(NULL)              #If las is empty throw NULL 
  return(las)                                  # Return the filtered point cloud
}

#Directory with normalized LAS files
myproject <- readLAScatalog("C:/JAIME/00_tests/norm/", progress = TRUE)
plot(myproject)

#Options for the catalog engine
opt_filter(norm)       <- "-drop_z_below 0 -keep_first"
opt_chunk_buffer(norm) <- 10
opt_chunk_size(norm)   <- 0
opt_output_files(norm) <- "C:/JAIME/dataLAS_test/norm_p90/{ORIGINALFILENAME}_p90"

options <- list(
  need_output_file = TRUE,    # Throw an error if no output template is provided
  need_buffer = TRUE,         # Throw an error if buffer is 0
  automerge = TRUE)           # Automatically merge the output list (here into a LAScatalog)

#Apply functionto the catalog
output  <- catalog_apply(myproject, lasfilternoise.LAScluster, sensitivity = 1.2, .options = options)
return(output)

When applying this code to a normalized LAScatalog and then calculate the metrics (grid_metrics) for the resulting outliers filtered data, the results eliminates from the final raster what seems to be crops with heights close to 0 m as can be seen in the first picture, while if I run grid_metrics on the normalized data directly, the results are good. I have tried chnanging the NULLs for 0s but still. The outliers filter is doing something that I can not see enter image description hereneither how to solve it.

Grid resulting by applying the outliers filter

Grid resuting from not applying the outliers filter

6
  • 1
    You took the example from this vignette but you took only a third of the example. You can't expect it to work that way. Start by taking the whole example and if you still encounter issues, ask a question.
    – JRR
    Jan 11, 2020 at 15:25
  • 1
    And you should almost never call readLAS with a LAScatalog. The syntax is allowed for the case with 2 or 3 small files but it actually tries to load the full point cloud in memory. So indeed it crashed.
    – JRR
    Jan 11, 2020 at 15:28
  • Thanks for your responses. Exactly, that is the vignette that I looked up. I followed it and tried many scrypts based on it but due to my lack of experience in R and even programming I have not been able to come up with the proper one. The one that uses readLAS with a LAScatalog was the best approach I got but I was aware that reading such a big file was probably a bad idea. Jan 11, 2020 at 16:50
  • 1
    If you take the code from this vignette it should work. Here you took only a part of the example. I cannot help you more than by telling you to use the whole code from the vignette. If you do no understand the code you can ask for explanations of specific point you do not understand. But first you should read cran.r-project.org/web/packages/lidR/vignettes/… and the documentation ?LAScatalog-class, ?catalog_apply and maybe adv-r.had.co.nz/S3.html. lidR provides a powerful and versatile engine but it is indeed a little bit hard to use for beginners.
    – JRR
    Jan 11, 2020 at 17:24
  • Thank you for your advises and suggested documentation. It helped me. Although the scrypt seems to work it produces an unexpected error as I comment in the edited part of the question Jan 16, 2020 at 21:04

1 Answer 1

0

The proble was that I was not following the internal needed structure of catalog_apply. After studying the documentation facilatated by @JRR in the comments to this questions, I came up with a scryp that filters outliers. The code is in the edited part of the question.

1
  • Great question. Could you please include your edits as part of this answer rather than part of the question?
    – Aaron
    Jan 16, 2020 at 17:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.