Why is normalize_height() function aborting in lidR?

Question

I'm trying to run normalize_height() on 328 las files. Maybe or maybe not relevant: I'm not running using LAScatalog because I couldn't get that to run at all. Anyhow, I'm running using a parallelized foreach loop. Seems to work fine. 303 of the tiles were properly normalized and written to disk. For others I get this output (looping/parallelization not relevant):

library(lidR)
las <- readLAS("1715.las")
las <- normalize_height(las, tin())
#> Errors running normalize_height: "270 points not normalizable. Process aborted.

When I run las_check() on this file the suspicious results are:

• checking gpstime incoherences: 405091 pulses (points with the same gpstime) have points with identical ReturnNumber
• checking negative outliers: 177777 points below 0

My question is: how do I track down this problem? All I could think of was removing duplicates, which I did with no success. Do I need to remove the degenerated ground points too? If so, how to do that? The point cloud that triggered the error can be downloaded here and looks like:

Answer 1

As I suspected you have a dataset where ground points are unevenly distributed meaning you have many non ground points that are very far from actual ground points. For example the right part of the bridge. There is no ground point close to the bride so even without error the normalization will be weak.

Technically, the triangulation cannot interpolate outside the convex hull of the ground points. So everything that is not in the red polygon cannot be interpolated

Hopefully lidR extends the triangulation outside the convex hull defined by ground points to catch few points that may be slightly outside the hull by applying a nearest neighbor approach. But in your case you really have a massive amount of points outside the hull and particularly far from where the interpolation can actually be performed accurately. The algorithm failed at computing a ground elevation for 270 of those points. Without surprise those points are at the very edge on the right (in red)

I have reported a bug to improve that. You have several options

You can discard those points with na.rm = TRUE

las <- normalize_height(las, tin(), na.rm = TRUE)

You can use another interpolation method. While this won't fail it remains true that interpolation will be weak. In absence of ground points the interpolation is just a guess.

las <- normalize_height(las, knnidw())

You can use a larger buffer to catch more spatial context when processing a LAScatalog.

opt_chunk_buffer(ctg) <- 100
normalized <- normalize_height(ctg, tin())

By the way do not for loop on files. Doing so you do not load any spatial context at the edges of your tiles. Your DTM is incorrect and weak at the edges especially in your case with many missing ground points. Look at a DTM generated with a for loop on file on 4 files. You can see the edges of the files.

_{(source: github.io)}

This is why lidR has a LAScatalog processing engine. For loop on file should be almost always avoided. Instead region must be loaded with a buffer. lidR do that on-the-fly.