GRASS r.in.lidar tool generates incomplete rasters

Question

I have been trying to import a huge LIDAR file (10.5) Gb to GRASS GIS using r.in.lidar. It does the task, but only for part of the image, as shown below:

Canopy Height Model (CHM) generated with GRASS:

CHM generated with FUSION:

The DEM, DSM and CHM generated in GRASS were built from the following commands, respectively:

r.in.lidar -o input=/my/route.las output=lidar_min method=min
r.in.lidar -o input=/my/route.las output=lidar_p99 method=percentile pth=99
r.mapcalc "CHM = lidar_p99@ig - lidar_min@ig"

All the three layers are incomplete, only some parts of the image are processed. I have tried different solutions: importing the files that I fused for creating the big one, importing files that had been created using FUSION (a file with only the ground points). I tried exporting the above rasters to geotiff and open them in QGIS, but the result is the same.

The only thing that worked is converting the LIDAR file to ASCII (only the xyz columns), and then importing it with r.in.xyz.

r.in.xyz input=/my/route.txt output=lidar_min separator=comma method=min

and it worked. This is the DEM produced using r.in.xyz from GRASS.

Then, the data are complete.

I think one possible reason behind r.in.lidar is not working as expected might be that GRASS only imports the pulses that are on the nadir or very near to it or only some passes of the flight. Does somebody know the reason behind this behavior?

_{If it helps, this was the command line used in Fusion to generate the CHM that worked:
CanopyModel /outlier:0,20 /ground:D:\DTM_default.dtm /ascii D:\CHM_default.dtm 5 m m 0 0 0 0 D:\laslist.txt}

Sample

I uploaded a small sample of the data here (https://goo.gl/0S9p0l). It contains a file with only the xyz coordinates (.txt), other with the whole table (x,y,z, intensity, scan angle, etc.) (.las) and other two files that are DSMs created from the point files (.txt and .las).

Answer 1

The inconsistency is caused by invalid points in the dataset. When I run (libLAS) lasinfo on the provided sample, I get:

1170966 Unclassified (1) 
1565144 Ground (2) 

But when I execute:

las2las -i sample.las --valid_only -o sample2.las

I get much lower number of points:

448086 Unclassified (1) 
633077 Ground (2)

which is exactly what r.in.lidar reports with the original dataset since it uses only valid points:

...1081163 points found in region.

v.in.lidar is little bit more verbose and says:

1081163 points imported
1654947 input points were not valid

Both use LASPoint_IsValid() function from libLAS C API to determine point validity (so they consider invalid whatever libLAS considers invalid). Validity in libLAS depends on some ASPRS LAS validity definition. And it may be related to large scan angles according to libLAS documentation. It doesn't seem that las2las would be able to change the validity of points.

Please, open a ticket at GRASS GIS Trac if you think this is a bug or if you have some suggestion for enhancement.

Answer 2

I had this exact same problem with a small dataset (16 mil. points out of which GRASS was labeling as invalid about 30%). Luckily I also had another dataset that imported fine in GRASS so I compared them, and as wenzeslaus suggested, the only notable difference was that the second dataset had no scan angle information stored (i.e. all values were 0). This problem was not solved in the latest GRASS version 7.0.4. Release Candidate 1 they released today (19 April 2016) so in the meantime I suggest the following workaround:

Use the las2las free tool to generate a new .las dataset, like this:

las2las -i input_file_name.las -o output_file_name.las -scale_scan_angle 0.01

The last parameter will scale (or divide) the scan angle values by a factor of 0.01, so all the points will be imported in GRASS. Of course, if you need the actual scan angles for your further processing in GRASS this is still not a solution, but it works for now.