Memory allocation error encountered when automating gridmetrics in R

Question

I am trying to run Fusion's GridMetrics from R, I wrote this for one las file:

#First set up directories to keep the raw data 
#separate from that produced by the analysis, Input (.las) and (.dtm), 
#and output (.csv) directory in a dropbox folder.  
mainDir <- "F:/Dropbox/LiDAR_Prj"
inDir <- "inputDirectory"
outDir<- "outputDirectory"
dir.create(file.path(mainDir, outDir), showWarnings = FALSE)
#Run grid metrics in FUSION
#Read in the .las files and use FUSION to produce the .csv files with values for 14 m pixel.

system(paste(file.path("C:","Fusion", "gridmetrics.exe"),
"/outlier:-5,250 /minht:1.64 /strata:8.2,16.40",
file.path(mainDir, inDir, "*.dtm"), 13.1,45.93,
file.path(mainDir, outDir, "2H8546.csv"),
file.path(mainDir, inDir, "2H8546.las"),
sep=" "))

It worked very well. However, I would like to automate it because I have more than 400 tiles to process (file names are not a sequence).

I attempted to use a txt file listing the data files instead of the file names:

mainDir <- "F:/Dropbox/LiDAR_Prj"
inDir <- "inputDirectory"
outDir<- "outputDirectory"
dir.create(file.path(mainDir, outDir), showWarnings = FALSE)

#Run grid metrics in FUSION
#Read in the .las files and use FUSION to produce the .csv files with values for 14 m pixel.
system(paste(file.path("C:","Fusion", "gridmetrics.exe"),
"/outlier:-5,250 /minht:1.64 /strata:8.2,16.40",
file.path(mainDir, inDir, "*.dtm"), 13.1,45.93,
file.path(mainDir, outDir, "halistcsv.txt"),
file.path(mainDir, inDir, "halist.txt"),
sep=" "))

but a warning message pops up:

Too many points (couldn't allocate memory for point list)...no metrics computed
Run finished: Tue Nov 22 19:39:35 2016 (elapsed time: 21 seconds)
***There were errors during the run
Done
Warning message:
running command 'C:/Fusion/gridmetrics.exe ... halist.txt' had status 5

Could any of you help me to figure out an efficient way to automate this process?

Answer 1

The issue you are trying to solve has nothing to do with R. You are running application that is independent of R with the system() function.

The error that you got originates in program gridmetrics.exe. You can verify that by just printing the outcome of your paste() function and running that through command line. You should get the same error message as you are getting in R.

While I don't know the program gridmetrics.exe, I assume that the problem is that in your second example you are trying to run the program against a file that lists all your files instead of running the program for each file separately.

Assuming that your file "halist.txt" contains one filename per line, then you need to use this code:

mainDir <- "F:/Dropbox/LiDAR_Prj"
inDir <- "inputDirectory"
outDir<- "outputDirectory"
dir.create(file.path(mainDir, outDir), showWarnings = FALSE)

# path to the file with list of files
list_path = file.path(mainDir, inDir, "halist.txt")
# read the file as csv
files = read.csv(list_path, header = FALSE)


for (i in 1:nrow(files)){
  # split the file name by dot
  file_name = strsplit(files[i,1], "[.]")

  system(paste(file.path("C:","Fusion", "gridmetrics.exe"),
"/outlier:-5,250 /minht:1.64 /strata:8.2,16.40",
file.path(mainDir, inDir, "*.dtm"), 13.1,45.93,
file.path(mainDir, outDir, paste0(file_name[1],"csv.txt")),
file.path(mainDir, inDir, files[i,1]),
sep=" "))
}

This code will create one csv file for each file in your list. If you need to merge those files together it might be problematic due to the size.