0

Background:
When writing rasters using formats that do not support geolocation (for example .png), the Terra package saves auxiliary files containing this information. This can be prevented by modifying a setting of rgdal:

rgdal::setCPLConfigOption("GDAL_PAM_ENABLED", "FALSE")

(Works for Linux but not for Windows. For details, see the SE question: Preventing terra from writing auxiliary files when writing to disc).

Problem:
This setting turns out to be unreliable in parallelised workflow (using foreach and doParallel).

If a raster is defined within the foreach call, the auxiliary files will be skipped, as intended.

But if a raster is defined outside the foreach call the auxiliary files are still produced. Note that for a raster defined outside foreach has to be "packed" using terra::wrap() and turned into a raster again using terra::rast(). I have tried to include rgdal::setCPLConfigOption("GDAL_PAM_ENABLED", "FALSE") outside the foreach()in case the wrapped raster inherits the rgdal settings during wrapping. But this seems to have no effect.

Question:
Is there any way to get the setting to apply for rasters defined outside foreach()?

Reproducible example:

library("doParallel") 
library("foreach")  

folder_path <- paste0(tempdir(),"/raster/")
dir.create(folder_path)
list.files(folder_path)

cores <- makeCluster(1) # Set number of cores 
registerDoParallel(cores) 
on.exit(stopCluster(cores))


# Set GDAL to not produce auxiliary files when writing png-files.
rgdal::setCPLConfigOption("GDAL_PAM_ENABLED", "FALSE") # Does not have any effect when placed outside.

# Make raster:
test_raster <- terra::rast(nrows=5, ncols=5, vals=1:25)
temp_rast_pack <- terra::wrap(test_raster) # Wrap to transfer to the dopar environment


dopar_test <- function(folder_path, temp_rast_pack) {
  
  # Parallel processing
  foreach(i = 1:1) %dopar% {
    
    library("terra")
    
    test_raster_1 <- terra::rast(temp_rast_pack) # Unpack raster that was wrapped
    
    rgdal::setCPLConfigOption("GDAL_PAM_ENABLED", "FALSE") # Set mode to avoid the rendering of auxiliary files.
    
    
    # Write raster
    terra::writeRaster(test_raster_1, filename = paste0(folder_path, "wrapped_outside.png"), overwrite=TRUE)
    
    test_raster_2 <- terra::rast(nrows=5, ncols=5, vals=1:25)
    terra::writeRaster(test_raster_2, filename = paste0(folder_path, "defined_within.png"), overwrite=TRUE)
    rm(test_raster_2) # Hack to prevent foreach from throwing an error.
  }
}

# Run function
dopar_test(folder_path, temp_rast_pack)

# List files
list.files(folder_path)

My output is showing the following:

[1] "defined_within.png" "wrapped_outside.png" "wrapped_outside.png.aux.json"

2 Answers 2

1

Here is how you can suppress the .aux.xml files that GDAL creates

folder_path <- paste0(tempdir(),"\\rasters")
dir.create(folder_path, showWarnings = F)

library(terra)   
#terra 1.5.27
test_raster <- terra::rast(nrows=5, ncols=5, vals=1:25)
temp_rast_pack <- terra::wrap(test_raster) 

library(foreach)
library(doParallel)
registerDoParallel(cores=2)
dopar_test <- function(folder_path, temp_rast_pack) {
  foreach(i = 1) %dopar% {
    library(terra)
    setGDALconfig("GDAL_PAM_ENABLED", "FALSE")   
    test_raster_1 <- rast(temp_rast_pack) # Unpack raster that was wrapped
    writeRaster(test_raster_1, filename = file.path(folder_path, "wrapped_outside.png"), overwrite=TRUE)    
    test_raster_2 <- rast(nrows=5, ncols=5, vals=1:25)
    writeRaster(test_raster_2, filename = file.path(folder_path, "defined_within.png"), overwrite=TRUE)
  }
  invisible()
}

dopar_test(folder_path, temp_rast_pack)
list.files(folder_path)
#[1] "defined_within.png"           "wrapped_outside.png"          "wrapped_outside.png.aux.json"
 

The aux.json file is directly created by terra and not affected by the GDAL configuration setting. terra creates it to save units and time stamps, but there are none in this raster and it should not be there in this case. This has now been fixed in the development version.

0

Answer moved to previous question after that was reopened

2
  • Thankt you! When I updated the package it worked. However, I have written to the moderators to re-open my previous question. I hope that you eventually can post this answer there. I will not up vote it for the time being as it seems not to help with the question regarding parallelising.
    – Smerla
    Commented Apr 8, 2022 at 14:57
  • The other question is reopened now, so if you could provide this answer there I would be happy to up vote it!
    – Smerla
    Commented Apr 9, 2022 at 11:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.