4

Background:
When writing .png rasters to disc using the terra R package, not only the intended raster is written, but also auxiliary files (.png.aux.xml and .png.aux.json files, depending on version of the package) with geographic information (since the PNG file standard does not support that).

Problem
Those files causes me a bit of a pain, since I'm writing many thousand small rasters and have no interest in the auxiliary information. Currently I am relying on code removing the newly written aux.xml and aux.json, but I believe this is a terribly inefficient solution (that, in the end, consumes quite a bit of time given the large amount of files). Further, the aux-files take up multiple times the disk space that my rasters do, so leaving the problem unaddressed is not really a good option.

Question
I cannot seem to find any option turning this feature off. Is there an option that I am missing? or any other ways to solve this?

Minimal reproducible example:

library(terra)
# Make raster:
test_raster <- rast(nrows=5, ncols=5, vals=1:25)

# Create a temporary folder and path to write to:
folder_path <- paste0(tempdir(),"\\rasters")
dir.create(folder_path, showWarnings = F)
  
# Write raster
writeRaster(test_raster, file.path(folder_path, "raster.png"), overwrite=TRUE)

# List files: (will show both .png and .png.aux.xml)
list.files(folder_path)

(The example works on my Windows machine, but I have not checked if the files paths are suitable for other systems.)

5
  • The page format is not spatial (no information in the header) so, the relevant spatial information needs to be stored in external files. If your problem is that you do not want those auxiliary files included in a file search just use list.files(getwd(), "png$") to limit returned files to those ending with the file extension. Commented Apr 4, 2022 at 12:51
  • Thanks for the comment. I see the intended use for the auxiliary files, and this is a very sensible default behaviour. However, they take up unnecessary disk space for me, so it is not just the problem of seeing them. I edit my question to clarify that.
    – Smerla
    Commented Apr 4, 2022 at 13:01
  • I think your additional question about Linux needs to be asked as a new question.
    – PolyGeo
    Commented Apr 6, 2022 at 12:45
  • We use a One question per Question model. Asking a question above a different question with an accepted answer does not correspond to this model. A Question should ask a question, not have an initial sentence talking about the answer. If you want to ask a different question, ask it in a different Question.
    – Vince
    Commented Apr 8, 2022 at 1:18
  • Although I think that is a sensible approach I believe it was a bit too strictly applied in this case. As a reference for future readers, I believe it is more useful to have information regarding the same problem in the same question. However, I have edited the question, and hopefully it will be re-opened to allow Robert Hijmans more detailed answer to be posted.
    – Smerla
    Commented Apr 8, 2022 at 14:10

2 Answers 2

4

There's a function in rgdal that can set the GDAL option to disable this - the option gets respected by terra's writeRaster apart from a little warning:

Nothing in the box:

> list.files()
character(0)

Set the option:

> rgdal::setCPLConfigOption("GDAL_PAM_ENABLED", "FALSE")
[1] "FALSE"

Write the raster:

> writeRaster(test_raster,"raster.png", overwrite=TRUE)
Warning message:
[rast] unknown extent

What's in the box?

> list.files()
[1] "raster.png"

Shazam, no auxiliary XML files.

I don't see a way to set this with anything in the terra package though, or to pass this option into writeRaster. The gdal= option only passes driver options, not GDAL CPL options as far as I can tell.

4
  • Thank you! This worked like a charm on a Linux server when I ran my example. However, I still get the aux-files (and no warning message regarding unknown extent, which I assume is an indicator that the setting is not properly stored) when i run it on a Windows system. It also does not work on the Linux server when I run parallell threads (regardless of setting it both before and after the parallelisation). But I suppose a lot of things can get strange then. I'll still flag your answer as correct, since it works. But do you have any idea of why it does not in the cases mentioned above?
    – Smerla
    Commented Apr 4, 2022 at 19:38
  • Possibly Windows has two different GDAL libs installed, one that rgdal is using, and one that terra is using. Not sure about the parallel system - if you want to create a new Q showing how you are doing things in parallel and how this fails I could look at it (there's several ways of doing parallel in R so need to see your code examples).
    – Spacedman
    Commented Apr 4, 2022 at 20:24
  • That would be great! Much appreciated. I will have to work a bit on an reproducible example, but will post a question tomorrow or so and let you know.
    – Smerla
    Commented Apr 4, 2022 at 20:46
  • I have now added a new question regarding the use on linux servers: gis.stackexchange.com/questions/428278/…
    – Smerla
    Commented Apr 7, 2022 at 15:10
2

With terra version >= 1.5-27 you can use setGDALconfig.

library(terra)
#terra 1.5.27

test_raster <- rast(nrows=5, ncols=5, vals=1:25)
folder_path <- paste0(tempdir(),"\\rasters")
dir.create(folder_path, showWarnings = F)

setGDALconfig("GDAL_PAM_ENABLED", "FALSE")
  
writeRaster(test_raster, file.path(folder_path, "raster.png"), overwrite=TRUE)
#Warning message:
#[rast] unknown extent
 
list.files(folder_path)
#[1] "raster.png"
1
  • This has been a handy solution for xml files etc, however how do I stop the .prj file being written, particularly for .asc outputs in writeRaster? Argument prj=F appears not to work in terra (unless i need to upgrade)
    – Sam
    Commented Mar 30, 2023 at 11:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.