gdal_rasterize errors

Question

I've got a geopackage which is a set of polygons covering Australia. I am trying to turn them into a raster using gdal_rasterize command. I get errors which I don't know how to solve.

The command I use is:

gdal_rasterize --config CHECK_DISK_FREE_SPACE NO -co "PREDICTOR=2" -co "COMPRESS=LZW" -of GTiff -co BIGTIFF=YES -co TILED=YES -co NUM_THREADS=ALL_CPUS -ot UInt16 -a my_layer -tr 0.000277777781132076 0.000277777781132076 -a_nodata 0 "c:/tmp/input.gpkg" "c:/tmp/output.tif"

The errors I get are a bit jumbled in the cmd line output, but seem to consist of mainly this repeated a few times

ERROR 1: Using code not yet in table
band 1: IReadBlock failed at X offset 496, Y offset 312
ERROR 1: TIFFReadEncodedTile() failed.

Does anyone have a suggestion to fix this?

Answer 1

For immediate help turn off the multithreading, so don't use -co NUM_THREADS=ALL_CPUS. If you can use GDAL from the current main branch it probably works reliably also with multithreading. See this announcement in the gdal-dev mailig list https://lists.osgeo.org/pipermail/gdal-dev/2023-October/057810.html.

For GDAL 3.6.0 to 3.7.2, use of multi-threaded GTiff compression+decompression, in particular within the same file, as for example in a "gdalwarp -co COMPRESS=... -co NUM_THREADS=..." workflow can lead to deadlocks (process stalled forever) and/or data corruption (sometimes without errors at generation). Cf https://github.com/OSGeo/gdal/issues/8470 for a reproducer. The fix is in https://github.com/OSGeo/gdal/pull/8561