Error occurred while writing dirty block from GDALRasterBand::IRasterIO

Question

I get the following error when trying to run rio-terrain on a GeoTIFF file :

$ rio aspect input.TIF output.TIF
Running aspect on sequential raster blocks
Blocks done:  [#################################---]   93%  00:00:30
Traceback (most recent call last):
  File "rasterio/_io.pyx", line 1389, in rasterio._io.DatasetWriterBase.write
  File "rasterio/shim_rasterioex.pxi", line 142, in rasterio._shim.io_multi_band
  File "rasterio/_err.pyx", line 182, in rasterio._err.exc_wrap_int
rasterio._err.CPLE_AppDefinedError: An error occurred while writing a dirty block from GDALRasterBand::IRasterIO

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/rio", line 8, in <module>
    sys.exit(main_group())
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/zenaterra/.local/lib/python3.6/site-packages/rio_terrain/cli/aspect.py", line 77, in aspect
    dst.write(result.astype(profile['dtype']), 1, window=write_window)
  File "rasterio/_io.pyx", line 1391, in rasterio._io.DatasetWriterBase.write
rasterio.errors.RasterioIOError: Read or write failed. An error occurred while writing a dirty block from GDALRasterBand::IRasterIO

Although input.TIF didn’t seem corrupted, I downloaded it again but it did not help.

What can I do ?

Answer 1

This error may occur because you are reaching the TIFF format 4 GiB size limit. A workaround is to use BIGTIFF files, keeping in mind that Big TIFFs are not globally supported outside of the GDAL software community.

Although the rio command line of rasterio has a --co option to provide GDAL GeoTiff creation options and could be used as (not tested)

rio command --co bigtiff=is_safer input.tif output.tif

this option is currently not supported by rio-terrain (as of 0.0.28), and the environment variable solution suggested here doesn’t seem to work either.

For now a workaround is to add the following line to the rio-terrain source files (e.g. /home/username/.local/lib/python3.6/site-packages/rio_terrain/cli/slope.py) after profile = src.profile :

profile.update(BIGTIFF = "IF_SAFER")

(or "YES" or "IF_NEEDED", depending on your needs).