How to write BIGTIFF raster files with rasterio

Question

A few months ago I had a hard time in finding a way to read BIGTIFF files. In the end I only managed to do so by creating a new conda environment with the following options:

conda create -n env_name -c conda-forge gdal vs2015_runtime=14

I must admit I have no idea what that instruction does (vs2015_runtime=14???), but it did the job. Now I am simply trying to merge a number of tif tiles into one unique tif raster file. I am using the code below:

...
...

#Merge tiles
mosaic, out_trans = merge(tiles)

#Update metadata
out_meta.update({"driver": "GTiff",
                 "height": mosaic.shape[1],
                 "width": mosaic.shape[2],
                 "nodata": -1,
                 "transform": out_trans,
                 "compress": 'lzw'
                })

#Output path
out_path = '...'

#Build mosaic
with rasterio.open(out_path, "w", **out_meta) as dest:
    dest.write(mosaic)
dest.close()

This code has actually worked fine on other occasions when trying to merge tif tiles, but this time it's giving me the following error messages:

ERROR 1: .\GB_OUTPUT\ALR\GB_COMBINED_ALR.tif, band 1: An error occurred while writing a dirty block from FlushCache
ERROR 1: TIFFAppendToStrip:Maximum TIFF file size exceeded. Use BIGTIFF=YES creation option.
ERROR 1: TIFFAppendToStrip:Maximum TIFF file size exceeded. Use BIGTIFF=YES creation option.
ERROR 1: TIFFAppendToStrip:Maximum TIFF file size exceeded. Use BIGTIFF=YES creation option.
ERROR 1: TIFFAppendToStrip:Maximum TIFF file size exceeded. Use BIGTIFF=YES creation option.

There are 2 errors there. One BIGTIFF related, and another one about some dirty block. Any idea why I can't create BIGTIFF files and how can I overcome the issue? Does the dirty block just mean there's a corrupt tif tile?

Answer 1

I tried with the following and it worked fine:

with rasterio.open(out_path, 
                   "w", 
                   **out_meta, 
                   tiled=True, 
                   blockxsize=256, 
                   blockysize=256, 
                   BIGTIFF='YES') as dest: