I work with large drone image raster files, on the order of 40,000 x 40,000 and above. I have a large uncompressed GeoTIFF file and I want to use
rasterio to rewrite the file in compressed format. I can do this by loading all data into memory, but is there a way to execute this writing without loading everything to memory?
For my original file I can open it with:
dat = rasterio.open("grid_001.tif")
Then to rewrite the file with compression I tried:
profile = dat.profile.copy() profile.update( compress='lzw') with rasterio.open("grid_001_compressed.tif", 'w', **profile) as dst: dst.write(dat)
This will give me an error:
ValueError: Source shape (44134, 44162) is inconsistent with given indexes 3
This is an expected error, because when I
open the dataset it creates an iterator or lazy object without actually accessing the data. Now, if I did a command like:
dat = dat.read()
This will load all of the data from the file into memory, and I can an array of dimensions
[3, 44134, 44162] that I can write. BUT, this takes a lot of memory to implement.
Hence, is there a way to perform the same operation, but without loading everything into memory? I am not sure if windowed reads would help in this case or anything.