Rasterio creating composite/stack much larger than the individual bands

Question

I am using rasterio to create a stack of eight bands and each individual band is about 1GB, but when it's stacking them the output (with a lzw compression) is a tif that's over 53GB! This seems huge to me. Anyone know if this is normal?

def stack_bands(list_of_rasterio_bands, temp_dir, outputname, output_ext):
    # this will stack bands in the order they are in the list, returns a rasterio object of what was created
    print("[INFO] creating {}".format(outputname))

    composite_path = os.path.join(temp_dir, outputname+output_ext)

    arr_stack = []
    for band in list_of_rasterio_bands:
        band_arr = band.read(1, masked=True)
        arr_stack.append(band_arr)

    out_meta = list_of_rasterio_bands[0].meta.copy()
    out_meta.update({"count": len(list_of_rasterio_bands),
                     "nodata": -10000,
                     "compress": 'lzw'})

    with rasterio.open(composite_path, "w", **out_meta, BIGTIFF="IF_SAFER") as dest:
        for band_nr, src in enumerate(arr_stack, start=1):
            dest.write(src, band_nr)
    print("[INFO] {} saved".format(outputname))

    return rasterio.open(composite_path)

Answer 1

Check what is the resolution of your data (Uint8, Uint16, Float32, etc). If the original files are Uint8 and when passing through your function they are converted to Float32 (some libraries do that), that would explain such huge file.

Run gdalinfo on the 1 GB rasters and see the band type. Then check the type the band when opened with rasterio io (band_arr inside the for loop).

Hope that helps.

J.