File size inflation using GDAL

Question

New to GDAL, have some tiff files and I'm trying to do the following:

"0 to NoData" to avoid black background in Mapbox:

gdalwarp -dstnodata 0 -dstalpha [input file] [output file]

The problem is, the output file size for this tasks is huge (≈150MB to ≈10GB).

The same thing occurs when I next add a color relief:

gdaldem color-relief -of GTiff [input file] [color text file] [output file]  -alpha

The file size roughly doubles...

My only guess is that the data is remaining as a value and is somehow being interpreted as such, so my only option is to compress the file using LZW afterwards. However, this process takes a ridiculously long time to complete, so I'd like to find a more efficient way. I typically don't see such great file inflation, but this dataset is new to me.

EDIT:

Here is the gdalinfo readout...

Driver: GTiff/GeoTIFF
Files: [input file]
Size is 129753, 78920
Coordinate System is:
PROJCS["NAD_1983_Albers",
    GEOGCS["NAD83",
        DATUM["North_American_Datum_1983",
            SPHEROID["GRS 1980",6378137,298.2572221010042,
                AUTHORITY["EPSG","7019"]],
            AUTHORITY["EPSG","6269"]],
        PRIMEM["Greenwich",0],
        UNIT["degree",0.0174532925199433],
        AUTHORITY["EPSG","4269"]],
    PROJECTION["Albers_Conic_Equal_Area"],
    PARAMETER["standard_parallel_1",29.5],
    PARAMETER["standard_parallel_2",45.5],
    PARAMETER["latitude_of_center",23],
    PARAMETER["longitude_of_center",-96],
    PARAMETER["false_easting",0],
    PARAMETER["false_northing",0],
    UNIT["metre",1,
        AUTHORITY["EPSG","9001"]]]
Origin = (-2353941.226999999955297,3160112.077999999746680)
Pixel Size = (30.000000000000000,-30.000000000000000)
Metadata:
  AREA_OR_POINT=Area
  DataType=Generic
Image Structure Metadata:
  COMPRESSION=LZW
  INTERLEAVE=BAND
Corner Coordinates:
Upper Left  (-2353941.227, 3160112.078) (127d48'20.50"W, 47d51'28.35"N)
Lower Left  (-2353941.227,  792512.078) (119d57'48.72"W, 27d30'44.36"N)
Upper Right ( 1538648.773, 3160112.078) ( 74d45'30.86"W, 49d54'18.35"N)
Lower Right ( 1538648.773,  792512.078) ( 80d 8'39.87"W, 29d 2'53.17"N)
Center      ( -407646.227, 1976312.078) (100d52' 1.48"W, 40d42'15.31"N)
Band 1 Block=128x128 Type=Byte, ColorInterp=Gray
  NoData Value=0
  Metadata:
    RepresentationType=THEMATIC
  Image Structure Metadata:
    NBITS=2

Answer 1

Your existing data is compressed. You're getting an increase in file-size because the default is uncompressed and you aren't specifying a compression option (-co compress=LZW).

However, it's a longstanding issue that gdalwarp doesn't deal with compression well. The solution is to gdalwarp without compression then gdal_translate with compression.

To avoid two lengthy processes, gdalwarp to VRT first, it's really quick, then gdal_translate with the -co COMPRESS=LZW option.

i.e.

gdalwarp params... -of vrt input_file.tif output_file.vrt
gdal_translate -co compress=LZW output_file.vrt output_file.tif

If using GDAL 2+ you can combine this into a single operation by writing the VRT to /vsistdout/ and piping that to gdal_translate and specifying /vsistdin/ as the input. For example:

gdalwarp -q params... -of vrt input_file.tif /vsistdout/ | gdal_translate -co compress=lzw  /vsistdin/ output_file.tif

You should be able to simply specify -co COMPRESS=LZW in your gdaldem color-relief command, i.e

gdaldem color-relief -co COMPRESS=LZW -of GTiff [input file] [color text file] [output file]  -alpha