With lzw and deflate compression using -co predictor=2 can help with imagery that is smoothly varying as it compresses the differences from pixel to pixel instead of the absolute values, and these will tend to be small and have more patterns (ref). Predictor is only useful with lzw and deflate compression, the option has no effect with other methods.
gdal_translate -co compress=lzw -co predictor=2 ...
There is conflicting info on the differences between predictors 2 & 3 and when each is best applied (ref1, ref2). Perhaps fuel for another question.
To build on @alfonx's answer of using compressed overviews: This allows the base image to be stored lossless, for data integrity, and the pyramids to be lossy, for speed and some space savings. It's almost the best of both worlds. For the smallest possible overviews with gdaladdo on RGB images: use jpeg compression, averaged resampling instead of the default nearest neighbour (makes the overviews smoother), and YCBCR photometric overview. See the gdaladdo reference page for more info on these options (though it doesn't say much about what photometric is all about).
This is part of a windows batch file I use to apply external jpeg overviews to all tiffs in a directory:
set _opts=-r average --config PHOTOMETRIC_OVERVIEW YCBCR
set _opts=%opts% --config INTERLEAVE_OVERVIEW PIXEL --config COMPRESS_OVERVIEW JPEG
for %%a in (*.tif) do gdaladdo -ro %_opts% %%a 2 4 8 16 32 64`