Using the following two commands yields completely different file sizes:
gdal_translate test.tif -of GTiff -co "COMPRESS=JPEG" -co "JPEG_QUALITY=100" output.tif
and
gdal_translate test.tif -of JPEG -co "QUALITY=100" -co "WORLDFILE=YES" output.jpg.
(I Wrote the commands from my memory, not sure that the syntax is correct)
The jpg file (second command) is 2 times smaller than the compressed tif file. Why?
By default an RGB image will be written to an RGB color model JPEG image, but this is not actually the most efficient way of writing to JPEG. It is better to convert to the YCbCr color space, and encode that. This is in fact the typical form of standalone JPEGs and what GDAL will produce when writing to a free standing JPEG file. Compressing a 4K x 2.6K image I see:
gdal_translate rgb.tif out.tif -co COMPRESS=JPEG
$ ls -l out.tif
-rw-rw-r-- 1 warmerdam warmerdam 7416934 Nov 4 10:09 out.tif
gdal_translate rgb.tif out_ycbcr.tif -co COMPRESS=JPEG -co PHOTOMETRIC=YCBCR
$ ls -l out_ycbcr.tif
-rw-rw-r-- 1 warmerdam warmerdam 2251264 Nov 4 10:09 out_ycbcr.tif
With YCbCr encoding the size of the output is pretty much the same as the JFIF JPEG image.
Warning 6: Driver GTiff does not support QUALITY creation option. is there such thing as a quality option? was able to get a 950mb file down to 198mb with @FrankWarmerdam's answer, wondering if I could get a little more with an option likeQUALITY