I have a GeoTIFF image, size 11gb. It's too big to work with. How can I compress it in JPEG format with QGIS? I tried through GDAL and did not succeed.
Try to use BigTIFF in your operation.
gdal_translate -co BIGTIFF=YES -co compress=JPEG input.tif output.tif
it would help if you amended your quesiton to show us your GDAL script.
A simply go to Raster/Conversion/Translate. Then in the Converted field go to "..." - Save to file, Choose: JPG or JP2