Take the 2-minute tour ×
Geographic Information Systems Stack Exchange is a question and answer site for cartographers, geographers and GIS professionals. It's 100% free, no registration required.

I have a folder of GIS data that consists mainly of GeoTIFF files. The whole set weighs in at about 1.2 GB. I noticed that if I pack the contents into a tarball, it smashes down to about 82 MB. I would like to check the set into a revision control system sot it can be worked on by other people and it looks like there is some space that can be squeezed out.

The GDAL GeoTIFF driver page lists plenty of options that may be used to create compressed GeoTIFF files. There are also plenty of options that affect the way each algorithm works.

The help page does a good job at describing the options but doesn't elaborate on how to select an algorithm or the tradeoffs that are associated with the varying level of compression. This leads to the following questions:

  • The pros of using compression are a dramatic savings in space. What are the cons? Is information lost when the image is compressed?

  • How should one go about choosing an algorithm and compression level. Do some types of images lend themselves to a certain algorithm?

share|improve this question
add comment

6 Answers

up vote 36 down vote accepted

To select compression method you need to use a command like:

gdal_translate -co "COMPRESS=method" src_dataset dst_dataset

When you use compression biggest trade-off is extra processing time which is required to uncompress the image, and after uncompressing the image would still consume same amount of memory. About information loss there are two basic types of compression:

  • lossless - which preserve original data values
  • lossy - which degrade data to save even more space

You would lossless algorithms when original data values must be preserved, like DEMs, or raster features. Algorithms like PACKBITS, DEFLATE and LZW are lossless and can be ordered according compression ratio:

  1. LZW - highest compression ratio, highest processing power
  2. DEFLATE
  3. PACKBITS - lowest compression ratio, lowest processing power

Compression ratio still depends on data, if the data has a lot of similar values PACKBITS will yield good results.

Contrary to lossless you would use lossy algorithms like JPEG to compress rasters that don't have to return exact values. For instance, orthophotos or satellite imagery can be compressed using lossy algorithms.

share|improve this answer
1  
+1 for the nice answer. PACKBITS is a form of run-length encoding (en.wikipedia.org/wiki/Run-length_encoding) which will work well for data with lots of adjacent same values (if for example, you have lots of NULLs or a classified raster) and LZW is a more robust algorithm which is effective on more kinds of data. The general trade-off is between space and speed as mentioned, so what's appropriate depends on your use and data. Also, some software doesn't support certain kinds of GeoTiff compression. –  scw Aug 12 '10 at 19:04
2  
this is a good, relevant post linfiniti.com/2011/05/… –  oeon Jun 1 '11 at 5:01
    
Good answer, it summarizes your options well. Remember also that each of those compression methods has parameters you can set, which will influence the outcome considerably. @j03lar50n, glad you found my blog article useful ... –  R Thiede Sep 1 '11 at 20:34
add comment

With lzw and deflate compression using -co predictor=2 can help with imagery that is smoothly varying as it compresses the differences from pixel to pixel instead of the absolute values, and these will tend to be small and have more patterns (ref). Predictor is only useful with lzw and deflate compression, the option has no effect with other methods.

gdal_translate -co compress=lzw -co predictor=2 ...

The predictor savings can be dramatic. I just re-compressed a directory of 16bit geotiff elevation models using up 17GB with the default LZW settings into just 5GB with predictor=2.

There is conflicting info on the differences between predictors 2 & 3 and when each is best applied (ref1, ref2). Perhaps fuel for another question.

To build on @alfonx's answer of using compressed overviews: This allows the base image to be stored lossless, for data integrity, and the pyramids to be lossy, for speed and some space savings. It's almost the best of both worlds. For the smallest possible overviews with gdaladdo on RGB images: use jpeg compression, averaged or gaussian resampling instead of the default nearest neighbour (makes the overviews smoother), and YCBCR photometric overview. See the gdaladdo reference page for more info on these options (though it doesn't say much about what photometric is all about).

This is part of a windows batch file I use to apply external jpeg overviews to all tiffs in a directory:

set _opts= -r gauss --config PHOTOMETRIC_OVERVIEW YCBCR ^
--config COMPRESS_OVERVIEW JPEG --config JPEG_QUALITY_OVERVIEW 85

for %%a in (*.tif) do gdaladdo -ro %_opts% %%a 2 4 8 16 32 64

Notes

GDAL 1.6.0 introduced gauss resampling which can lead to better results average in case of sharp edges with high contrast or noisy patterns. Powers of 2 levels (2 4 8 ...) should be used so a 3x3 resampling Gaussian kernel is selected.

JPEG_QUALITY_OVERVIEW 85 - if not specified the default of 75% is used, which does yield smaller file, but I find 85% a better compromise in the size vs quality trade off.

share|improve this answer
add comment

For big rasters GeoTiff offers the possibility to store (pre-)downscaled overviews as extra images to the GeoTiff file. This can be done with gdaladdo (= GDAL ADD Overview). When creating these overviews, you can manually tell gdal to comress them too:

gdaladdo --config COMPRESS_OVERVIEW JPEG 

Speeds up viewing your data without adding too much size. Note: Geotools applications like Geoserver, uDig, AtlasStyler, Geopublisher can all use this feature and profit from overviews.

share|improve this answer
add comment

To enable partial image decompression, simply use TILED=YES .

Peter

share|improve this answer
add comment

My experience comparing GeoTIFF vs. Earth Resource Mapping's ECW (Enhanced Compressed Wavelet) compression is that ECW is orders of magnitude better when compressing high resolution aerial photos. Another important advantage of wavelet based compression is that, unlike older formats as GeoTIFF, JPEG - not JPEG 2000 -, just a portion of the image can be decompressed [ref. 1]. The importance of this advantage must not be underestimated, specially when working with "larger than about half computer memory size.".

It seems - I have never had the chance of testing it - that MrSID, another propietary, wavelet based file format, also exhibits higher compression ratios than "older" formats and selective decompression.

ref. 1: http://www.ifp.uni-stuttgart.de/publications/phowo01/Ueffing.pdf

share|improve this answer
    
dariapra, remember that GeoTIFF-Packbits or GeoTIFF-LZW are lossless compressions while ECW and JPEG are lossy. Lossless or lossy compression must be carefully chosen depending on future data usage. –  markusN Nov 16 '10 at 7:10
    
I am not claiming that a loosy compression format is always a valid storage format. What I wanted to mean is that using a format like ECW is suitable in some production environments. For instance, ECW is a more suitable format than GeoTIFF if we have a MapServer instance serving ortophoto layers via WMS. Nothing forbids that you also store the ortophoto using lossless compression. –  dariapra Nov 16 '10 at 12:04
add comment

Ultimately you'll probably need to experiment with the different options and see what meets your needs.

I've been making increased use of JPEG-compressed GeoTIFFs over wavelet-based formats. My results have been pretty good. Using GDAL to do this has yielded compression ratios comparable to wavelet-based formats without too much data loss. The performance hit that comes with decompression has been acceptable.

What I like most about this approach is that GeoTIFF support is almost universal, while support for wavelet-based formats isn't always assured and is sometimes subject to thorny licensing issues.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.