How are GeoTIFF pyramids/overviews standardised?

Question

While pyramids/overviews are not part of the GeoTIFF standard, many tools support creating them. For example vips/nip2, Orfeo Toolbox (otb) and ossim which all promise some support for creating those. But I have no idea if they would all create files in a format that the others support. Looking through the documentation of the tools does not really mention anything about that.

vips is not a geospatial tool and I could not find any enduser-friendly documentation but the IIPImage docs mention its support for "Tiled Pyramidal TIFF" : http://iipimage.sourceforge.net/documentation/images/

otb does not mention anything about a format or spec for its "Multi Resolution Pyramids": https://www.orfeo-toolbox.org/CookBook/CookBooksu65.html

ossim says it supports various output formats for its "reduced resolution data sets" but I don't know what they actually mean: https://trac.osgeo.org/ossim/wiki/img2rr

GDAL also does not really specify things about its "overview images": http://www.gdal.org/gdaladdo.html

So they all have pyramids/overviews but it is not clear if they are cross-compatible.

On more general pages I found the following quotes:

http://iipimage.sourceforge.net/documentation/images/ says

Tiled Multi-Resolution (or Tiled Pyramidal) TIFF is simply a tiled multi-page TIFF image, with each resolution stored as a separate layer within the TIFF. This is a standard TIFF extension and is supported by most image processing applications including Photoshop, GIMP, VIPS and ImageMagick. The libtiff codec library is also perfectly capable of reading and writing such images.

Is that what everyone uses?

The Library of Congress also has some info: http://www.digitalpreservation.gov/formats/fdd/fdd000237.shtml They note:

Pyramid TIFF files created by different applications are not necessarily the same in structure. In particular, judging from analysis with JHOVE and the identify command in ImageMagick, Adobe's Photoshop and Image Magick generate files with different internal TIFF structures; in both cases, most software that can handle TIFFs appears to recognize the primary (full-size) TIFF without problem.

---

So, are those formats standardised and specified and documented somewhere? How can I find out which tool can produce them in compatible ways to others? Are there any geospatial tags in overviews/pyramids or could I use any software to create them on the image data?

Answer 1

Short answer: I suspect there is no such standard for TIFF or GeoTIFF overviews. There are multiple implementations, methods and formats to define overviews for GeoTIFFs.

---

GeoTIFF is based on the TIFF format (PDF specification for Revision 6.0, from 1992). The format has support for multi-page documents or subfiles, similar to a multi-page PDF.

A GeoTIFF file with 4 internal overviews is essentially a 5-page TIFF file, as viewed with identify:

$ identify -quiet file.tif
file.tif[0] TIFF 2027x2823 2027x2823+0+0 8-bit Grayscale DirectClass 188KB 0.000u 0:00.000
file.tif[1] TIFF 1014x1412 1014x1412+0+0 8-bit Grayscale DirectClass 188KB 0.000u 0:00.009
file.tif[2] TIFF 507x706 507x706+0+0 8-bit Grayscale DirectClass 188KB 0.000u 0:00.009
file.tif[3] TIFF 254x353 254x353+0+0 8-bit Grayscale DirectClass 188KB 0.000u 0:00.009
file.tif[4] TIFF 127x177 127x177+0+0 8-bit Grayscale DirectClass 188KB 0.000u 0:00.009

You can split these into separate TIFF files with (e.g.) convert file.tif file%d.tif, or GIMP.

A GeoTIFF file with an external overview uses a sidecar file. There is no standard approach how external overview files are created or used. For example, sometimes you may see a so-called ".ovr" file, which is a multi-page TIFF document:

$ identify -quiet file.tif
file.tif TIFF 2027x2823 2027x2823+0+0 8-bit Grayscale DirectClass 132KB 0.000u 0:00.000
$ identify -quiet file.tif.ovr
file.tif.ovr[0] TIFF 1014x1412 1014x1412+0+0 8-bit Grayscale DirectClass 2.101MB 0.000u 0:00.000
file.tif.ovr[1] TIFF 507x706 507x706+0+0 8-bit Grayscale DirectClass 2.101MB 0.000u 0:00.000
file.tif.ovr[2] TIFF 254x353 254x353+0+0 8-bit Grayscale DirectClass 2.101MB 0.000u 0:00.000
file.tif.ovr[3] TIFF 127x177 127x177+0+0 8-bit Grayscale DirectClass 2.101MB 0.000u 0:00.009

Other external overviews may use a ".rdd" or ".aux" sidecar file, which is a HFA/Erdas Imagine image file format. This is probably the most esoteric of the overview formats, since it is not even a TIFF file.

Answer 2

Some links about GeoTIFF:

• http://www.gdal.org/frmt_gtiff.html
• https://trac.osgeo.org/geotiff/

The overviews maybe stored internally or externally.

Internal overviews make geotiff file bigger and no simple way to update them. External overviews may be stored in .rrd or .ovr formats. The .rrd is more ancient but support by very old software. Anyhow .ovr is recommended to use now.

To create .rrd you need to set config option and -ro switch for external overviews as follow:

gdaladdo -ro --config USE_RRD ON sample.tiff 2 4 8 16

Answer 3

The pyramid format for TIFF is not standarized. However, in practice, the commonly accepted format is the .ovr from ESRI ArcGIS and the GDAL workflow by gdaladdo.

I have look at the code in gdal frmts/gtiff and have found that the code is mostly transported from the libtiff/contrib/addtiffo (see link).

The libtiff version of add overview is coded more clearly and I have succeffully ported the code into my own project.

I have tested it, it is compatible with ESRI ArcMap, and therefore is also compatible with gdal.

Han