2

I have a TIFF file of first day beyond 10 degrees at 2 m height covering the northern part of North America. The size of the file is 1.24 Gb, but when I load it into Python (3.7) I get the classic This is a BigTIFF error. I do not understand why I get that error. Here is the TIFF file information:

CRS EPSG:4326 - WGS 84 - Geographic
Extent  -172.5304334680000125,35.3037906650000011 : -52.5694104270000224,83.1300963930000023
Unit    degrees
Width   26708
Height  10648
Data type   Float32 - Thirty two bit floating point
GDAL Driver Description GTiff
GDAL Driver Metadata    GeoTIFF
Dimensions  X: 26708 Y: 10648 Bands: 21
Origin  -172.53,83.1301
Pixel Size  0.004491576420585592123,-0.004491576420736288419

I thought the limit was 4 Gb; so why do I get this error? Here is the beginning of the script until the error:

import gdal  # raster management
import ogr  # vector managment
import numpy as np  # gridded data
import os  # file managment
import csv  # for saving results in csv format
path = "D:/temperature/"
#%%
#load data
r_ds = gdal.Open(path + '2mheight/temperature2m_NA.tif')

1 Answer 1

3

TIFF cannot be larger than 4 GB but there is no other lower limit for the size of BigTIFF than the size of the compulsory headers. It is for example possible to create a single pixel BigTIFF file that has a filesize of only 253 bytes with gdal_create command

gdal_create -outsize 1 1 -of GTiff -co BIGTIFF=yes small_bigtiff.tif

That the file is a BigTIFF can be verified with a hex editor. The file begins with bytes

49 49 2B 00 08 00 00 00

http://fileformats.archiveteam.org/wiki/BigTIFF#Identification

Your image is rather big and the program that has written it may have considered that the size may reach the 4 GB limit so it is safer to create the image as BigTIFF.

This is how the the GDAL GeoTIFF driver makes a decision between TIFF and BigTIFF https://gdal.org/drivers/raster/gtiff.html

BIGTIFF=YES/NO/IF_NEEDED/IF_SAFER: Control whether the created file is a BigTIFF or a classic TIFF.

YES forces BigTIFF.

NO forces classic TIFF.

IF_NEEDED will only create a BigTIFF if it is clearly needed (in the uncompressed case, and image larger than 4GB. So no effect when using a compression).

IF_SAFER will create BigTIFF if the resulting file might exceed 4GB. Note: this is only a heuristics that might not always work depending on compression ratios.

BigTIFF is a TIFF variant which can contain more than 4GiB of data (size of classic TIFF is limited by that value). This option is available if GDAL is built with libtiff library version 4.0 or higher. The default is IF_NEEDED.

When creating a new GeoTIFF with no compression, GDAL computes in advance the size of the resulting file. If that computed file size is over 4GiB, GDAL will automatically decide to create a BigTIFF file. However, when compression is used, it is not possible in advance to known the final size of the file, so classical TIFF will be chosen. In that case, the user must explicitly require the creation of a BigTIFF with BIGTIFF=YES if the final file is anticipated to be too big for classical TIFF format. If BigTIFF creation is not explicitly asked or guessed and the resulting file is too big for classical TIFF, libtiff will fail with an error message like “TIFFAppendToStrip:Maximum TIFF file size exceeded”.

You can convert your file into TIFF with gdal_translate

gdal_translate -of GTiff -co tiled=yes -co BIGTIFF=NO input.tif output.tif

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.