3

I want to create a color-relief and a hillshade with gdaldem and use them with mapnik. System: Ubuntu 14.04 LTS Server, gdal-bin 1.10.

Here are my steps:

  • Download all SRTM tiles from viewfinderpanoramas:

    wget -r -np -l 1 -A zip http://viewfinderpanoramas.org/Coverage%20map%20viewfinderpanoramas_org3.htm

  • Unpack all zip files

  • Fill all remaining voids (creates out of every .hgt a .hgt.tif file)

    for hgtfile in *.hgt;do gdal_fillnodata.py $hgtfile $hgtfile.tif; done

  • Merge all .tifs into one huge tif. This file will is the raw DEM with full resolution and the start for any further steps:

    gdal_merge.py -n 32767 -co BIGTIFF=YES -co TILED=YES -co COMPRESS=LZW -co PREDICTOR=2 -o ../raw.tif *.hgt.tif

  • Convert the raw file into Mercator projection, interpolate and shrink e.g. to 1 km/pixel:

    gdalwarp -co BIGTIFF=YES -co TILED=YES -co COMPRESS=LZW -co PREDICTOR=2 -t_srs "+proj=merc +ellps=sphere +R=6378137 +a=6378137 +units=m" -r bilinear -tr 1000 1000 raw.tif warp-1000.tif

  • Create color-relief with transparency (alpha channel) and lzw compression:

    gdaldem color-relief -co COMPRESS=LZW -co PREDICTOR=2 -alpha warp-1000.tif relief.txt relief-1000.tif

  • Create hillshade

    gdaldem hillshade -z 7 -compute_edges -co TILED=YES -co COMPRESS=JPEG warp-1000.tif hillshade-1000.tif

I get these output file sizes:

warp-5000.tif: 113 MB warp-1000.tif: 606 MB warp-500.tif: 1.7 GB

relief-5000.tif: 17 MB relief-1000.tif: 19 MB relief-500.tif: >60 GB (hdd was full!)

hillshade-5000.tif: 2.7 MB hillshade-1000.tif: 75 MB hillshade-500.tif: >70 GB (hdd was full!)

Why does the file size suddenly explode? Is there a way to create geotiffs with usable file size?

Please ask me if more you need more information about my approach. Thanks in advance!

  • Shading and compressing in the same operation might be a bit difficult. Note, just because an image is compressed doesn't mean that it's smaller file size; try uncompressed and then gdal_translate to compress, but first get yourself a new hard drive - if you've only got 60GB free space the file allocation is working overtime to place data unless your drive is < 300GB total... even then, you need more free space to do this - hard drives are cheap, time is expensive. – Michael Stimson Nov 3 '14 at 1:59
  • Sorry, my hard drive is 1 TB. I just had several tries with the SRTM tiles and my mapnik server and database consume 500 GB. Creating once uncompressed, compressing with gdal_translate and then deleting the uncompressed file, could work. I'll give it a try, thanks! – der-stefan Nov 3 '14 at 7:30
2

Compression doesn't work well in the process of generating a new file with "tiled=yes" or above a certain file size. I solved this problem as supposed by Michael:

Create your hillshade by compressing it with lossless LZW. In my example, the whole planet warped with 500 meter resolution, the hillshade-500.tif consumes 131 GB.

gdaldem hillshade -z 4 -compute_edges -co TILED=YES -co BIGTIFF=YES -co COMPRESS=LZW -co PREDICTOR=2 warp-500.tif hillshade-500.tif

Then compress it with lossy JPEG:

gdal_translate -co COMPRESS=JPEG -co TILED=YES hillshade-500.tif hillshade-500-comp.tif 

The resulting file size is quite as expected:

hillshade-500.tif: 131 GB

hillshade-500-comp.tif: 249 MB (= 1,9%)

So you first need enough disk space, but then can compress the file further.

Note: Add -co bigtiff=yes only if your file size exceeds 4 GB.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.