I have a 25 large geoTiff files, representing a DEM, to be published via Geoserver in order to have OGC WCS. The size of each uncompressed file ranges between 235 MB and 6GB. What would it be the best method to follow considering performance? I read a lot of documentation about image mosaic plugin and imagePyramid plugin, but I do not still figure out what to do, whether adding inner tiles and overviews on all geotiffs then publish using image mosaic or merging those file into one file then using pyramid plugin.
Based on the advice in "GeoServer on Steroids" I would aim for a mosaic of GeoTiffs. Page 8 clearly states to choose a mosaic when:
- A single file gets too big (inefficient seeks, too much metadata to read, etc..)
- Single granules can be large
- Use Tiling + Overviews + Compression on granules
but suggests moving to pyramids when
- Tremendously large dataset
- Too many files / too large files
- Need to serve at all scales
- Especially low resolution
So I would use GDAL to split them into a collection of GeoTiffs with tiles and overviews set and then drop them all into an Image Mosaic. Some experimentation could be useful to see what the best size of GeoTiff v. Number of granules is, but I'd err towards keeping it below a 100 if you are doing a lot of full views.
To split up a GeoTiff you can use a script like:
#!/bin/bash # If I was really bored I'd work out how to parse the gdalinfo output #Upper Left ( -25.000, 1240025.000) ( 9d22'14.87"W, 60d50'27.75"N) #Lower Left ( -25.0000000, -25.0000000) ( 7d33'24.37"W, 49d45'57.40"N) #Upper Right ( 660025.000, 1240025.000) ( 2d48'16.84"E, 60d57'27.18"N) #Lower Right ( 660025.000, -25.000) ( 1d37' 1.96"E, 49d50'34.37"N) #Center ( 330000.000, 620000.000) ( 3d 6'26.52"W, 55d28' 7.68"N) x=0 y=0 mx=700000 my=1300000 step=10000 for ((i=0; i < $mx; i+=$step)); do for ((j=0; j < $my; j+=$step)); do nx=$(($i + $step)) ny=$(($j + $step)) echo "gdal_translate -projwin" $nx $ny $i $j done done
Ciao, an observation on number VS size of files.
Opening more than 10 to 20 file to respond to a request would hinder scalability of WMS and WCS. On the other side having Geotiff that are too large (hard to say but >> 20gb as a start) would make the internal structure too big for fast serving.
As such you need to balance size and # of files to read on each request. By experience in your case you could simple reprocess each file and then mosaic. If you want you might want to first merge to reduce the number of files to use. Use compression to reduce the size if possible.