Okay, so I'm a little fed up with different raster-in-database solutions that I tried over the last 2 weeks which don't work currently (PostGIS raster) or can't handle my volume of data (Rasterlite), so I'm reading about the VRT format. Bear with me, I'd like your opinions on this, and also if what I propose makes sense at all:
VRT is basically a metadata XML file describing various properties of the actual raster file, like pixel dimensions, geolocation, etc..
Step 1: I converted (not really, just generated VRTs) all my GeoTIFF files into VRT.
Now, since I see I can merge the VRT, I assume its possible to create a mosaic of VRT metadata files, one big VRT to hold all metadata. Great, that means, metadata will probably be used for optimizing the rendering since QGIS will decide (assumption) which tiles fall into the current canvas extent and which not...
Step 2: create a VRT mosaic
Okay, now the next benefit I see is, I only need to add one entry into the QGIS table of contents and I can set different things upon it, like scale dependant rendering and null data value transparency if needed. One could argue, you can simply load GeoTIFFs into QGIS and put them in a group, true, but you don't have any control over a group except for setting the common SRS.
If I wanted to see my orthophoto on a small scale (large zoom-out) and still have speedy rendering I would repeat the steps, only with a resampled set of original raster and end up with a few big VRT datasets.
Does this make any sense? :)