I have a large number of small tiles merged into a single VRT. The rasters are only viewed at large scale, so only a few of them are visible at the same time. The map refresh in qgis is slowing down however, as the number of tiles in the vrt increases. Is there any indexing of raster possible in the vrt format itself or perhaps inside pyqgis (manipulating the raster layer that links to my .vrt).
Summarizing the comments so this question can be marked as answered:
VRT does not support any kind of indexing. There is an open GDAL ticket about this http://trac.osgeo.org/gdal/ticket/5762. However, there may be a workaround. It should be possible to create a set of smaller .VRT files which contain a reasonable amount of images and then build another vrt from the first generation .VRT files. Sort of a hand written b-tree index.
ECW might be an alternative, depending on the use case.