in QGIS I have 1 raster-layer (elevation) and 1 vector-layer (administrative units).
Now I would like to be able to query one of the administrative units and see the mean and min/max of the elevation within its area.
Which QGIS tool can do that?
Yes, this is called Zonal Statistics. You can run it with Zonal Statistics plugin, this is the core plugin, so it doesn't need to be installed separately, simply turn it on. Note that there is not that many statistics available.
although this is not exact answer for you as it does not use QGIS, I still think it could be valueable for you.
Some time ago I had to compare outputs from different modeling softwares so I had plenty of grids (=rasters, mostly in ascii or SAGA format) and add their min/max/averages to the shapefile polygons.
I used SAGA GIS (opensource like QGIS) tool Grid Statistics for Polygons:
Which added all the statistical data calculated from the raster file to the shapefile. It should be also able to add data from multiple rasters, but then there is a problem with the DBF limitation as it cuts the names of the columns containing the raster name and the statistical parameter. So you should use short names for the rasters or later edit the column names in QGIS to better represent which data is it.
I can think of a semi-automatic way to do this and it's not as elegant.
ID, min_elev, max_elev
0, 500, 1000
1, 200, 250
....
100, 100, 1000
Now when you query any polygon in the vector layer you will see min_elev and max_elev attributes.