This issue seems to have been solved already some time ago, but I´ll add my spoon to the soup anyway since I recently needed some raster stats myself. 

The QGIS methodology described by rudivonstaden worked like a charm, thanks for the tip. However, in my case I also wanted to know min&max values and std for the raster values within my polygons, and that functionality I could´nt find in QGIS. I found a solution through the function [isectpolyrst][1] in [GME][2], which is a favorite of mine for solving  GIS problems with.This function gave me just the stats I needed and rather quickly as well.


PS. GME has a bit of an unfortunate dependency (ArcGIS). So even though the program itself is freely available to everyone, you need ArcGIS to run it. But if you do have ArcGIS (any level of licence), I recommend you try it out.


  [1]: http://www.spatialecology.com/gme/isectpolyrst.htm
  [2]: http://www.spatialecology.com/gme/