It is possible but the polygons are a little ragged, see the attached picture. Street View images are (single-band) indexed TIFFs. Using the raster value tool told me that the pixel value for buildings was 204 and I used this value in the Raster Calculator to make a new TIFF in which all of the pixels in the image having a value of 204 were converted to ones and all of the pixels in the image having a value not equal to 204 were converted to zeros. The expression to do this is 'filename@1 = 204' (without the quotes).
It was then simply a matter of polygonising the new TIF: Raster -> Conversion -> Polygonize (Raster to vector).
In the picture I've overlaid the original image with just the 'buildings' polygons.
Low-down dirty stuff really.
Nick.