I did this by accident a few months ago, although it was with lines and it was a very large featureclass (I think it was on the order of 10 million features). I also don't remember if it was SQL Server 2008 or 2012.
One thing that I noticed in ArcMap was that everything was really, really slow. I profiled it. Found the query that was causing the slowness and popped it into SSMS and determined that it was the Filter method that was the culprit (iirc, ArcMap uses Filter with an envelope to grab OIDs). SQL Server was bringing back every single row in the table!!
When I switched over to geography, Filter was returning the correct number of features for the given geometry.
I don't recall other methods behaving badly, but I didn't leave the 4326 data in SQL geometry long enough to find out.
If your table size is not very large, this might not matter. But its something to keep in mind for larger tables. It also wouldn't matter if QGIS doesn't use