NOTE: Since I added this answer JLH has given a much better one, so please refer to his answer.
My original conclusion below:
I received no answers on this, so naturally I investigated it further myself.
My Conclusion: the Tiger data is precise enough to geocode to the correct street, but not to the actual lot.
I determined this after testing a good number of addresses on OpenStreetMap.org, which is also based on the Tiger data. The addresses were clustered in the exact same way as what I was seeing in my Tiger data.
The US Census Bureau says this about their data:
Due to resource constraints the enumerator updates were digitized
directly into the Census TIGER data base without geodetic controls or
the use of aerial photography to confirm the features' locational
Changes made by local officials do not have geodetic control.
But the good news is:
The Census Bureau is currently in the early stages of developing a
system to significantly improve the coordinate accuracy of features in
the TIGER data base and to devise a more effective approach to
Worthy of note
A study on the Tiger data accuracy was done in 2009: