In Tilemill, I have a #layer that consists of a CSV with 3 columns: [Name], [Latitude], and [Longitude].
I have plotted points using the latitude and longitude, but I would like to have the Name of each point only to be visible when it is hovered over.
I am presuming that I will need to write some kind of HTML code in the Templates -> Teaser tab, but am unsure of how to go about this.
Interaction of that sort is not possible in TileMill. The reason for this is that TileMill generates static .png images of your map as 256x256px tiles. To re-render markers, you would have to generate (and serve) a new tile every time a user hovered over the marker--or generate multiple tiles if the marker overlapped many.
It is possible to generate popup interaction in TileMill using UTF-8Grids, in the Templates -> Teaser tab as you mentioned, but that interaction will only display a popup, not change the style of the marker.
To pull off what you're talking about, you would need to use a javascript library like leaflet or mapbox.js.
