Unfortunately it appears that the ILayerExtensionDraw.BeforeLayerDraw and AfterLayerDraw events are only fired by feature and network layers (see wguidry's post here) and I need to support raster layers, map/image services, mosaic datsets, etc.
So in a nutshell here is what I ended up doing. Thanks @Kirk again for the ideas!
Created the following classes:
Timing: Stores the layer name, draw phase and elapsed time for a drawing event
TimingSequence: Stores all the timings in a drawing sequence
TimingSession: Stores all of the timing sequences in a timing session and manages the Stopwatch.
A timing session is started and stopped by a toggle button. When the timing session starts I enable IViewManager.VerboseEvents and disable it when the timing session ends.
I reset and start the Stopwatch at the start of a drawing sequence as indicated by IActiveViewEvents.ViewRefreshed and instantiate a new TimingSequence object.
Then, as each layer draws for a specific phase, I listen for IActiveViewEvents.AfterItemDraw, stop the Stopwatch, check that ITrackCancel.Continue() returns true (false indicates drawing was canceled) and record the layer name, phase and elapsed time in a new Timing object and add it to the current TimingSequence, and reset/restart the Stopwatch.
I only add a timing sequence to the timing session when IActiveViewEvents.AfterDraw fires in the esriViewForeground phase (the last drawing phase).
When the user ends the timing session by toggling the button again, I report the results (ideally it would be something nicer than an ugly MessageBox as it is now). It seems to be working smoothly though.
Problem: The AfterItemDraw event only seems to fire for layers in the root of the table of contents, not nested sublayers, so if you want individual measurements for a layer it must be moved into the root of the TOC and not within a group layer. See linked question: ArcObjects: AfterItemDraw only fires for layers in the root of TOC