Is there a keyboard shortcut to delete the last vertex created when making a line or polygon in ArcMap?
Basically the same functionality as right clicking and then picking "Delete Vertex"
CtrlZ is the standard Windows shortcut for undo, which works in ArcGIS Desktop.
See Keyboard shortcuts that can be used while editing
Also have a look at Keyboard shortcuts in ArcMap for more shortcuts that will work in ArcGIS, and the printable ArcGIS Desktop Tips and Shortcuts PDF.
As I was working on ArcMap 10.1 and 10.2 there was also a Backspace key working to delete last vertices, while continuing drawing. I guess it should be available in other versions too.