The way I think about this distinction is:
A layer (unless written to a layer file or saved as part of a map document) is only stored in memory, but it is not stored in an in_memory workspace.
Remember that when you are in ArcMap, what you see in the Catalog window are spatial datasets (i.e. not much more than x,y coordinates, coordinate system and attributes), and that they only obtain a myriad of other properties to configure such as symbology, MapTips, hyperlinks, etc after you have added them as layers (which appear in the Table of Contents). This is the same distinction as above.