With OpenLayers 2 you can load .osm files using
OpenLayers.Format.OSM, see the OpenLayers osm file example in the OSM wiki. Keep in mind that this approach will be rather limited compared to a tile server and will work only for small files.
With OpenLayers 3 you can achieve something similar according to this similar question on StackOverflow by using
ol.source.OSMXML. There is also an example of loading OSM XML vector data dynamically from a server (in this case using Overpass API).