You probably do both, but mostly on OpenLayers. The GeoServer part is mostly related to styling of layers.
OpenLayers calls GeoServer to render a picture (e.g. PNG or JPEG) that gets shown to the user as the map. That picture might have some other stuff rendered over the top of it on the client side (e.g. a feature location, which might come from the same GeoServer instance, or some other source). The client side rendering is up to OpenLayers.
As Francisco pointed out, you can preview your layer on GeoServer, and it uses OpenLayers to show the output to you (Layer Preview is just another website). That OpenLayers isn't your web site.
You don't have to use OpenLayers on the client side - Leaflet is also good, and might be easier to learn.
You don't have to use GeoServer on the server side - there are a range of map source options (e.g. MapServer is also popular). GeoServer is a good choice though :-) You can also use tile map servers (TMS) instead of WMS, depending on your needs.