GeoServer serves the images as established standards like WMS and WCS. These can be read by many clients like QGIS, OpenLayers or Leaflet. COGs on the other hand cannot be read by that many clients.
GeoServer can also serve the images in different coordinate reference systems, which is often required. Additionally it can create mosaics of COGs. This means many adjacent COGs would appear as one layer. Additionally COGs can be stores in a time-based mosaic and can then be queried by a timestamp.
Styling is also something that GeoServer can do in contrast to COGs. When you request COGs, you get the raw values of the pixels. GeoServer can provide various styles. This can be useful when you have digital elevation models or continuous data like temperature maps, that you would like display in a classified way.
All in all, GeoServer puts a nicer interface on top of COGs.