.
first of all you should read Publishing a GeoTIFF via GeoServer here. then Publishing a layer as wms here.
GeoServer can also publish raster imagery. This could be simple
georeferenced-images (such as blue marble imagery) all the way to
multi-band DEM (digital elevation model) data. In this section, we
will load a simple GeoTIFF containing shaded relief for land area. The
layer uses standard tri-band RGB values (0-255).

second is that you should read Using WMS service layers here and here. Web Mapping Service (WMS) services can be used in ArcMap (as well as in ArcScene and ArcGlobe) as map layers.
Adding Geospatial Data from a Web Mapping Service (WMS) in ArcMap, you can use this ways:
From Add Data button (use Go to Home Folder button) GIS Servers > Add WMS Server
From Catalog (use Go to Home Folder button) GIS Servers > Add WMS Server
in Table of Contents, right click on Layers and select Add Data > GIS Servers > Add WMS Server
more information you can check out here.

i hope it helps you...