Is a 10,000' peak still a 10,000' peak if sea levels have risen a foot? Is it a 9,999' peak now, or is elevation just a constant?
Seems odd that a patch of land a few feet above "sea level" could sit underwater.
For the purpose of measuring the height of objects on land, the usual datum used is mean sea level (MSL). This is a tidal datum which is described as the arithmetic mean of the hourly water elevation taken over a specific 19 years cycle. This definition averages out tidal highs and lows (caused by the gravitational effects of the sun and the moon) and short term variations. It will not remove the effects of local gravity strength, and so the height of MSL, relative to a geodetic datum, will vary around the world, and even around one country. Countries tend to choose the mean sea level at one specific point to be used as the standard “sea level” for all mapping and surveying in that country. A geodetic vertical datum takes some specific zero point, and computes elevations based on the geodetic model being used, without further reference to sea levels. Usually, the starting reference point is a tide gauge, so at that point the geodetic and tidal datums might match, but due to sea level variations, the two scales may not match elsewhere. An example of a gravity-based geodetic datum is NAVD88, used in North America, which is referenced to a point in Quebec, Canada. Ellipsoid-based datums such as WGS 84, GRS80 or NAD83 use a theoretical surface that may differ significantly from the geoid.
So to answer your question, if they are using MSL datum, the height of a peak would change, but it's a 19 year average, so you won't see sudden changes. However, most elevations are taken using a geodetic datum, which is based on a specific zero point that doesn't change, so you wouldn't see the elevations change with sea level change.
As felixIP and PolyGeo pointed out, this question/answer should probably be moved to a more general forum. However, it does have context for GIS (i.e. a client has these concerns about sea level change).