Geoserver will set the 'Native' CRS to what it determines the source data's coordinate system to be. You can set the 'Declared CRS' and the option ' Force Declared', which will force the layer to use the Declared CRS, but thats usually only if the source CRS is unable to be read by Geoserver. However in your instance, this should force it to use the declared (3857).
Some reading on this can be found here: https://docs.boundlessgeo.com/suite/1.1.0/geoserver/configuration/crshandling/configurecrs.html
In terms of isolating the layer itself in its own getCapabilities. hmm. Not sure if this is ideal, but, the documentation here: https://docs.geoserver.org/2.13.2/user/services/wms/reference.html
Thishttps://docs.geoserver.org/stable/en/user/services/wms/reference.html This states that a 'getCapabilities' is a
"The GetCapabilities operation requests metadata about the operations, services, and data (“capabilities”) that are offered by a WMS server."
Suggesting that, if you have this
http://localhost:8080/geoserver/wms?
service=wms&
version=1.1.1&
request=GetCapabilities
but you want
http://localhost:8080/geoserver_new/wms?
service=wms&
version=1.1.1&
request=GetCapabilities
then you will need an entirely new instance of Geoserver. Sounds like overkill. My recommendation is that you simply add a new workspace as per the suggestion above, which can isolate the layer definition from your existing layer.
As an example, you currently have
http://localhost:8080/geoserver/wms?
request=GetMap
&service=WMS
&version=1.1.1
&layers=topp:states
Where topp is the workspace. But with a new workspace, you will have:
http://localhost:8080/geoserver/wms?
request=GetMap
&service=WMS
&version=1.1.1
&layers=new_workspace:states2
And then simply apply the coord system settings that you desire, to the new_workspace:states2 layer.
