# Mathematics behind converting scale to resolution?

I have a ArcGIS service configured as shown in this pastbin. Specifically, the first LOD looks like this:

``````{
"level": 0,
"resolution": 222.2222222222222,
"scale": 256000
}
``````

I determined how to convert from the scale of 256000 to arrive at a resolution of 222.222:

``````var dotsPerInch = 96.0;
var inchesPerFoot = 12.0;
var dotsPerUnit = dotsPerInch * inchesPerFoot;
var scale = 256000;
var resolution = scale / dotsPerUnit;
``````

But the associated WMTS service reports a "ScaleDenominator" value of 241904.7619047619:

``````<TileMatrix>
<ows:Identifier>0</ows:Identifier>
<ScaleDenominator>241904.7619047619</ScaleDenominator>
<TopLeftCorner>-1.77905E7 4.643889999999999E7</TopLeftCorner>
<TileWidth>256</TileWidth>
<TileHeight>256</TileHeight>
<MatrixWidth>3</MatrixWidth>
<MatrixHeight>3</MatrixHeight>
</TileMatrix>
``````

What are the mathematics behind this value? That is, given a scale of 256000 how did ESRI arrive at a ScaleDenominator of 241904.7619?

• thank you for explanation of scale/resolution. I couldn't find it anywhere! Jan 28, 2013 at 13:40

WMTS assumes a DPI 90.7 instead of 96 as is clearly documented in the WMTSCapabilities document which states,

"The tile matrix set that has scale values calculated based on the dpi defined by OGC specification (dpi assumes 0.28mm as the physical distance of a pixel)."

0.28 mm per pixel = 0.0110236 inches per pixel or 90.71446714322 pixels per inch.

If you replace 96 in the equation above with 90.71428571429 you'll get the ScaleDenominator value so ESRI used a different conversion constant. After a little research I learned that

1 in = 2.54 cm (I thought this was an approximation but it's by definition)

Since there are 25.4 mm in one inch then 25.4 / .28 = 90.71428571429 DPI which is the value we're after for DPI. Here is a site which confirms this calculation.

• +1 There are two different feet; you have described the International Foot. The definition of the other one, the US Survey Foot, is that one foot = 12/39.37 meter (which, pace Eric Weisstein, is still in extensive use). They differ by two parts per million: a small amount, but large enough that when coordinate systems have values running into the millions or tens of millions, it makes an appreciable difference. Jul 17, 2012 at 20:14
• It is the OGC WMTS specification, not the WMTSCapabilities document that defines 0.28mm per pixel. Jul 28, 2014 at 12:33
• @JohnBarça It is the actual ESRI WMTSCapabilities response that I am quoting here.
– ca0v
May 26, 2016 at 11:49
• Yes, but ESRI took this from the OGC spec, they didn't invent this, or, for a change, come along late to the party and ignore the spec, using their size to impose their view of how things should be. Nov 24, 2020 at 15:45