I have the following MySQL function:
DELIMITER $$
DROP FUNCTION IF EXISTS `fn_vincenty_distance`$$
CREATE FUNCTION `fn_vincenty_distance` (
lat1 FLOAT, lon1 FLOAT,
lat2 FLOAT, lon2 FLOAT
) RETURNS FLOAT
NO SQL
DETERMINISTIC
COMMENT 'Returns the distance in degrees on the
Earth between two known points
of latitude and longitude
using the Vincenty formula
from http://en.wikipedia.org/wiki/Great-circle_distance'
BEGIN
RETURN DEGREES(
ATAN2(
SQRT(
POW(COS(RADIANS(lat2))*SIN(RADIANS(lon2-lon1)),2) +
POW(COS(RADIANS(lat1))*SIN(RADIANS(lat2)) -
(SIN(RADIANS(lat1))*COS(RADIANS(lat2)) *
COS(RADIANS(lon2-lon1))) ,2)),
SIN(RADIANS(lat1))*SIN(RADIANS(lat2)) +
COS(RADIANS(lat1))*COS(RADIANS(lat2))*COS(RADIANS(lon2-lon1))));
END$$
DELIMITER ;
I cannot reproduce some of the airport distances in miles. I'm using 69.2 miles
as the distance between each degree of latitude at the equator:
Case 1
SFO-DFW
fn_vincenty_distance(37.6188056, -122.3754167, 32.896800990, -97.038002010)*69.2
Returns 1463.70
which is close to 1464.373
from https://www.airmilescalculator.com/distance/sfo-to-dfw/
Case 2
MIA-ORD
fn_vincenty_distance(25.7953611, -80.2901158, 41.9745219, -87.9065972)*69.2
Returns 1200.77
which is not close to 1197.062
from https://www.airmilescalculator.com/distance/mia-to-ord/
Question
How can I get the same answer being produced from airmilescalculator.com?
This is to match the calculation by airlines, for example from AA.com:
ATAN2
output times the radius of the sphere in miles.