I have the following MySQL function:
DELIMITER $$ DROP FUNCTION IF EXISTS `fn_vincenty_distance`$$ CREATE FUNCTION `fn_vincenty_distance` ( lat1 FLOAT, lon1 FLOAT, lat2 FLOAT, lon2 FLOAT ) RETURNS FLOAT NO SQL DETERMINISTIC COMMENT 'Returns the distance in degrees on the Earth between two known points of latitude and longitude using the Vincenty formula from http://en.wikipedia.org/wiki/Great-circle_distance' BEGIN RETURN DEGREES( ATAN2( SQRT( POW(COS(RADIANS(lat2))*SIN(RADIANS(lon2-lon1)),2) + POW(COS(RADIANS(lat1))*SIN(RADIANS(lat2)) - (SIN(RADIANS(lat1))*COS(RADIANS(lat2)) * COS(RADIANS(lon2-lon1))) ,2)), SIN(RADIANS(lat1))*SIN(RADIANS(lat2)) + COS(RADIANS(lat1))*COS(RADIANS(lat2))*COS(RADIANS(lon2-lon1)))); END$$ DELIMITER ;
I cannot reproduce some of the airport distances in miles. I'm using
69.2 miles as the distance between each degree of latitude at the equator:
fn_vincenty_distance(37.6188056, -122.3754167, 32.896800990, -97.038002010)*69.2
1463.70 which is close to
1464.373 from https://www.airmilescalculator.com/distance/sfo-to-dfw/
fn_vincenty_distance(25.7953611, -80.2901158, 41.9745219, -87.9065972)*69.2
1200.77 which is not close to
1197.062 from https://www.airmilescalculator.com/distance/mia-to-ord/
How can I get the same answer being produced from airmilescalculator.com?
This is to match the calculation by airlines, for example from AA.com:
ATAN2output times the radius of the sphere in miles.