I am unable to reproduce correct calculation of angular distance of the globe.

I selected two point near my home on Google Earth:

enter image description here

and measured the distance between them by the ruler tool.

Distance in meters is approximately 100 and distance in arcseconds is reported by program as 3.19

I have calculated this formula

(100/6371008.8)/2/Pi *360*60*60

and found it equals to 3.23 which is close to the value, reported by Google Earth and the difference is probably caused by geoid shape.

Now I tried to calculate the same value trough vectors on the computer and encountered much bigger error

enter image description here

As you see, first I converted geographic coordinates to 3D cartesian ones and then calculate an angle between two vectors by normalizing inner product. Unfortunately, the result is 2.18, which is much more wrong.

Intermediate value also presented.

Looks like the main problem is calculating inverse cosine for the value, close to 1. Is this true?

How to overcome this problem?


I have always used the Haversine formula for finding the distance (angular or Earth's surface) between two points. Which formula are you using?

I'm not sure how you've turned lat/lon 2D coordinates into x, y, z 3D coordinates. Have you tried taking the Euclidean distance and checking that it is reasonable?

As I'm sure you know, some spherical calculations can break down over very short distances.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.