# Converting distance between nautical miles and degrees

This is such a basic question that I'm kinda of embarassed. Here is the situation:

We have a certain funcionality developed that:

• Will check for a tolerance value (this input is given by the user as NM);
• Buffer a polygon by that value (all polygons are in WGS-84);
• Iterate through selection (buffer x points) and creates a record for each intersected point.

The tester at my corporation is saying the tool is giving wrong results. Here is his workflow, for testing:

• Creates a polygon;
• Define a nearby point;
• Measure the distance between the polygon and point (using measure tool);
• Configure tolerance to be larger or equal distance;
• RUn the tool;

I have a problem in the middle, is that our user needs to store these tolerance values as nautical miles, and I need to convert them to degrees for them to be used in our buffer (I'm using IUnitConverter). All buffers are constructed using ITopologicalOperator.

My guess is the conversion is being done purely in a numerical way and then, buffering by less than it should.

I've read tons of articles speaking about projection engine.dll, but that is for calculating distances, not buffers (I mean, if I need to implement a custom buffering solution, that will take into account each vertex...oh dear).

Can you guys recommend an approach for this? How can I convert this measure and still hit these edge cases?

Thanks again

-
Why are you using degrees for your buffers? It is not a planar unit. –  Stephen Quan Feb 12 '12 at 22:24
All my geometries are in WGS-84. The buffer method input is in degrees? Unless I'm missing something... –  George Feb 12 '12 at 22:26
I think you need to double check that. It's okay to specify your geometry in WGS_84, but, I believe buffer takes projected coordinates, usually meters. This could suggest that your input be out by a factor of roughly 100000? –  Stephen Quan Feb 12 '12 at 22:46
@BicycleDude check the docs. first line of the remarks section: edndoc.esri.com/arcobjects/9.1/componenthelp/esriGeometry/… –  George Feb 12 '12 at 22:48
@BicycleDude I'm pretty sure also that I've would caught on that a long time ago if the distance was THAT distorted... –  George Feb 12 '12 at 22:49

I know we had a bit of a discussion in the comments, but, I would like to put it out there what I think the solution is. So far, what you've told me is:

• You're data is in WGS-84 (i.e. mapUnit is in DecimalDegrees)
• You haven't specified DistanceUnit but I'm going to assume meters

Let's do the following workflow:

1. Create a polygon in WGS-84 in DecimalDegrees
2. Create a point in WGS-84 in DecimalDegrees
3. Measure the distance between polygon and point in Meters
4. Use IUnitConverter to present this to the tester in NM
5. Have the input input a tolerance in NM
6. Use IUnitConverter to convert this back to Meters
7. Run the buffer tool specifying the tolerance in Meters

Now, you can, if you wish move your logic moreso to DecimalDegrees as you've specified that the buffer tool can work that way. So, you can try to rework steps 6 & 7 as you wish. Currently, my gut feeling is your problem may be at step 3 where your DistanceUnit is different than your MapUnit.

-