Why do dist2Line and st_nn give slightly different distances?

Question

I have some points and am finding the distance to the nearest state border. My question is: why do dist2Line and st_nn give slightly different answers for the distances?

The code below is based on this question and answer How to associate point to nearest state boundary outside of the state the point is in as well as an answer from Calculate distance between points and nearest polygon in R.

Here is one approach:

library(sp)
library(spData)
library(tidyverse)

states <- as(us_states,"Spatial")

#create some points
pts <- data.frame(x1=c(-100.5, -98.6, -98), x2=c(35, 41, 44))

geosphere::dist2Line(p = pts, line = states)

Here is the other approach (that gets rid of the state polygon the point is in.

# create sf objects with same crs -> WGS 84
states <- us_states |> st_transform(st_crs(4326))

pts <- data.frame(x = c(-100.5, -98.6, -98), 
                  y = c(35, 41, 44)) |> st_as_sf(coords = c("x", "y"), crs = 4326)

for (i in 1:dim(pts)[1]) {
    
    pt <- pts[i, ]
    state_pt <- states[pt, ]
    
    n_states <- dplyr::filter(states, NAME != state_pt[["NAME"]])
    
    result <- nngeo::st_nn(pt, n_states, k = 1, returnDist=TRUE)
    print(result[[2]] |> unlist())
}

The answers are:

45558.45 
110992.87  
118342.74 

vs. 

45507.54
110868.8
118392.9

Answer 1

Me again. Checking the documentation it seems like nngeo::st_nn() and sf::st_distance() both return the great-circle distance aka spherical distance. In case of your first point (associated with Oklahoma), the result is 45,507.54 m.

geosphere::dist2Line() in turn (c.f. ?distGeo) gives you an "highly accurate estimate of the shortest distance between two points on an ellipsoid (default is WGS84)" - resulting in 45,558.45 m for Oklahoma.

Since ellipsoids are not perfect spheres, there are deviations in distance calculations between these methods based on the foundations used (sphere vs. ellipsoid).

No idea if references to Wikipedia are appreciated here, but this illustration gives you a pretty good idea of the problem in my opinion.