Georeference single drone image from EXIF data

Question

Looking for an algorithm (python, formulas, whatever...) to georeference a single drone image based on its EXIF data.

Specifically, I need to translate drone GPS latitude and longitude (after conversion to projected coordinates, of course), relative altitude, drone camera orientation (pitch, roll, yaw), and drone sensor parameters (sensor size and focal length) to ground coordinates of the centre and four corners of the drone image.

I'm comfortable assuming flat terrain, so this would be independent of the DEM, and recognize measured GPS coordinates will have imprecisions affecting the results too.

I'm aware that commercial UAV mapping software and services, such as Pix4D, DroneDeploy, and MapsMadeEasy.com, solve the much more complicated problem of stitching many such images together and extracting 3D terrain information, including dealing with those imprecisions. I'm looking to solve the much simpler problem of "what is this one image looking at, more or less".

Based on Getting coordinates of pixel from georeferenced drone photo?, I've also started looking at literature about "monophotogrammetry" and "monoplotting" (example: https://www.wsl.ch/en/services-and-products/software-websites-and-apps/monoplotting-tool.html), but the emphasis seems to be on interpolation of features/lines on the photo after manually identifying GCPs. In this context, I'm after the math to approximately but automatically identify the corners only.

I've dusted off my geometry and trigonometry knowledge, and can do simple cases with simple formulas. For instance, ignoring roll and yaw, the centre of the photo is (I think!) the following distance in front of the drone's GPS location, and likewise the horizontal field of view at the centre:

fwd_dist_img_ctr = altitude * tan(-pitch)
ground_fov_img_ctr = hypotenuse(altitude,fwd_dist_img_ctr) * sensor_width / focal_length

I'm hoping for leads how to extend this to a) the four corners of the image, and b) incorporating roll and yaw, and therefore camera direction (bearing) too.

Adding: seems this is embedded in GRASS' i.ortho.photo (https://grass.osgeo.org/grass78/manuals/i.ortho.photo.html), but buried behind a GUI focused on GCPs, correction, and stitching a map from multiple photos. I'm looking for the minimally complicated math to do it automatically, per single image.

Eventually, this will be used to visualize a layer of research drone photos on a map by the ground footprint displayed, rather than by GPS location of the drone/sensor itself. Probably as a processing algorithm in QGIS after extracting all of the pertinent EXIF tags as attributes.

Answer 1

As requested, posting my own (partial) answer. The actual math is not too bad, but there are surrounding issues making application difficult:

1. UAV camera focal length specs seem to be before sensor cropping. In addition, there is post-processing to remove distortion. Therefore the published focal length, sensor size, and/or angular field of view specs need correction to accurately determine the image corners. (This is approximated with the ccf factor below, but is inaccurate in a way that varies between different cameras. Therefore, probably need to manually calibrate for each drone.)

2. The simplification of assuming terrain is flat is (in my case, at least) sufficiently often violated that the projected image rectangle is not reliable. This is because not only is the assumption that terrain is flat over the image area, but also that its elevation is the same as the drone take-off point, since only then is the relative altitude correct.

With those limitations, here's the math in Python-like pseudo-code (I actually prototyped in Excel, and then gave up for the reasons above).

# photo parameters, here with example data
CamX,CamY = 300828, 4956483 # camera/drone GPS coordinates converted to a projected CRS
A = 95.1 # relative altitude, EXIF tag RelativeAltitude for DJI drones
pitch = -42.6/180*3.14159 # degrees from EXIF tag GimbalPitch converted to radians. Always negative.
dir = -163.2/180*3.14159 # camera azimuth (direction clockwise from N). EXIF tag FlightYawDegrees

# camera paramaters
Cf = 4.49 # camera focal length in mm. This is for DJI Mavic Mini.
SX = 6.3 # sensor width in mm
aspect = 0.75 # aspect ratio, usually 3:4 or 9:16
Sd = SX * sqrt(1+aspect^2) # sensor diagonal in mm. Should correspond to Cf, but usually doesn't, see ccf below

# calculations
ratXh = SX/Cf/2 # ratio of sensor half-width to focal length (at image centre)
ratYh = aspect * ratXh # ditto for sensor half-height
ccf = sqrt(1+ratYh^2) # "corner correction factor" due to sensor crop and fisheye correction. For 0.75 aspect this becomes 1.13, but needs to be calibrated for each camera!
phiXh,phiYh = atan(ratXh),atan(ratYh) # 1/2-FOV angle in X,Y directions at image centre. Will be in radians.

Kc,Kf,Kb = A/tan(-pitch+{0,1,-1}*phiYh) # ground distance of camera ground projection to image; centre, front, back
Rc,Rf,Rb = sqrt(A^2+{Kc,Kf,Kb}^2) # full distance, hypotenuse of ground distance and altitude triangle
Wch,Wfh,Wbh = {Rc,Rf,Rb} * ratXh / {1,ccf,ccf} # 1/2 width of frame in ground coordinates, at centre, front, back; includes ccf fudge factor in corners

# now express the projection centre and corners in K,W coordinate system, i.e. ignoring rotation in dir
Centre_W,_K = 0,Kc
BR_K = BL_K = Kf
TR_K = TL_K = Kb
BL_W = -BR_W = Wfh
TL_W = -TR_W = Wbh 

# finally rotate using dir
Centre_x,BR_x,BL_x,TR_x,TR_y = CamX + (corresponding _W) * cos(dir) + (corresponding_K) * sin(dir)
corresponding _y = CamY - (corresponding _W) * sin(dir) + (corresponding_K) * cos(dir)