Wrong parameters when georeferencing single drone images

Question

I am using a DJI Mavic 3M drone to capture images from the ground view. DJI provides a guide on how to correct the images for the lens distortion. Each captured image contains XMP metadata about the GPS location of the drone, the yaw, pitch and roll angles of the gimbal, matrix parameters for undistorting the image using OpenCV.

From this, I want to compute the WGS84 coordinates for any pixel on the image.

I marked some points on the ground. The camera was pointing directly down, having a pitch orientation of nearly -90°. The results weren't good enough for my use case. Down below the average error:

µ: 0.3784 m | σ: 0.1822 m

I found these issues:

1. The ground sample distance is way too small. Our images have a dimension of 5280x3956 pixels. The focal length of the camera is 12.29mm, the sensor width is 17.3mm and the sensor height is 13mm. The drone flew around 12m. This results in a ground sample distance of around 3.2mm/pixel. I sprayed a 2mx2m rectangle on the ground and the calculation resulted in a 1.8mx1.8x rectangle. I had to increase the GSD to around 3.75mm/pixel

2. The yaw angle is also slightly wrong. In the metadata, the yaw angle of the gimbal is stored. However, I needed to subtract some degrees (~5°) to improve the results

3. There was also a slight offset in the coordinates

Is anyone familiar with georeferencing on a single image and can point out what steps I am missing? I already have undistorted the image using the parameters stored in the metadata. I am also aware that this question has been asked in some form or another (#1, #2), but the answers unfortunately didn't help me.

Below is a plot showing the problem. The red crosses are the markers I sprayed on the ground, the blue dots are the points that I have computed. The green cross is the position of the drone in the moment it captured the image.

Answer 1

My experience is with a Mavic Mini, which does not have published lens distortion specs and has only consumer-grade GPS, not RTK correction. With that caveat in mind, I've found you just need to accept a level of imprecision from purely drone-derived georeferencing consistent with what you've experienced.

To do better, you need to add GCPs and create a point cloud and/or orthophoto/DEM pairing that covers them, manually georeference it, and then (if needed for some reason) adjust the georeferencing of specific images to that constructed mosaic.

Reasons for this include (some of these may be better with your Mavic 3 than my mini):

1. Drone reported altitude is relative to the launch point, and so calculated GSD is at best in the horizontal plane lying through the launch point. It does not take much terrain variation for that to be off from genuine ground level, e.g. a 1.2m (4') variation in terrain with translate to a 10% difference in (apparent) GSD at your 12m nominal flight altitude.

2. There are reports of significant (up to 5m over a round-trip) reported altitude inconsistencies reported for some DJI drones. I think this is using just the barometric sensor, without RTK information, but I'm not sure. See eg https://mavicpilots.com/threads/altitude-accuracy.114737/

3. On my mini, I've found the drone yaw angle is true, but the gimbal yaw versus the drone is not corrected for wind. This can be a couple of degrees, I believe. (Also I think there's some inconsistency in the reporting XMP tags of whether the gimbal angle includes the drone yaw or not, sorry can't remember)

4. On the mini at least, there is significant lens distortion. The data you link to for Mavic 3 lens distortion parameters is not available, and I'm not sure how to interpret the numbers provided in the link you gave, so I can't test it, but on my Mini the barrel distortion is sufficient that I've had to apply an empirical correction of nearly 10% to the calculated FOV angle.

5. I did have one anomalous data collection session where the reported gimbal angles were just off. I did a drone IMU recalibration and the issue fixed itself.

While your more expensive DJI drone ought to be better regarding these points, my experience was enough to not trust attempts at precise georeferencing based solely on calculations from image XMP data, and rely on properly pinpointed GCPs instead.

It's also telling that services like https://www.mapsmadeeasy.com/features that create maps from drone images rely primarily on 3D stitching, with stretch and tilt using GCP data as a post-processing correction, and using EXIF/XMP tags in each image only as supplementary information. It sort of confirms these issues are widespread.

Answer 2

I'm working on a similar problem trying to get projected coordinates, X and Y of objects from their pixel coordinates in the UAV images. This paper https://doi.org/10.3390/s22020604 outlines the method I am trying to get working using the UAV camera intrinsic matrix, rtk position and yaw, pitch and roll of the sensor. This method works okay for objects that are close to the ground and for fairly flat surfaces as the height reported in the xmp metadata is relative to the take off point. I know the Mavic 3M has a terrain follow feature and this may be worth looking at to keep a constant height. I'll be happy to let you know how I go once I get some python code running a bit better for this task.

Here is a link to the code used in the above mentioned paper, https://github.com/professorfabioandrade/georef.