I have recently received a dataset that supposedly mapped the altitude a given region (a DEM). I would like to determine the accuracy of the new map by comparing it with a previously generated DEM. I am not sure what kind of metric I can use to ensure that the performance is good enough. (Good enough means that the horizontal deviation of a feature can not be off too far and that the vertical deviation can not be off too far)
One way to attack this would be to descritize the space into small chunks and then look at the distribution of the vertical residuals. If that block has vertical residuals that are less than the vertical deflection then I can assign that block a TRUE value and if it does not I could assign the block a FALSE value. I can then simply calculate the mean of the TRUE/FALSE values across all the blocks to get an overall value such as 67% or something which would tell me how good my mapping performance was.
There are huge problems with this method:
- It does not seem to factor in the horizontal deflection requirement
- It seems as though it would violate the iid (identical independently distributed) assumption normally required of the mean calculation. This is the same as stating that the individual boxes would be correlated.
- There does not seem to be a defensible method to choose how big the descritization box size should be.
An alternative could be that I simply place special markers with known 3D locations and then simply measure their vertical and horizontal displacement. This seems like it could share some of the same problems as the earlier method. (how many markers should I place? are the measurements correlated if they are too close?)
Before running off and doing something silly, I wanted to see if the community has a golden metric or method that has good statistical properties. Something that is continuously measured across the DEM, takes into account both the vertical deflection and the horizontal deflection, and I can look at the distribution and make a defensible argument? Please let me know if this question is unclear and thanks for any suggestions.
P.S. Could somebody could comment on the need of the two individual mapping accuracy levels (one for vertical deflection and one for horizontal deflection)? It seems to me that a horizontal mis-mapping would be reflected in a vertical deflection. Is it standard to have two separate requirements for mapping?