Take the 2-minute tour ×
Geographic Information Systems Stack Exchange is a question and answer site for cartographers, geographers and GIS professionals. It's 100% free, no registration required.

I'm trying to do an objective comparison of two different maps for the same region. At the moment, I'm struggling with defining criteria that will allow me to do a dispassionate evaluation.

Does anyone have any ideas on how to do this, or how I should approach the problem?

As you can see, neither map is superior, a few gaps on the blue set, a few on the red.

enter image description here

share|improve this question
7  
The question is very poorly defined ? What do you mean by quality ? Are there any presumptions about completeness, accuracy, geometry ... ? Is there a third reference dataset against which to compare ? –  petzlux Feb 4 '13 at 14:47
    
Just a thought, why not compare them using high resolution aerial imagery? Maybe even convert them to KML and assess their accuracy in Google Earth. –  Aaron Feb 4 '13 at 15:00
2  
@Martin, Are you asking for a method for highlighting the geometric differences of the two layers? –  artwork21 Feb 4 '13 at 15:10
1  
There is a lot of literature on the topic. Some papers to get you started: underdark.wordpress.com/projects/… –  underdark Feb 4 '13 at 16:49
1  
Doesn't this question really have little to do with cartography or maps but rather data quality? Perhaps you could rephrase your question and provide more context on the data quality side. –  blah238 Feb 4 '13 at 18:03
show 1 more comment

3 Answers 3

up vote 12 down vote accepted

This reply describes an objective method to measure arbitrary discrepancies between two spatial datasets. Such discrepancies can include shifts of position, changes of shape, and features present in one dataset but not in another. This reply does not provide any means to determine which is "better," because that depends on much more than just the data and it particularly depends on what the data will be used for.

Background

A good foundation for a large set of such measurements relies on the Euclidean distance transform of each dataset. This views each dataset as representing a collection of points in the plane. Let's call these collections B for the blue features and R for the red features.

For any point x in the plane, the Euclidean distance transform of a point set A computes the greatest lower bound of the distances between x and A. We may think of this transform as creating a "surface" whose height at x equals the shortest distance from x to A. Thus this surface has valleys at all points of A, where its height is zero, and rises at a 1:1 slope away from A. It is clear that the distance transform in turn determines A (or technically its metric closure, which for GIS datasets is the same as A) as the set of all points at a height of zero. Thus the distance transform completely captures all the spatial information of A that the GIS is able to represent.

Figure 1

This figure shows the distance transforms of B (at the left) and R (at the right) in pseudo-relief.

Comparing two datsets

To compare B and R, overlay each with the distance transform of the other:

Figure 2

The distance values are shown as colors graduated from blues (near 0) through reds.

The left map, for instance, shows the points of B and colors them according to their distances from R. The roles of B and R are switched in the right map.

Already these aid the eye in making comparisons: each map shows the points of one dataset and, by its use of color, emphasizes the points that are far from any point in the other dataset. Note that both maps are needed for the comparison, because each shows points not on the other.

On detailed maps, the color can be difficult to see, so we might choose to blur it a little for presentation or visual evaluation:

Figure 3

NB: The colors are not comparable between the two maps: within each map they are scaled to show the full range of distances in that map.

Statistical analysis of the differences

The beauty of this approach lies in what can be done in post-processing. Using a raster to represent the distance transforms and their overlays, we can easily obtain statistics--local and global--to measure the discrepancies. For instance, we could focus on all distances larger than some small threshold explore their frequency distribution:

Figure 4

In this histogram blue bars are for the blue features, red bars for the red features. (Note the logarithmic scale on the horizontal axis.) This histogram shows the original overlaid data, not the derivative blurred data. It has selected only those distances larger than three pixels in the original image.

These histograms show that it is much more likely for blue features to lie far from red features than vice versa: the blue bars are higher than the red and they extend out to greater distances (at the right). The whole arsenal of descriptive statistics is now available for quantifying the differences between the two datasets. These statistics can be applied to the entire region of interest or "windowed" over it to explore how the two datasets differ according to location.

Implementation

Most raster GISes provide a Euclidean distance transform (such as EuclideanDistance in ArcGIS and r.grow.distance in GRASS), and all support the simple (masking) overlay needed to do this analysis. Blurring, if desired, can be done with a neighborhood mean or kernel convolution (which includes the "Gaussian blur" available in all image processing software). Most GISes do not provide adequate support for full statistical analysis of raster data, though, but they are good at exporting such data in formats readable by statistical and mathematical software such as R or Mathematica (which made all the figures here).

share|improve this answer
    
As always, your answer is a fascinating read, and I get find new things to learn & think about. Could you also tell me how you created the 'hand-drawn' type of graph? –  Devdatta Tengshe Feb 5 '13 at 5:49
2  
@Devdatta The histogram was created in Mathematica by post-processing with a modification of code at mathematica.stackexchange.com/questions/11350/xkcd-style-graphs. Additional examples of such graphics are at stats.stackexchange.com/questions/49123/… and stats.stackexchange.com/questions/48973/… and stats.stackexchange.com/questions/48467/…. –  whuber Feb 5 '13 at 12:15
add comment

Since it is a geometry validation problem from the look of it, load current imagery and see if you can validate the streets as existing or as present. Likely one data set is newer and / or more complete. Find out which dataset has the greatest horizontal accuracy and which has been collected most recently and by what process. Is it a difference between tiger, osm, gis, and gps traces? Imagery is your friend here, at least accurate imagery like HAIP of recent vintage. The criterion might be, completeness, coverage, accuracy, correctness, currency.

share|improve this answer
add comment

I think following on from the comments and answers above (perhaps this should also be a comment rather than an answer) but i would use what ever mapping i have available to validate the files by visual comparison. Pick features where there is accurate plotted data available..such as road networks (in England Scotland and Wales is freely available from OS).

Following on from the suggestion to use google earth and imagery. Use the openlayers plugin to load satellite imagery into the background and compare again.

A lot of free accurate maps out there to use as a base for comparison.

share|improve this answer
    
Unfortunatly, I started to make this comparison based on the fact, that OSM and other OpenSource maps are really not as accurate as we seem to think. But the input on using satellite images is a lead to follow! –  Styp Feb 4 '13 at 16:25
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.