3

I have a few indoor map construction algorithms and the ground truth map. I am trying to compare their construction accuracies w.r.t. the ground truth map.

Like any other scientific evaluations, mere visual comparisons are not enough. I need to quantize the errors so that I can claim which one is superior.

I am only interested in the corridors. So basically by construction accuracy, I mean how accurately the corridors (the corridor walls) are constructed. So I am indifferent about the room walls, etc.

I find this not easy, because this problem lie in between image processing and graph theory. On the one hand, I cannot simply compare the two images with image processing techniques, because I am not interested those trivial details mentioned above. On the other hand, the constructed map is not a collection of vertices and edges. So graph theory may not work here, either.

I have thought about overlapping the constructed on over the ground truth and see how well they are overlapped. A problem with this is what if my constructed map is of a slightly different scale with the original one? Then they will overlap very poorly, albeit the constructed map is actually ok.

So is there any good standard way of comparing two indoor maps?


An Example

Ground truth is the background floor map. The red curve is what the algorithm constructs. I wish to compare and quantify the construction errors. (The thickness of the red curve is only for visual purposes. It is actually a deterministic mathematically expressed curve)

enter image description here

  • 2
    Can you post an example clip of the three maps? Indoor maps is not a term often used on this group and we should see what you specifically call an indoor map. – Brad Nesom Oct 23 '13 at 5:15
  • @BradNesom Thanks for the advice! Done as told. Please check. :) – Sibbs Gambling Oct 23 '13 at 5:29
  • If you georeference the two maps using ground control points, will it not stretch the two maps to the same scale? Then you could overlap them and visualise the difference a bit better? – Rob Lodge Oct 23 '13 at 7:53
  • @RobLodge I need numerical analysis so as to be scientific.. – Sibbs Gambling Oct 23 '13 at 8:44
  • You may find this link helpful: gis.stackexchange.com/questions/26952/… If you were to trace your two areas of interest and create a series of vertices for your two indoor maps, compute their respective geometries and then overlay them, you could carry out the aforementioned distance calculation to quantify your accuracy. – Rob Lodge Oct 23 '13 at 9:09
1

Maybe you can derive something from the work of Furukawa et al. at University of Washington. They do Structure-from-motion and probably have buried some ground truth comparisons in their papers. I just found http://homes.cs.washington.edu/~furukawa/papers/iccv09.pdf as a starting point, and they also have a "interior project" at http://grail.cs.washington.edu/projects/interior/ .

1

If you use the georeferencing tools.
Selecting the ground truth as target, and set it to 3rd order polynomial, collect all the points you are interested in and output the overall rms, and all the point residuals.
You can choose the save button (upper left) and get a file output.
Do you need more scientific than that?

residual

You would need to make one of your images transparent so it could rest on top of the other and still be able to see the pertinent reference points.

transparent

example shown dependent on your colorspace and the background color white.

esri Help for georeferencing

  • Thanks! I am confused about collect all the points you are interested in and output the overall rms, and all the point residuals. Also, then how is the error quantified please? I did not get it. – Sibbs Gambling Oct 23 '13 at 15:30
  • If I had actually been doing a real project the overall RMS would be displayed in the lower portion of the link window. As I stated. Do you need more scientific than this? because in GIS the only other ground truth analysis we would normally ask for is the ppm which would require a much larger sampling area to complete. It may be that you need to migrate this question to a more analytical site. – Brad Nesom Oct 23 '13 at 15:53
  • perhaps you could take the ouptu x,y, rms and do some statistical analysis on those data. – Brad Nesom Oct 23 '13 at 15:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.