There are several commercial and opensource tools for regression automation. While many of them could be employed to test GIS apps, they fall short, at times, dealing with the unique characteristics and use-cases of GIS apps and their imagery.
What are some tools for validating GIS apps?
I came across one such tool called Eggplant. Check out their video link in the bottom. They claim to not only compare data sets but also compare images. Seems interesting!
Are there other such commercial or opensource tools we can evaluate?