Sunday, February 17, 2013

Validating WebDriver Tests Without the WebDriver API

One thing we have noticed working on WebTestingExplorer is just how excruciatingly slow WebDriver can be. Perhaps you have never noticed this because you only validate a few elements at the end of a WebDriver test. But in WebTestingExplorer use cases, we often need to examine hundreds and hundreds of WebElements and their attribute values after each action. And here, WebDriver's architecture where every call to the browser is a local RPC is a massive fail in terms of performance (and, I think, stability, although that is harder to pin down).

This is one of the (many) reasons I wish the testing practice was not standardizing on WebDriver: I feel like browser automation is far from a solved problem and would like to see what other smart people can continue to put forward. Nonetheless, we feel like we have to use it for WebTestingExplorer, both to provide synergy and integration between our tool and other testing efforts, and because there just aren't a lot of good alternatives under active development.

That said, we had been theorizing for awhile that we could improve life significantly if we started trying to implement WebTestingExplorer state checking and oracles on top of an alternative HTML-parsing library. A couple of weeks ago, I selected jsoup as the library and finally took a shot at this. The results were really incredible -- an order of magnitude or more speedup in many of our exploration and bug-checking scenarios, and fewer (read: zero) inexplicable, random exceptions flying out of WebDriver. Going forward, I hope to try and rework some of the WebTestingExplorer APIs to make this more natural.

But even if you're not using WebTestingExplorer, I think the overall approach is worth considering for any web unit/system/integration testing you might be doing. The idea is to command the browser using WebDriver as you normally do. Then, having completed that, grab the current DOM HTML of the page with WebDriver like this:

WebDriver driver;
// ...
JavascriptExecutor js = (JavascriptExecutordriver;
String domString = (Stringjs.executeScript("return document.documentElement.outerHTML;");

Done with WebDriver, feed that to the jsoup parser and enjoy its simple, fast API while writing your assertions. Yeah, you lose all the fancy element-is-displayed logic that WebDriver is constantly trying to perform, but hopefully (as in, you would think) it's not too hard to write assertions that don't care much about that. Enjoy!

No comments: