Friday, December 21, 2012

Where to Find cybervillainsCA.cer in Selenium

In the WebTestingExplorer documentation, we instruction users to create a Firefox profile (just for testing!!!) that trusts the cybervillainsCA.cer certificate normally bundled with Selenium distributions. The main reason we do this is to make it possible to inspect HTTPS traffic in a proxy and check for bugs in real-time as we explore the web application.

I've had several questions about how, exactly, to find cybervillainsCA.cer. The search results on this topic are unimpressive enough that I decided to write this quick blog post, hopefully attract some search engine love, and clear this up for everybody. So here you go:

1. Download selenium-server-standalone-x.xx.x.jar (as of this writing, at version 2.28.0).
2. Execute the following commands from a terminal from wherever you saved it:
  1. mkdir selenium
  2. cp selenium-server-standalone-*.jar selenium
  3. cd selenium
  4. jar xvf selenium-server-standalone-*.jar
  5. ls sslSupport/cybervillains.cer
After step 4, you may get an error from jar. Ignore it. After step 5, you have verified that the file is there, and you can do whatever you need to with it. Enjoy!

Sunday, December 16, 2012

Waiting for GWT RPCs in WebDriver Tests

When writing WebDriver tests for GWT applications, we often find ourselves writing WebDriverWait's to synchronize the tests with asynchronous GWT RPC's. These waits are often based on some DOM-observable result of the RPC completing -- for example, waiting until a label is changed by code in the onSuccess handler.

This has a couple of drawbacks. First, we would prefer to assert on the results of the RPC, not rely on them to control our tests. An assertion failure is preferable to a wait timeout when the onSuccess handler does something incorrectly. Second, if the behavior of RPC handling changes, our wait conditions will break and have to be maintained.

We would rather attach our wait condition to the RPC itself, rather than side-effects. Unfortunately, GWT's RPC mechanism does not build in anything for this, and we have to do it ourselves. In the remainder of this post, I will outline a simple (perhaps simplistic) way to do this.

The overall approach is to maintain a hidden field as the RPC goes out and comes back. In the .ui.xml:

<g:Hidden ui:field="ready" name="ready"/>

And the UiBinder declaration:

@UiField Hidden ready;

We'll create a helper method that updates this field based on whether we are "ready" or "busy" running an RPC:

public void setReady(boolean isReady) {
  if (isReady) {
    ready.setValue("ready");
  } else {
    ready.setValue("busy");
  }
}

Then our RPC call will follow this pattern:

public void onAction() {
  view.setReady(false);
  service.getData(new AsyncCallback<String>() {
    @Override
    public void onFailure(Throwable caught) {
      // Show failure message...
      setReady(true);
    }

    @Override
    public void onSuccess(String result) {
      // Show the result...
      setReady(true);
    }
  });  
}

Now in our WebDriver test, we can write a WebDriverWait like this:

WebDriverWait wait = new WebDriverWait(driver, 5000);
wait.until(ExpectedConditions.textToBePresentInElementValue(By.id("ready"), "ready");

There are some obvious drawbacks to this simplistic approach -- for example, it is not very extensible to an environment with multiple or batched RPC's. In a future post, I'll take on some of those challenges.

Saturday, November 3, 2012

Manually Deleting the WebDriver Firefox Profile Directory

Some time ago, I starred a Selenium WebDriver bug that has a long discussion about how WebDriver leaves the Firefox profile directory behind in tmp after the profile goes away. Allegedly it was fixed at some point, but it still gets pinged from time to time, so who really knows.

Usually you will discover this when you are running a long set of tests that don't reuse the same Firefox, and then your hard drive fills up. This can be, well, a little frustrating. But on the long list of known WebDriver issues that never seem to really get fixed, this one is fairly minor, because it's simple to work around.

The comments in the bug suggest an @AfterClass method that uses Selenium's TemporaryFileSystem class to do the cleanup. This seems pretty satisfactory for most purposes. Nonetheless, depending on what else is going on behind the scenes, you may not want to clean out the entire Selenium-managed temporary directory in the middle of a test run.

I thought I would explain how we accomplish the same thing in WebTestingExplorer. We have a wrapper around the WebDriver instance that we use. It has a close() method that gets called between test cases. One of the things we do in that close() method is work around this bug:


    List<String> profileDirs = Lists.newArrayList("anonymous*webdriver-profile",
        "userprofile*copy", "seleniumSslSupport*");
    File tmpDir = new File(System.getProperty("java.io.tmpdir"));
    FilenameFilter profileDirsFilter = new WildcardFileFilter(profileDirs);
    File[] files = tmpDir.listFiles(profileDirsFilter);
    for (File profileDir : files) {
      LOGGER.info("Cleaning up tmp profile directory: " + profileDir.getAbsolutePath());
      try {
        FileUtils.deleteDirectory(profileDir);
      } catch (IOException e) {
        LOGGER.log(Level.WARNING,
            "Failed to delete tmp profile directory: " + profileDir.getAbsolutePath(), e);
      }
    }


As you can see, we fairly strategically delete the things that we found Selenium and Firefox can create each time a new FirefoxProfile/FirefoxDriver gets instantiated. Theoretically, if this bug ever gets fixed-fixed, we could take this code out, but actually, I don't see any reason not to just leave it forever. It's better than having the issue regress on us.

Sunday, October 28, 2012

Adding Guice to the GWT Sample Application

When you work on pre-existing projects most of the time, it's easy to take for granted the effort that can be involved in setting up all of the fancy infrastructure you've become accustomed to. I set up a new GWT application from scratch last week and while adding Guice support to the server side, I wasted some time because I forgot to disable the servlet that the Google Eclipse Plugin "New Project" wizard set up for me in web.xml.

So for fun, I looked on the web and was rather underwhelmed by the instructions for setting up Guice particularly in the GWT context. To help other people (including my future self), I will walk through it here, in a coherent manner in a single place. The following assumes you are starting from the result of the GWT "New Project" wizard (I called my project "GuiceStarter") and want to Guice-enable the GreetingService.

1. Download Guice, copy the .jar files into war/WEB-INF/lib, and add them to the Eclipse Build Path.
2. Create a module in the "server" package as follows:

public class GuiceStarterAppModule extends AbstractModule {
  @Override
  protected void configure() {
    bind(GreetingServiceImpl.class);
  }
}

3. Create the servlet listener config in the "server" package as follows:

public class GuiceServletConfig extends GuiceServletContextListener {

  @Override
  protected Injector getInjector() {
    return Guice.createInjector(new ServletModule() {
      @Override
      protected void configureServlets() {
        serve("/guicestarter/greet").with(GreetingServiceImpl.class);
      }
    }new GuiceStarterAppModule());
  }
}

4. In web.xml, take out the existing and for greetServlet.

5. In web.xml, add the Guice configuration:


<listener>
    <listener-class>com.scottmcmaster365.weatherapp.server.GuiceServletConfig</listener-class>
</listener>

<filter>
    <filter-name>guiceFilter</filter-name>
    <filter-class>com.google.inject.servlet.GuiceFilter</filter-class>
</filter>

<filter-mapping>
    <filter-name>guiceFilter</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>

6. Test this by modifying GreetingServiceImpl as follows:

@Singleton
public class GreetingServiceImpl extends RemoteServiceServlet implements
    GreetingService {

  @Inject
  public GreetingServiceImpl() {  // set a breakpoint here
  }
  // etc.

7. Set a breakpoint in the constructor, debug the Web Application, and make sure the breakpoint gets hit.
8. Start using Guice on the server-side as you normally would.

In a future post, I may also add Guice (Gin) to the client-side.

Friday, October 19, 2012

Selenium FirefoxProfile setProxyPreferences is @Deprecated++

I recently updated the Selenium and BrowserMob Proxy versions in WebTestingExplorer. WebTestingExplorer will proxy requests through BrowserMob, primarily to capture the HTTP response code because this is bafflingly but intentionally not supported in the WebDriver API. While you're here, surf over to Google Code and put a star on Selenium issue #141.

After I did this, I watched some tests running in WebTestingExplorer and noticed that certain HTTP form POSTs were not responding as expected. I've always thought that the WebDriver+BrowserMob setup seemed a little fragile (another reason why you should go star Selenium issue #141), so I instantly suspected it, and sure enough, when I pulled out the proxy, the form POSTs worked fine.

I took another look at the code where we set the proxy into the FirefoxProfile, which looked like this:

  @Override
  public WebDriver createWebDriver(WebDriverProxy proxy) throws Exception {
    if (proxy != null) {
      profile.setProxyPreferences(proxy.getSeleniumProxy());
    }
    return new FirefoxDriver(profile);
  }

What you can't see here but I saw in Eclipse was the @Deprecated strikethrough on the call to FirefoxProfile.setProxyPreferences: "This is now handled by the driver itself."

I figured that while I was settling in to debug this code anyway, I would remove the deprecation warning. So I updated to the new style as follows:

  @Override
  public WebDriver createWebDriver(WebDriverProxy proxy) throws Exception {
    DesiredCapabilities capabilities = DesiredCapabilities.firefox();
    capabilities.setCapability(FirefoxDriver.PROFILE, profile);
    if (shouldUseProxy) {
      capabilities.setCapability(CapabilityType.PROXY, proxy);
    }
    return new FirefoxDriver(capabilities);
  }

I run through my tests again. Now it works! Even though it is generally considered bad manners to break something when you @Deprecate it, I'm quite suspicious that the Selenium folks did that here.

Wednesday, January 11, 2012

WebTestingExplorer Documentation Moving Forward

I started writing some documentation for WebTestingExplorer in the site Wiki. Because I would like people to start trying it out, I'm hoping that it will help them get started without necessarily having to pour over all the code and Javadocs. The core API documentation is (and always will be) in the Javadocs, but I will use the Wiki to explain some of the higher-level concepts and design philosophies. So if you get the chance, download, build, try it out, and, of course, let me know how it goes.

Sunday, January 1, 2012

Suppressing Firefox favicon Requests from WebDriver

Right now, WebTestingExplorer only works with Firefox. There are a couple of reasons for that which I won't go into here. Anyway, we decided to wire our HTTP requests to go through a proxy by default. This allows us to look for specific status codes, presumably "undesirable" ones like the 500-series, or possibly 404's.

When we did this and pointed the tool at a specific URL, the first thing we noticed was Firefox firing off a bunch of requests to various locations on the target site fishing around for the favicon, most of which (kind of by definition) ended with a 404. I'm surprised that for as long as I have used Firefox attached to a proxy for testing and debugging, I had never taken note of this before. In this case, it's really annoying -- after all, one of the points of using the proxy in the first place was to verify that all requests came back with a valid response.

The first thing I considered was adding special-case code for all favicon requests in the handlers for the proxy. That seemed kind of lame, so the second thing I thought about was looking for a way to suppress the requests in the first place. It turns out that there are Firefox preferences for this -- browser.chrome.favicons and browser.chrome.site_icons -- as described here.

I have two main points left for this post. First, I wanted to show the code for how set these preferences in Selenium WebDriver when you build your FirefoxProfile:

final FirefoxProfile profile = new FirefoxProfile();

// The following preferences control Firefox's default behavior of sending
// requests for favicon.ico, which results in lots of bogus 404's for sites
// that don't have favicons. 
profile.setPreference("browser.chrome.favicons"false);
profile.setPreference("browser.chrome.site_icons"false);
    
FirefoxDriver driver = new FirefoxDriver(profile);

Second, I will pitch my opinion that in most automated testing scenarios, you probably ought to be suppressing favicon requests from Firefox, even if your site actually has one. A good way to do this is to save the preferences into a custom Firefox profile used for automated testing. If you don't already have one of those, you should. I will try to provide some tips on how to maintain and use one of those in a future post.

Introducing WebTestingExplorer

My friend and I recently started working on a new open-source project, WebTestingExplorer (hosted on code.google.com). We are setting out to do a high-quality, industrial-strength implementation of some of the ideas we worked on in graduate school at the University of Maryland, specifically applied to web application testing. The super-high-level idea is to incrementally "explore" the web user interface, simultaneously looking for bugs and generating test cases. Although the project is in a very early state, the current implementation can do a few useful things, such as:
  • Detecting unexpected HTTP status codes and Javascript errors during exploration.
  • Generating and replaying regression test cases, looking for differences in state.
A few tenets we have adopted are:
  • Most everything regarding the exploration and test case replay process is configurable.
  • Configuration is done primarily via code, not command-line arguments or XML files.
  • The application-under-test is inherently "testable" (or we can make it so), such as having unique id's for important HTML elements, and hooks that enable reliable timing of actions.
If you are interested in automated test case generation and/or web application testing, feel free to start playing with it and sending feedback through the Google Code project. Although the primary vehicle for project communication will be the Project Wiki, I hope to blog here about some of the interesting issues we encounter. So until next time...