How to create screenshots in your Selenium Webdriver tests

As they say, a picture often says more than a thousand words. This also applies to test execution reports – no matter the clarity of your error messages, often a screenshot of the status of your browser instance at the moment a particular error occurs is the most efficient way to record what has gone wrong. In this post, I will show you how to generate a screenshot in Selenium Webdriver, which you can then save to disk for later reference and/or include in your custom reports.

First, we need a simple test script in which a screenshot is created at a given moment. Here it is:

public static void runTest() {

	WebDriver driver = new FirefoxDriver();

	driver.get("http://parabank.parasoft.com");

	// log in
	driver.findElement(By.name("username")).sendKeys("john");
	driver.findElement(By.name("password")).sendKeys("demo");
	driver.findElement(By.xpath("//input[@value='Log In']")).click();
		
	//create screenshot
	createScreenshot(driver,"C:\\temp\\screen.png");

	driver.quit();
}

The createScreenshot method takes two arguments: the driver instance and a physical location where the created screenshot will be stored. The implementation of this method is pretty straightforward:

public static void createScreenshot(WebDriver driver, String location) {

	// generate screenshot as a file object
	File scrFile = ((TakesScreenshot)driver).getScreenshotAs(OutputType.FILE);
	try {
		// copy file object to designated location
		FileUtils.copyFile(scrFile, new File(location));
	} catch (IOException e) {
		System.out.println("Error while generating screenshot:\n" + e.toString());
	}
}

When we run this test, we see that a screenshot is created and stored in the location we provided (in my case, C:\temp):
Screenshot as created by our code

Creating screenshots when an alert is active
One of my readers, Kritika Gupta, nicely alerted me that the method described above does not work when there is a Javascript alert active. Instead, an UnhandledAlertException is thrown and no screenshot is created. This seems to be a known issue with Selenium, as can be read here. I have found a way to circumvent this problem by using the Java Robot class. It works as follows:

public static void createScreenshotUsingRobot(WebDriver driver, String location) {
		
	BufferedImage image = null;
	try {
		image = new Robot().createScreenCapture(new Rectangle(Toolkit.getDefaultToolkit().getScreenSize()));
	} catch (HeadlessException | AWTException e1) {
		e1.printStackTrace();
	}
	try {
		ImageIO.write(image, "png", new File(location));
	} catch (IOException e) {
		e.printStackTrace();
	}
}

When we use this method to create screenshot, it even works when Javascript alert messages are active, as can be seen in the screenshot below, which has been taken using the code above:
Screenshot taken using the Java Robot class
Please note that the code above takes a screenshot of your complete desktop (as you can see) instead of a screenshot from the active browser window only. Also, remember that this code does not take care of handling the popup. This should be done in your main test code.

The Eclipse project I used for this example can be downloaded here.

Up and running with: Selendroid

This is the second article in our series on new, popular or otherwise interesting tools used in test automation. You can read all posts within this series by clicking here.

What is Selendroid?
Selendroid is a test automation framework for testing Android native and hybrid applications. Selendroid tests are written using the Selenium Webdriver client API, which allows for full integration with existing Selenium frameworks.

Where can I get Selendroid?
Selendroid can be downloaded from the Selendroid website.

How do I install and configure Selendroid?
Before you can start setting up Selendroid and writing tests, you need to download and install the latest Android SDK first. Clear instructions on how to do this can be found here. Make sure you also create at least one Android virtual device (AVD) and test whether it can be run properly, as we are going to need this virtual device to deploy our app under test and run our Selendroid tests on it.

NOTE: The instructions specify the creation of an AVD with the latest Android version (KitKat, API level 19). This is all wonderful, but if you’re (like me) working on a virtual machine or any other machine with limited resources, you’re probably better off creating an AVD based on Android Gingerbread (API level 10). You will need to install the correct SDK components for that API level using the Android SDK Manager. Oh, and it’s probably wise as well to use GPU emulation by enabling ‘Use host GPU’ in the AVD settings while you’re at it. This will make your test run much faster. Or, in my case, it will make your test run at all.

After you have successfully installed the Android SDK, you can install Selendroid following these instructions. When you’ve done so, start Selendroid and load the test app (the .apk file) they provide here. When you point your browser to

http://localhost:4444/wd/hub/status

you’ll see something similar to this:

Selendroid is running
As you can see, Selendroid is running and it has detected our newly created AVD.

Creating a first test script
Now we’re done setting up our environment we can go to the interesting part. To start mobile testing using Selendroid, create a new Java project, add the Selendroid and Selenium .jar files as dependencies and create the following script:

public static void runSelendroidTest() throws Exception {
		
	// specify test capabilities (your 'test environment')
	SelendroidCapabilities capa = new SelendroidCapabilities("io.selendroid.testapp:0.9.0");
		
	// explicitly state that we want to run our test on an Android API level 10 device
	capa.setPlatformVersion(DeviceTargetPlatform.ANDROID10);
		
	// explicitly state that we use an emulator (an AVD) for test execution rather than a physical device
	capa.setEmulator(true);

	// start a new WebDriver
	WebDriver driver = new SelendroidDriver(capa);
		
	// execute a very simple test
	WebElement inputField = driver.findElement(By.id("my_text_field"));
	Assert.assertEquals("true", inputField.getAttribute("enabled"));
	inputField.sendKeys("Selendroid");
	Assert.assertEquals("Selendroid", inputField.getText());
		
	// quit testing
	driver.quit();
		
}

The comments in the code really explain it all. You can instantly see that Selendroid tests are indeed very similar to regular Selenium tests, with the exception that they are run on an app that is deployed on an AVD rather than on a website that you access using a browser.

Running your test
Running your Selendroid tests is done just like you’d run Selenium tests, so there’s really no need to go into more detail on that here.

When you run your tests, you’ll see that an Android emulator (the AVD) is started, the app is loaded, tests are executed and the emulator is closed again. Selendroid can also be used to execute tests on apps that are deployed on physical Android devices connected to your machine, something I haven’t tried myself.

A screenshot of the test app

Useful features
Selendroid offers a lot of to the tester that wants to create more useful and more complex tests. For example, you can very accurately emulate a user swipe action from right to left using the following code snippet:

WebElement pages = driver.findElement(By.id("some_id"));
TouchActions flick = new TouchActions(driver).flick(pages, -100, 0, 0);
flick.perform();

Using the instructions presented here, just try and see for yourself what you can do with Selendroid. And of course, please share your experiences here.

Further reading
Again, the official Selendroid site can be found here. It offers a lot of information on the possibilities and features of Selendroid.

Happy mobile testing!

Create your own HTML report from Selenium tests

As I am learning more and more about using Selenium Webdriver efficiently (experiences I try to share through this blog), I’m slowly turning away from my original standpoint that user interface-based test automation is not for me. I am really starting to appreciate the power of Selenium, especially when you use proper test automation framework design patterns such as the Page Object pattern I wrote about earlier. However, Selenium by default lacks one vital aspect of what makes a good test automation tool to me: proper reporting options. Luckily, as Selenium is so open, there’s lots of ways to build custom reporting yourself. This post shows one possible approach.

My approach
Personally, I prefer HTML reports as they are highly customizable, relatively easy to build and can be easily distributed to other project team members. To build a nice HTML report, I use the following two step approach:

  • Execute tests and gather statistics about validations executed
  • Create HTML report from these statistics after test execution has finished

In this post I’ll use the following test script as an example. I created a page with five HTML text fields, for which I am going to validate the default text. Nothing really realistic, but remember it’s only used to illustrate my reporting concept.

public static void main (String args[]) {
		
	WebDriver driver = new HtmlUnitDriver();
	driver.get("http://www.ontestautomation.com/files/report_test.html");
		
	for (int i = 1; i <=5; i++) {
		WebElement el = driver.findElement(By.id("textfield" + Integer.toString(i)));
		Assert.assertEquals(el.getAttribute("value"), "Text field " + Integer.toString(i));
	}
		
	driver.close();	
}

When we run this script, one error is generated as text field 4 contains a different default value (go to the URL in the script to see for yourself).

Custom reporting functions
To be able to create a nice HTML report, we first need some custom reporting functions that store test results in a way we can reuse them later to generate our report. To achieve this, I created a report method in a Reporter class that stores validation results in a simple List:

public static void report(String actualValue,String expectedValue) {
	if(actualValue.equals(expectedValue)) {
		Result r = new Result("Pass","Actual value '" + actualValue + "' matches expected value");
		details.add(r);
	} else {
		Result r = new Result("Fail","Actual value '" + actualValue + "' does not match expected value '" + expectedValue + "'");
		details.add(r);
	}
}

The Result object is a simple class with three class variables: result (which is either Pass or Fail), a resultText (which is a custom description) and a URL for a screenshot (the use of which we will see later).

For every test we execute in our Selenium script, instead of using the standard TestNG / JUnit assertions, we use our own reporting function. You might want to throw an error as well when a validation fails, but I’m happy just to write it to my report and let test execution continue.

After test execution is finished, we are going to write the test results we gathered to a file. For this, I used an extremely simple HTML template (yes, I was too lazy even to indent it properly):

<html>
<head>
<title>Test Report</title>
<head>
<body>
<h3>Test results</h3>
<table>
<tr>
<th width="10%">Step</th>
<th width="10%">Result</th>
<th width="80%">Remarks</th>
</tr>
<!-- INSERT_RESULTS -->
</table>
</body>

In this template I am going to insert my test results, using a simple replace function

public static void writeResults() {
	try {
		String reportIn = new String(Files.readAllBytes(Paths.get(templatePath)));
		for (int i = 0; i < details.size();i++) {
			reportIn = reportIn.replaceFirst(resultPlaceholder,"<tr><td>" + Integer.toString(i+1) + "</td><td>" + details.get(i).getResult() + "</td><td>" + details.get(i).getResultText() + "</td></tr>" + resultPlaceholder);
		}
			
		String currentDate = new SimpleDateFormat("dd-MM-yyyy").format(new Date());
		String reportPath = "Z:\\Documents\\Bas\\blog\\files\\report_" + currentDate + ".html";
		Files.write(Paths.get(reportPath),reportIn.getBytes(),StandardOpenOption.CREATE);
			
	} catch (Exception e) {
		System.out.println("Error when writing report file:\n" + e.toString());
	}
}

Finally, we need to use these custom reporting functions in our test script:

public static void main (String args[]) {
		
	WebDriver driver = new HtmlUnitDriver();
	Reporter.initialize();
	driver.get("http://www.ontestautomation.com/files/report_test.html");
		
	for (int i = 1; i <=5; i++) {
		WebElement el = driver.findElement(By.id("textfield" + Integer.toString(i)));
		Reporter.report(el.getAttribute("value"), "Text field " + Integer.toString(i));
	}
		
	Reporter.writeResults();
	driver.close();	
}

The initialize() method simply clears all previous test results by emptying the List we use to store our test results. When we run our test, the following HTML report is generated:

The HTML test report

Here, we can clearly see that our test results are now available in a nicely readable (though not yet very pretty) format. In one of my next posts, I am going to enhance these HTML reporting functions with some additional features, such as:

  • Screenshots in case of errors
  • Use of stylesheets for eye candy
  • Test execution statistics

Hopefully the above will get you started creating nicely readable HTML reports for your Selenium tests!

The Eclipse project used in the above example can be downloaded here. The HTML report template can be downloaded here (right click, save as).