How to create screenshots in your Selenium Webdriver tests

As they say, a picture often says more than a thousand words. This also applies to test execution reports – no matter the clarity of your error messages, often a screenshot of the status of your browser instance at the moment a particular error occurs is the most efficient way to record what has gone wrong. In this post, I will show you how to generate a screenshot in Selenium Webdriver, which you can then save to disk for later reference and/or include in your custom reports.

First, we need a simple test script in which a screenshot is created at a given moment. Here it is:

public static void runTest() {

	WebDriver driver = new FirefoxDriver();

	driver.get("http://parabank.parasoft.com");

	// log in
	driver.findElement(By.name("username")).sendKeys("john");
	driver.findElement(By.name("password")).sendKeys("demo");
	driver.findElement(By.xpath("//input[@value='Log In']")).click();
		
	//create screenshot
	createScreenshot(driver,"C:\\temp\\screen.png");

	driver.quit();
}

The createScreenshot method takes two arguments: the driver instance and a physical location where the created screenshot will be stored. The implementation of this method is pretty straightforward:

public static void createScreenshot(WebDriver driver, String location) {

	// generate screenshot as a file object
	File scrFile = ((TakesScreenshot)driver).getScreenshotAs(OutputType.FILE);
	try {
		// copy file object to designated location
		FileUtils.copyFile(scrFile, new File(location));
	} catch (IOException e) {
		System.out.println("Error while generating screenshot:\n" + e.toString());
	}
}

When we run this test, we see that a screenshot is created and stored in the location we provided (in my case, C:\temp):
Screenshot as created by our code

Creating screenshots when an alert is active
One of my readers, Kritika Gupta, nicely alerted me that the method described above does not work when there is a Javascript alert active. Instead, an UnhandledAlertException is thrown and no screenshot is created. This seems to be a known issue with Selenium, as can be read here. I have found a way to circumvent this problem by using the Java Robot class. It works as follows:

public static void createScreenshotUsingRobot(WebDriver driver, String location) {
		
	BufferedImage image = null;
	try {
		image = new Robot().createScreenCapture(new Rectangle(Toolkit.getDefaultToolkit().getScreenSize()));
	} catch (HeadlessException | AWTException e1) {
		e1.printStackTrace();
	}
	try {
		ImageIO.write(image, "png", new File(location));
	} catch (IOException e) {
		e.printStackTrace();
	}
}

When we use this method to create screenshot, it even works when Javascript alert messages are active, as can be seen in the screenshot below, which has been taken using the code above:
Screenshot taken using the Java Robot class
Please note that the code above takes a screenshot of your complete desktop (as you can see) instead of a screenshot from the active browser window only. Also, remember that this code does not take care of handling the popup. This should be done in your main test code.

The Eclipse project I used for this example can be downloaded here.

Five considerations before choosing an automated testing solution

Selecting a tool to aid your project or organisation with the implementation of automated software testing can be a daunting task. Many tools are available, each of them with its own benefits and drawbacks. The following considerations will provide you with some much-needed guidance in the test tool selection process.

What kind of technology is to be automated?
The choice for a specific automated testing tool heavily depends on the type of application(s) your tests are written for. Many tools are specifically designed for a single type of application, such as Selenium, which is targeted towards websites and web applications running in a browser. Other tools are able to handle a lot of different types of applications, such as browser applications, Windows applications and SAP. Tools that fall into the latter category tend to be more expensive, but might be the better choice if your test scripts cover different types of applications.

Do I go for open source or for commercially licensed tooling?
At first sight, going the open source route when selecting an automated testing tool might seem the most cost-effective option, due to the lack of license fees and support contract fees that typically come with commercial test tools. However, open source tools come with necessary investments of their own, as it often takes more time and more technical (programming) knowledge to set up and maintain automated test scripts. One often overlooked advantage of open source test tooling is the fact that the more popular ones are often supported a large community on the Internet. This community can be tapped into for technical support and often offers a library of additional features and extensions for the tool in case.

Do I prefer in house implementation or do I want to outsource test automation activities?
Another consideration to make is whether to have your own team implement the automated test scripts or to leave this to a third party. Keeping automated test development in house has a number of benefits, such as:

  • The people working on test automation are likely to be very familiar with the application(s) under test.
  • Knowledge gained on test automation is retained within the organisation.
  • The opportunity to work on test automation might be a good motivator for people in the organisation willing to learn something new.

On the other hand, outsourcing test automation implementation also has certain benefits, the most important being that dedicated test (automation) service providers likely have a lot of experience with the selected tool, enabling them to implement it much more efficiently.

How much training is required for the implementation of test automation?
Some tools require more technical knowledge upfront before users can successfully automate tests with it, while others do all they can to abstract from the technical details and offer an interface that makes it possible to implement and run automated tests for everybody. The former category of tools typically requires more training for them to be used. Another factor to be considered is the programming or scripting language used by the automated test tool. If, for instance, a tool is Java-based and all you have in your company are PHP developers, more training might be required than when your software development department – and the testers included in it – works with Java on a daily basis.

Does the tool match our current software testing process?
Finally, another thing worth considering is the extent to which the tool matches your current software testing process. Questions that you might want to ask with respect to this are:

  • Do the reports produced by the tool match the information requirements from the software development team and from management?
  • Does the tool integrate nicely with my current or desired automated build or continuous integration software delivery setup?
  • Can scripts be run by everybody within the software development and testing team? Note that this differs from who is going to develop the scripts.

When there is a mismatch between the features of the tool under consideration and your current software testing process, there are two further options. One is to consider a different tool that better fits the process, the other is to adapt the process to fit around the tool. The former is generally the more preferable, but in some cases where the testing tool fits the software to be tested perfectly, it might be worthwhile to adapt your testing process in some points to get a good match between the two.

Data driven testing using a test data database

In a previous post I explained how to set up a data driven test in Selenium Webdriver using data from an Excel worksheet. However, you might have your potential test data stored in a database rather than in Excel. In this post, I will show you how to set up and run a data driven test using data from a database. In this example, I will use a REST webservice as the object to be tested and therefore won’t use Selenium Webdriver, but you can easily apply this approach to your Selenium tests as well.

First, we need a table containing our test data. For this example, I have created a simple table in a local MySQL installation, containing the following data:

The table containing our test data

As our test object, I am going to use a public REST service API that returns, amongst other data, the city and state corresponding to a US zip code in JSON format (click here for an example).

Now, let’s create a test that calls this API for all zipcodes in our test data table and verify whether the city and state returned by the service match the expected values stored in our table.

To do that, we first need to create a connection to our database table, retrieve the test data from it and call our test method for every row in the table. This is done using the following piece of code:

public static void runTest() {
		
	try {
			
		// Retrieve database connection properties from the properties file
		String driver = DBDrivenProperties.getProperty("db.driver");
		String dburl = DBDrivenProperties.getProperty("db.url");
		String dbname = DBDrivenProperties.getProperty("db.dbname");
		String dbquery = DBDrivenProperties.getProperty("db.query");
		String dbuser = DBDrivenProperties.getProperty("db.username");
		String dbpassword = DBDrivenProperties.getProperty("db.password");
			
		// Load the MySQL JDBC driver
		Class.forName(driver);
			
		// Create a connection to the MySQL database
		Connection conn = DriverManager.getConnection(dburl + dbname, dbuser, dbpassword);
			
		// Create a statement to be executed
		Statement stmt = conn.createStatement();
			
		// Execute the query
		ResultSet rs = stmt.executeQuery(dbquery);
			
		// Loop through the query results and run the REST service test for every row
		while (rs.next()) {
			String zipcode = rs.getString("zipcode");
			String city = rs.getString("city");
			String state = rs.getString("state");
			try {
				testService(zipcode,city,state);
			} catch (IOException | JSONException e) {
				System.out.println(e.toString());
			}
		}
			
		// Close the database connection
		conn.close();
			
	} catch (ClassNotFoundException | SQLException e) {
		System.out.println(e.toString());
	}		
}

For this method to run, we need to add a MySQL JDBC driver to the classpath of our project, otherwise we get an error when we try to load the driver. You can get yours here.

For clarity, I have put all the configuration data that is needed to connect to the database and get the results from it in a separate properties file that looks like this:

The properties file

You can find the code that I use to retrieve the property values from this file in the project files. See the end of this post for a link to it.

Now that we have retrieved our test data from the database, let’s write the actual test method. This is very similar to the one I used in a previous post:

public static void testService(String zipcode, String city, String state) throws IOException, JSONException {
		
	System.out.println("Validating response for " + DBDrivenProperties.getProperty("rest.url") + zipcode + "...");
		
	// Retrieve the base URL for the REST service and append the zipcode parameter
	String restURL = DBDrivenProperties.getProperty("rest.url") + zipcode;
		
	// Call the REST service and store the response
	HttpUriRequest request = new HttpGet(restURL);
	HttpResponse httpResponse = HttpClientBuilder.create().build().execute(request);

	// Convert the response to a String format
	String result = EntityUtils.toString(httpResponse.getEntity());

	// Convert the result as a String to a JSON object
	JSONObject jo = new JSONObject(result);
		
	// Get the array containing the places that correspond to the requested zipcode
	JSONArray ja = jo.getJSONArray("places");
		
	// Assert that the values returned by the REST service match the expected values in our database
	Assert.assertEquals(city, ja.getJSONObject(0).getString("place name"));
	Assert.assertEquals(state, ja.getJSONObject(0).getString("state"));	
}

The only thing that is new compared to the code in my previous post on testing REST webservices is that in this response, the elements I am interested in (being the city and state corresponding to the zip code in the request) are stored within a result array called places. To retrieve these, I need to dig one level deeper into my JSON response object using the JSONArray object. Other than that, the test method is pretty straightforward.

One warning I need to address before you go ahead and create your own tests using test data from a database is that I used a very broad query in this example (a simple SELECT * FROM table). This potentially generates a lot of results and subsequently a lot of test iterations. Even though this gives great test coverage, it also takes more time to execute all test cases. Especially when you use this approach in combination with Selenium Webdriver, you might want to use a narrower query (or limit the number of results returned) to prevent your test from taking too long to finish.

The Eclipse project files including all code needed to get this to work can be downloaded here.