Up and running with: Selendroid

This is the second article in our series on new, popular or otherwise interesting tools used in test automation. You can read all posts within this series by clicking here.

What is Selendroid?
Selendroid is a test automation framework for testing Android native and hybrid applications. Selendroid tests are written using the Selenium Webdriver client API, which allows for full integration with existing Selenium frameworks.

Where can I get Selendroid?
Selendroid can be downloaded from the Selendroid website.

How do I install and configure Selendroid?
Before you can start setting up Selendroid and writing tests, you need to download and install the latest Android SDK first. Clear instructions on how to do this can be found here. Make sure you also create at least one Android virtual device (AVD) and test whether it can be run properly, as we are going to need this virtual device to deploy our app under test and run our Selendroid tests on it.

NOTE: The instructions specify the creation of an AVD with the latest Android version (KitKat, API level 19). This is all wonderful, but if you’re (like me) working on a virtual machine or any other machine with limited resources, you’re probably better off creating an AVD based on Android Gingerbread (API level 10). You will need to install the correct SDK components for that API level using the Android SDK Manager. Oh, and it’s probably wise as well to use GPU emulation by enabling ‘Use host GPU’ in the AVD settings while you’re at it. This will make your test run much faster. Or, in my case, it will make your test run at all.

After you have successfully installed the Android SDK, you can install Selendroid following these instructions. When you’ve done so, start Selendroid and load the test app (the .apk file) they provide here. When you point your browser to

http://localhost:4444/wd/hub/status

you’ll see something similar to this:

Selendroid is running
As you can see, Selendroid is running and it has detected our newly created AVD.

Creating a first test script
Now we’re done setting up our environment we can go to the interesting part. To start mobile testing using Selendroid, create a new Java project, add the Selendroid and Selenium .jar files as dependencies and create the following script:

public static void runSelendroidTest() throws Exception {
		
	// specify test capabilities (your 'test environment')
	SelendroidCapabilities capa = new SelendroidCapabilities("io.selendroid.testapp:0.9.0");
		
	// explicitly state that we want to run our test on an Android API level 10 device
	capa.setPlatformVersion(DeviceTargetPlatform.ANDROID10);
		
	// explicitly state that we use an emulator (an AVD) for test execution rather than a physical device
	capa.setEmulator(true);

	// start a new WebDriver
	WebDriver driver = new SelendroidDriver(capa);
		
	// execute a very simple test
	WebElement inputField = driver.findElement(By.id("my_text_field"));
	Assert.assertEquals("true", inputField.getAttribute("enabled"));
	inputField.sendKeys("Selendroid");
	Assert.assertEquals("Selendroid", inputField.getText());
		
	// quit testing
	driver.quit();
		
}

The comments in the code really explain it all. You can instantly see that Selendroid tests are indeed very similar to regular Selenium tests, with the exception that they are run on an app that is deployed on an AVD rather than on a website that you access using a browser.

Running your test
Running your Selendroid tests is done just like you’d run Selenium tests, so there’s really no need to go into more detail on that here.

When you run your tests, you’ll see that an Android emulator (the AVD) is started, the app is loaded, tests are executed and the emulator is closed again. Selendroid can also be used to execute tests on apps that are deployed on physical Android devices connected to your machine, something I haven’t tried myself.

A screenshot of the test app

Useful features
Selendroid offers a lot of to the tester that wants to create more useful and more complex tests. For example, you can very accurately emulate a user swipe action from right to left using the following code snippet:

WebElement pages = driver.findElement(By.id("some_id"));
TouchActions flick = new TouchActions(driver).flick(pages, -100, 0, 0);
flick.perform();

Using the instructions presented here, just try and see for yourself what you can do with Selendroid. And of course, please share your experiences here.

Further reading
Again, the official Selendroid site can be found here. It offers a lot of information on the possibilities and features of Selendroid.

Happy mobile testing!

Create your own HTML report from Selenium tests

As I am learning more and more about using Selenium Webdriver efficiently (experiences I try to share through this blog), I’m slowly turning away from my original standpoint that user interface-based test automation is not for me. I am really starting to appreciate the power of Selenium, especially when you use proper test automation framework design patterns such as the Page Object pattern I wrote about earlier. However, Selenium by default lacks one vital aspect of what makes a good test automation tool to me: proper reporting options. Luckily, as Selenium is so open, there’s lots of ways to build custom reporting yourself. This post shows one possible approach.

My approach
Personally, I prefer HTML reports as they are highly customizable, relatively easy to build and can be easily distributed to other project team members. To build a nice HTML report, I use the following two step approach:

  • Execute tests and gather statistics about validations executed
  • Create HTML report from these statistics after test execution has finished

In this post I’ll use the following test script as an example. I created a page with five HTML text fields, for which I am going to validate the default text. Nothing really realistic, but remember it’s only used to illustrate my reporting concept.

public static void main (String args[]) {
		
	WebDriver driver = new HtmlUnitDriver();
	driver.get("http://www.ontestautomation.com/files/report_test.html");
		
	for (int i = 1; i <=5; i++) {
		WebElement el = driver.findElement(By.id("textfield" + Integer.toString(i)));
		Assert.assertEquals(el.getAttribute("value"), "Text field " + Integer.toString(i));
	}
		
	driver.close();	
}

When we run this script, one error is generated as text field 4 contains a different default value (go to the URL in the script to see for yourself).

Custom reporting functions
To be able to create a nice HTML report, we first need some custom reporting functions that store test results in a way we can reuse them later to generate our report. To achieve this, I created a report method in a Reporter class that stores validation results in a simple List:

public static void report(String actualValue,String expectedValue) {
	if(actualValue.equals(expectedValue)) {
		Result r = new Result("Pass","Actual value '" + actualValue + "' matches expected value");
		details.add(r);
	} else {
		Result r = new Result("Fail","Actual value '" + actualValue + "' does not match expected value '" + expectedValue + "'");
		details.add(r);
	}
}

The Result object is a simple class with three class variables: result (which is either Pass or Fail), a resultText (which is a custom description) and a URL for a screenshot (the use of which we will see later).

For every test we execute in our Selenium script, instead of using the standard TestNG / JUnit assertions, we use our own reporting function. You might want to throw an error as well when a validation fails, but I’m happy just to write it to my report and let test execution continue.

After test execution is finished, we are going to write the test results we gathered to a file. For this, I used an extremely simple HTML template (yes, I was too lazy even to indent it properly):

<html>
<head>
<title>Test Report</title>
<head>
<body>
<h3>Test results</h3>
<table>
<tr>
<th width="10%">Step</th>
<th width="10%">Result</th>
<th width="80%">Remarks</th>
</tr>
<!-- INSERT_RESULTS -->
</table>
</body>

In this template I am going to insert my test results, using a simple replace function

public static void writeResults() {
	try {
		String reportIn = new String(Files.readAllBytes(Paths.get(templatePath)));
		for (int i = 0; i < details.size();i++) {
			reportIn = reportIn.replaceFirst(resultPlaceholder,"<tr><td>" + Integer.toString(i+1) + "</td><td>" + details.get(i).getResult() + "</td><td>" + details.get(i).getResultText() + "</td></tr>" + resultPlaceholder);
		}
			
		String currentDate = new SimpleDateFormat("dd-MM-yyyy").format(new Date());
		String reportPath = "Z:\\Documents\\Bas\\blog\\files\\report_" + currentDate + ".html";
		Files.write(Paths.get(reportPath),reportIn.getBytes(),StandardOpenOption.CREATE);
			
	} catch (Exception e) {
		System.out.println("Error when writing report file:\n" + e.toString());
	}
}

Finally, we need to use these custom reporting functions in our test script:

public static void main (String args[]) {
		
	WebDriver driver = new HtmlUnitDriver();
	Reporter.initialize();
	driver.get("http://www.ontestautomation.com/files/report_test.html");
		
	for (int i = 1; i <=5; i++) {
		WebElement el = driver.findElement(By.id("textfield" + Integer.toString(i)));
		Reporter.report(el.getAttribute("value"), "Text field " + Integer.toString(i));
	}
		
	Reporter.writeResults();
	driver.close();	
}

The initialize() method simply clears all previous test results by emptying the List we use to store our test results. When we run our test, the following HTML report is generated:

The HTML test report

Here, we can clearly see that our test results are now available in a nicely readable (though not yet very pretty) format. In one of my next posts, I am going to enhance these HTML reporting functions with some additional features, such as:

  • Screenshots in case of errors
  • Use of stylesheets for eye candy
  • Test execution statistics

Hopefully the above will get you started creating nicely readable HTML reports for your Selenium tests!

The Eclipse project used in the above example can be downloaded here. The HTML report template can be downloaded here (right click, save as).

Using the Page Object Design pattern in Selenium Webdriver

In a previous post, we have seen how using an object map significantly reduces the amount of maintenance needed on your Selenium scripts when your application under test is updated. Using this object map principle minimizes duplication of code on an object level. In this post, I will introduce an additional optimization pattern that minimizes code maintenance required on a higher level of abstraction.

Even though we have successfully stored object properties in a SPOM (a Single Point Of Maintenance), we still have to write code that handles these objects every time our script processes a given page including that object in our set of test scripts. If our set of test scripts requires processing a login form five times throughout the execution, we will need to include the code that handles the objects required to log in – a username field, a password field and a submit button, for example – five times as well. If the login page changes but the objects defined previously remain the same – for example, an extra checkbox is included to have a user agree to certain terms and conditions – we still need to update our scripts five times to include the processing of the checkbox.

To eliminate this code redundancy and maintenance burden, we are going to use a different approach known as the Page Object design pattern. This pattern uses page objects that represent a web page (or a form within a page, if applicable) to separate test code (validations and test flow logic, for example) from page specific code. It does so by making all actions that can be performed on a page available as methods of the page object representing that page.

So, assuming our test scripts needs to login twice (with different credentials), instead of this code:

public static void main(String args[]) {
	
	// start testing
	WebDriver driver = new HtmlUnitDriver();
		
	// first login
	driver.get("http://ourloginpage");
	driver.findElement(objMap.getLocator("loginUsername")).sendKeys("user1");
	driver.findElement(objMap.getLocator("loginPassword")).sendKeys("pass1");
	driver.findElement(objMap.getLocator("loginSubmitbutton")).click();
		
	// do stuff
		
	// second login
	driver.get("http://ourloginpage");
	driver.findElement(objMap.getLocator("loginUsername")).sendKeys("user2");
	driver.findElement(objMap.getLocator("loginPassword")).sendKeys("pass2");
	driver.findElement(objMap.getLocator("loginSubmitbutton")).click();
		
	// do more stuff
	
	// stop testing
	driver.close();
}

we would get

public static void main(String args[]) {
		
	// start testing
	WebDriver driver = new HtmlUnitDriver();
		
	// first login
	LoginPage lp = new LoginPage(driver);
	HomePage hp = lp.login("user1","pass1");
		
	// do stuff
		
	// second login
	LoginPage lp = new LoginPage(driver);
	HomePage hp = lp.login("user2","pass2");
		
	// do more stuff
		
	// stop testing
	driver.close();
}

Now, when we want to go to and handle our login page, we simply create a new instance of that page and call the login method to perform our login action. This method in turn returns a HomePage object, which is a representation of the page we get after a successful login action. A sample implementation of our LoginPage object could look as follows:

public class LoginPage {
	
	private final WebDriver driver;
	
	public LoginPage(WebDriver driver) {
		this.driver = driver;
		
		if(!driver.getTitle().equals("Login page")) {
			// we are not at the login page, go there
			driver.get("http://ourloginpage");
		}
	}
	
	public HomePage login(String username, String password) {
		driver.findElement(objMap.getLocator("loginUsername")).sendKeys("username");
		driver.findElement(objMap.getLocator("loginPassword")).sendKeys("password");
		driver.findElement(objMap.getLocator("loginSubmitbutton")).click();
		return new HomePage(driver);
	}	
}

It contains a constructor that opens the login page if it is not visible already. Alternatively, you could throw an exception and stop test execution whenever the login page is not the current page, depending on how you want your test to behave. Our LoginPage class also contains a login method that handles our login actions. If ever the login screen changes, we only need to update our test script once thanks to the proper use of page objects.

When the login action is completed successfully, our test returns a HomePage object. This class will be set up similar to the LoginPage class and provide methods specific to the page of our application under test it represents.

In case we also want to test an unsuccessful login, we simply add a method to our LoginPage class that executes the behaviour required:

public LoginPage incompleteLogin(String username) {
	driver.findElement(objMap.getLocator("loginUsername")).sendKeys("username");
	driver.findElement(objMap.getLocator("loginSubmitbutton")).click();
	return this;
}

This alternative login procedure does not enter a password. As a result, the user is not logged in and the login page remains visible, hence we return the current LoginPage object here instead of a HomePage object. If we want to test this type of incorrect login in our script, we simply call our new incorrectLogin method:

public static void main(String args[]) {
		
	// start testing
	WebDriver driver = new HtmlUnitDriver();
		
	// incorrect login
	LoginPage lp = new LoginPage(driver);
	lp = lp.incompleteLogin("user1");
	Assert.assertEquals("You forgot to type your password",lp.getError());
		
	//stop testing
	driver.quit();
}

The getError method is implemented in our LoginPage class as well:

public String getError() {
	return driver.findElement(objMap.getLocator("errorField")).getText();
}

This getError method is the result of another best practice. In order to keep your test code as much separated from your object code, always place your assertions outside of your page objects. If you need to validate specific values from a page, write methods that return them, as we did in the example above using the getError method.

To wrap things up, using the Page Object design pattern, we introduced another Single Point of Maintenance or SPOM in our Selenium test framework. This means even less maintenance required and higher ROI achieved!

An example Eclipse project using the pattern described above can be downloaded here.