Up and running with: Selendroid

This is the second article in our series on new, popular or otherwise interesting tools used in test automation. You can read all posts within this series by clicking here.

What is Selendroid?
Selendroid is a test automation framework for testing Android native and hybrid applications. Selendroid tests are written using the Selenium Webdriver client API, which allows for full integration with existing Selenium frameworks.

Where can I get Selendroid?
Selendroid can be downloaded from the Selendroid website.

How do I install and configure Selendroid?
Before you can start setting up Selendroid and writing tests, you need to download and install the latest Android SDK first. Clear instructions on how to do this can be found here. Make sure you also create at least one Android virtual device (AVD) and test whether it can be run properly, as we are going to need this virtual device to deploy our app under test and run our Selendroid tests on it.

NOTE: The instructions specify the creation of an AVD with the latest Android version (KitKat, API level 19). This is all wonderful, but if you’re (like me) working on a virtual machine or any other machine with limited resources, you’re probably better off creating an AVD based on Android Gingerbread (API level 10). You will need to install the correct SDK components for that API level using the Android SDK Manager. Oh, and it’s probably wise as well to use GPU emulation by enabling ‘Use host GPU’ in the AVD settings while you’re at it. This will make your test run much faster. Or, in my case, it will make your test run at all.

After you have successfully installed the Android SDK, you can install Selendroid following these instructions. When you’ve done so, start Selendroid and load the test app (the .apk file) they provide here. When you point your browser to

http://localhost:4444/wd/hub/status

you’ll see something similar to this:

Selendroid is running
As you can see, Selendroid is running and it has detected our newly created AVD.

Creating a first test script
Now we’re done setting up our environment we can go to the interesting part. To start mobile testing using Selendroid, create a new Java project, add the Selendroid and Selenium .jar files as dependencies and create the following script:

public static void runSelendroidTest() throws Exception {
		
	// specify test capabilities (your 'test environment')
	SelendroidCapabilities capa = new SelendroidCapabilities("io.selendroid.testapp:0.9.0");
		
	// explicitly state that we want to run our test on an Android API level 10 device
	capa.setPlatformVersion(DeviceTargetPlatform.ANDROID10);
		
	// explicitly state that we use an emulator (an AVD) for test execution rather than a physical device
	capa.setEmulator(true);

	// start a new WebDriver
	WebDriver driver = new SelendroidDriver(capa);
		
	// execute a very simple test
	WebElement inputField = driver.findElement(By.id("my_text_field"));
	Assert.assertEquals("true", inputField.getAttribute("enabled"));
	inputField.sendKeys("Selendroid");
	Assert.assertEquals("Selendroid", inputField.getText());
		
	// quit testing
	driver.quit();
		
}

The comments in the code really explain it all. You can instantly see that Selendroid tests are indeed very similar to regular Selenium tests, with the exception that they are run on an app that is deployed on an AVD rather than on a website that you access using a browser.

Running your test
Running your Selendroid tests is done just like you’d run Selenium tests, so there’s really no need to go into more detail on that here.

When you run your tests, you’ll see that an Android emulator (the AVD) is started, the app is loaded, tests are executed and the emulator is closed again. Selendroid can also be used to execute tests on apps that are deployed on physical Android devices connected to your machine, something I haven’t tried myself.

A screenshot of the test app

Useful features
Selendroid offers a lot of to the tester that wants to create more useful and more complex tests. For example, you can very accurately emulate a user swipe action from right to left using the following code snippet:

WebElement pages = driver.findElement(By.id("some_id"));
TouchActions flick = new TouchActions(driver).flick(pages, -100, 0, 0);
flick.perform();

Using the instructions presented here, just try and see for yourself what you can do with Selendroid. And of course, please share your experiences here.

Further reading
Again, the official Selendroid site can be found here. It offers a lot of information on the possibilities and features of Selendroid.

Happy mobile testing!

Create your own HTML report from Selenium tests

As I am learning more and more about using Selenium Webdriver efficiently (experiences I try to share through this blog), I’m slowly turning away from my original standpoint that user interface-based test automation is not for me. I am really starting to appreciate the power of Selenium, especially when you use proper test automation framework design patterns such as the Page Object pattern I wrote about earlier. However, Selenium by default lacks one vital aspect of what makes a good test automation tool to me: proper reporting options. Luckily, as Selenium is so open, there’s lots of ways to build custom reporting yourself. This post shows one possible approach.

My approach
Personally, I prefer HTML reports as they are highly customizable, relatively easy to build and can be easily distributed to other project team members. To build a nice HTML report, I use the following two step approach:

  • Execute tests and gather statistics about validations executed
  • Create HTML report from these statistics after test execution has finished

In this post I’ll use the following test script as an example. I created a page with five HTML text fields, for which I am going to validate the default text. Nothing really realistic, but remember it’s only used to illustrate my reporting concept.

public static void main (String args[]) {
		
	WebDriver driver = new HtmlUnitDriver();
	driver.get("http://www.ontestautomation.com/files/report_test.html");
		
	for (int i = 1; i <=5; i++) {
		WebElement el = driver.findElement(By.id("textfield" + Integer.toString(i)));
		Assert.assertEquals(el.getAttribute("value"), "Text field " + Integer.toString(i));
	}
		
	driver.close();	
}

When we run this script, one error is generated as text field 4 contains a different default value (go to the URL in the script to see for yourself).

Custom reporting functions
To be able to create a nice HTML report, we first need some custom reporting functions that store test results in a way we can reuse them later to generate our report. To achieve this, I created a report method in a Reporter class that stores validation results in a simple List:

public static void report(String actualValue,String expectedValue) {
	if(actualValue.equals(expectedValue)) {
		Result r = new Result("Pass","Actual value '" + actualValue + "' matches expected value");
		details.add(r);
	} else {
		Result r = new Result("Fail","Actual value '" + actualValue + "' does not match expected value '" + expectedValue + "'");
		details.add(r);
	}
}

The Result object is a simple class with three class variables: result (which is either Pass or Fail), a resultText (which is a custom description) and a URL for a screenshot (the use of which we will see later).

For every test we execute in our Selenium script, instead of using the standard TestNG / JUnit assertions, we use our own reporting function. You might want to throw an error as well when a validation fails, but I’m happy just to write it to my report and let test execution continue.

After test execution is finished, we are going to write the test results we gathered to a file. For this, I used an extremely simple HTML template (yes, I was too lazy even to indent it properly):

<html>
<head>
<title>Test Report</title>
<head>
<body>
<h3>Test results</h3>
<table>
<tr>
<th width="10%">Step</th>
<th width="10%">Result</th>
<th width="80%">Remarks</th>
</tr>
<!-- INSERT_RESULTS -->
</table>
</body>

In this template I am going to insert my test results, using a simple replace function

public static void writeResults() {
	try {
		String reportIn = new String(Files.readAllBytes(Paths.get(templatePath)));
		for (int i = 0; i < details.size();i++) {
			reportIn = reportIn.replaceFirst(resultPlaceholder,"<tr><td>" + Integer.toString(i+1) + "</td><td>" + details.get(i).getResult() + "</td><td>" + details.get(i).getResultText() + "</td></tr>" + resultPlaceholder);
		}
			
		String currentDate = new SimpleDateFormat("dd-MM-yyyy").format(new Date());
		String reportPath = "Z:\\Documents\\Bas\\blog\\files\\report_" + currentDate + ".html";
		Files.write(Paths.get(reportPath),reportIn.getBytes(),StandardOpenOption.CREATE);
			
	} catch (Exception e) {
		System.out.println("Error when writing report file:\n" + e.toString());
	}
}

Finally, we need to use these custom reporting functions in our test script:

public static void main (String args[]) {
		
	WebDriver driver = new HtmlUnitDriver();
	Reporter.initialize();
	driver.get("http://www.ontestautomation.com/files/report_test.html");
		
	for (int i = 1; i <=5; i++) {
		WebElement el = driver.findElement(By.id("textfield" + Integer.toString(i)));
		Reporter.report(el.getAttribute("value"), "Text field " + Integer.toString(i));
	}
		
	Reporter.writeResults();
	driver.close();	
}

The initialize() method simply clears all previous test results by emptying the List we use to store our test results. When we run our test, the following HTML report is generated:

The HTML test report

Here, we can clearly see that our test results are now available in a nicely readable (though not yet very pretty) format. In one of my next posts, I am going to enhance these HTML reporting functions with some additional features, such as:

  • Screenshots in case of errors
  • Use of stylesheets for eye candy
  • Test execution statistics

Hopefully the above will get you started creating nicely readable HTML reports for your Selenium tests!

The Eclipse project used in the above example can be downloaded here. The HTML report template can be downloaded here (right click, save as).

How to compare WSDL versions in your automated test framework

A colleague of mine posed an interesting question last week. He had a test setup with three different machines on which his application under test was installed and deployed. He wanted to make sure in his test that the web service interface offered by these deployments was exactly the same by comparing the WSDLs associated with each installation. However, the tool he used (Parasoft SOAtest) only supports regression testing of single WSDL instances, i.e., it can validate whether a certain WSDL has been changed over time, but it cannot compare two or more different WSDL instances.

Luckily, SOAtest supports extension of its functionality using scripting, and I found a nice Java API that would do exactly what he asked. In this post, I’ll show you how this is done in Java. In SOAtest, I did it with a Jython script that imported and used the appropriate Java classes, but apart from syntax the solution is the same.

The Java API I used can be found here. The piece of code that executes the comparison is very straightforward:

private static void compareWSDL(){
		
	// configure the log4j logger
	BasicConfigurator.configure();
		
	// create a new wsdl parser
	WSDLParser parser = new WSDLParser();

	// parse both wsdl documents
	Definitions wsdl1 = parser.parse("Z:\\Documents\\Bas\\wsdlcompare\\ParaBank.wsdl");
	Definitions wsdl2 = parser.parse("Z:\\Documents\\Bas\\wsdlcompare\\ParaBank2.wsdl");

	// compare the wsdl documents
	WsdlDiffGenerator diffGen = new WsdlDiffGenerator(wsdl1, wsdl2);
	List<Difference> lst = diffGen.compare();
		
	// write the differences to the console
	for (Difference diff : lst) {
		System.out.println(diff.dump());
	}
}

For this example, I used two locally stored copies of a WSDL document where I changed the definition of a single element (I removed the minOccurs=”0″ attribute). The API uses Log4J as the logging engine, so we need to initialize that in our code and add a log4j.properties file to our project:
Log4J properties file
When we run our code, we can see that the WSDL documents are compared successfully, and that the difference I injected by hand is detected nicely by the WSDL compare tool:
Console output for the WSDL comparison
A nice and clean answer to yet another automated testing question, just as it should be.

An example Eclipse project using the pattern described above can be downloaded here.