Up and running with: XMLUnit

This is the sixth article in our series on new, popular or otherwise interesting tools used in test automation. You can read all posts within this series by clicking here.

What is XMLUnit?
From the XMLUnit.org website: XMLUnit provides you with the tools to verify the XML you emit is the one you want to create. It provides helpers to validate against an XML Schema, assert the values of XPath queries or compare XML documents against expected outcomes. The most important part is a diff-engine that provides you with full control over what kind of difference is important to you and which part of the generated document to compare with which part of your reference document.

Where can I get XMLUnit?
XMLUnit can be downloaded from the XMLUnit GitHub site. Snapshot binaries for XMLUnit 2.0 (the version that is covered in this post) can also be obtained from the Maven repository.

How do I install and configure XMLUnit?
Installing XMLUnit is as simple as downloading the latest xmlunit-core snapshot version and adding it as a dependency to your Java project.

Creating a first XMLUnittest
Let’s start with a very simple test that validates the value of a specific element of a predefined XML message:

public void aFirstTest() {
	Source source = Input.fromString("<foo>bar</foo>").build();
	XPathEngine xpath = new JAXPXPathEngine();
	String content = xpath.evaluate("/foo/text()", source);
	Assert.assertEquals(content, "bar");

Note that I have created this XMLUnit test as a TestNG test to make running the tests easier and to have TestNG automatically generate a test report.

Useful features
Using XMLUnit, XML validations can not only be done based on predefined strings containing XML, but also directly on the XML result of a call to a REST service. This is done in a very similar way:

public void restCallTest() {
	Source source = Input.fromURI("http://parabank.parasoft.com/parabank/services/bank/customers/12212").build();
	XPathEngine xpath = new JAXPXPathEngine();
	String content = xpath.evaluate("//city/text()", source);
	Assert.assertEquals(content, "Beverly Hills");

The only thing that is changed is the use of Input.fromURI instead of Input.fromString. Similar methods are available to perform XPath-based validations directly from an InputStream, a File, or a Node object, amongst others.

XMLUnit can not only perform element-based validation, it can also do a full compare of two XML messages. In this example, the XML returned by the same REST service call used in the previous example is compared against a predefined XML messages stored on the local file system (as response.xml in the messages subdirectory):

public void compareTest() {
	DiffBuilder db = DiffBuilder.compare(Input.fromFile("messages/response.xml")).withTest(Input.fromURI("http://parabank.parasoft.com/parabank/services/bank/customers/12212").build());
	Diff d = db.build();
	if(d.hasDifferences()) {
		Iterator<Difference> diffs = d.getDifferences().iterator();
		while(diffs.hasNext()) {
	Assert.assertEquals(d.hasDifferences(), false);

Any differences between the two XML files are written to the TestNG report file using the default Reporter:

XMLUnit differences written to a TestNG report

Finally, XMLUnit can also validate XML messages against an XSD document:

public void validationTest() {
	Validator v = Validator.forLanguage(Languages.W3C_XML_SCHEMA_NS_URI);
	ValidationResult result = v.validateInstance(Input.fromFile("messages/order.xml").build());
	Iterator<ValidationProblem> problems = result.getProblems().iterator();
	while (problems.hasNext()) {
	Assert.assertEquals(result.isValid(), true);

Again, any issues that occur during schema validation can be written to the TestNG report using the default Reporter:

XMLUnit schema issues in TestNG report

Further reading
An Eclipse project including the tests I’ve demonstrated above and the reports that have been generated can be downloaded here.

Book: Alan Page – The A word

When not working on testing and test automation myself, I like to read books, articles and blog posts that are related to my profession. One book I have recently finished reading is The A Word by Alan Page. Although it’s more of a collection of revised blog posts from his blog The Angry Weasel than a book, it’s been a very interesting read nonetheless.

For those of you that aren’t familiar with his writing, Alan writes a lot about his experiences with testing and test automation. He is pretty well known for his skeptical view on how a lot of people see automated testing as the solution for everything, especially when it comes to GUI-based test automation. Although I do tend to write regularly about test automation using Selenium, I am not a big advocate of using UI-based test automation tools for all things test automation related myself (see for example this post). Therefore a lot of things covered in The A Word resonated with me and I found it a very pleasant read, although it is rather short at around 70-80 pages.

Book cover for The A Word - Alan Page

I wholeheartedly recommend this book to anybody that has anything to do with test automation, as Alan offers valuable food for thought for test engineers, test managers and all others doing or relying on automated tests. It might just make you think about whether you’re doing the right stuff and doing your stuff right..

The A Word is available on LeanPub. All profits go to the American Cancer Society, so that alone should be a reason to pick it up and leave a donation.

FindBy strategies for Selenium explained

The @FindBy annotation is used in Page Objects in Selenium tests to specify the object location strategy for a WebElement or a list of WebElements. Using the PageFactory, these WebElements are usually initialized when a Page Object is created. In this post, I will demonstrate various ways in which you can use @FindBy annotations to efficiently locate (groups of) WebElements.

The @FindBy annotation is used to locate one or more WebElements using a single criterion. For example, to identify all elements that have the same class attribute, we could use the following identification:

@FindBy(how = How.CLASS_NAME, using = "classname")
private List<WebElement> singlecriterion;

If we are sure there is only a single element that is identified by our location strategy, for example when we use the element ID, we can also directly assign the result to a WebElement variable:

@FindBy(how = How.ID, using = "elementid")
private WebElement element;

To instantiate the elements, we call the initElements method of the PageFactory class:

PageFactory.initElements(driver, this);

@FindBys and @FindAll
In some cases we want (or need) to use more than a single criterion to identify one or more objects, for instance when page elements do not have a unique ID. In this case, there are two possible annotations that can be used:

  • The @FindBys annotation is used in case elements need to match all of the given criteria
  • The @FindAll annotation is used in case elements need to match at least one of the given criteria

Let’s take a look at an example that illustrates the difference between the two.

The Parabank homepage contains two textboxes, one for the username and one for the password. Both elements have a name attribute that we are going to use to identify them within a Page Object.

Using @FindBys:

	@FindBy(how = How.NAME, using = "username"),
	@FindBy(how = How.NAME, using = "password")
private List<WebElement> bothcriteria;

The bothcriteria list should contain 0 elements, as there is no element that has both a name attribute with the value username and a name attribute with the value password.

Using @FindAll:

	@FindBy(how = How.NAME, using = "username"),
	@FindBy(how = How.NAME, using = "password")
private List<WebElement> eithercriterion;

The eithercriterion list should contain 2 elements, as there is one element that has a name attribute with the value username and also one that has a name attribute with the value password.

For validation purposes, if we print the number of results found by all of the above strategies using

System.out.println("Using @FindBy, we found " + singlecriterion.size() + " element(s)");
System.out.println("Using @FindBys, we found " + bothcriteria.size() + " element(s)");
System.out.println("Using @FindAll, we found " + eithercriterion.size() + " element(s)");

we see this:
Results for different FindBy strategiesIt clearly works exactly as expected!

A more verbose FindBy
Finally, if you have a lot of elements within your Page Object, you can also use a more verbose way of specifying your @FindBy strategy. For example

@FindBy(className = "classname")

gives the exact same results as

@FindBy(how = How.CLASS_NAME, using = "classname")