How to compare WSDL versions in your automated test framework

A colleague of mine posed an interesting question last week. He had a test setup with three different machines on which his application under test was installed and deployed. He wanted to make sure in his test that the web service interface offered by these deployments was exactly the same by comparing the WSDLs associated with each installation. However, the tool he used (Parasoft SOAtest) only supports regression testing of single WSDL instances, i.e., it can validate whether a certain WSDL has been changed over time, but it cannot compare two or more different WSDL instances.

Luckily, SOAtest supports extension of its functionality using scripting, and I found a nice Java API that would do exactly what he asked. In this post, I’ll show you how this is done in Java. In SOAtest, I did it with a Jython script that imported and used the appropriate Java classes, but apart from syntax the solution is the same.

The Java API I used can be found here. The piece of code that executes the comparison is very straightforward:

private static void compareWSDL(){
		
	// configure the log4j logger
	BasicConfigurator.configure();
		
	// create a new wsdl parser
	WSDLParser parser = new WSDLParser();

	// parse both wsdl documents
	Definitions wsdl1 = parser.parse("Z:\\Documents\\Bas\\wsdlcompare\\ParaBank.wsdl");
	Definitions wsdl2 = parser.parse("Z:\\Documents\\Bas\\wsdlcompare\\ParaBank2.wsdl");

	// compare the wsdl documents
	WsdlDiffGenerator diffGen = new WsdlDiffGenerator(wsdl1, wsdl2);
	List<Difference> lst = diffGen.compare();
		
	// write the differences to the console
	for (Difference diff : lst) {
		System.out.println(diff.dump());
	}
}

For this example, I used two locally stored copies of a WSDL document where I changed the definition of a single element (I removed the minOccurs=”0″ attribute). The API uses Log4J as the logging engine, so we need to initialize that in our code and add a log4j.properties file to our project:
Log4J properties file
When we run our code, we can see that the WSDL documents are compared successfully, and that the difference I injected by hand is detected nicely by the WSDL compare tool:
Console output for the WSDL comparison
A nice and clean answer to yet another automated testing question, just as it should be.

An example Eclipse project using the pattern described above can be downloaded here.

An introduction to service virtualization

One of the concepts that is rapidly gaining popularity within the testing world – and in IT in general – is service virtualization (SV). This post provides a quick introduction to the SV concept and the way it leverages automated and manual testing efforts and thereby software development in general.

What is service virtualization?
SV is the concept of simulating the behaviour of an application or resource that is required in order to perform certain types of tests, in case this resource is not readily available or the availability or use of this resource is too expensive. In other words, SV is aimed at the removal of traditional dependencies in software development when it comes to the availability of systems and environments. In this way, SV is complementary to other forms of virtualization, such as hardware (VPS) or operating system (VMware and similar solutions) virtualization.

Behaviour simulation is carried out using virtual assets, which are pieces of software that mimic application behaviour in some way. Although SV started out with the simulation of web service behaviour, modern SV tools can simulate all kinds of communication that is performed over common message protocols. In this way, SV can be used to simulate database transactions, mainframe interaction, etc.

http://www.w3.org/WAI/intro/people-use-web/Overview.html

From a ‘live’ test environment to a virtual test environment

What are the benefits of using service virtualization?
As mentioned in the first paragraph of this post, SV can significantly speed up the development process in case required resources:

  • are not available during (part of) the test cycle, thereby delaying tests or negatively influencing test coverage;
  • are too expensive to keep alive (e.g., test environments need to be maintained or rented continuously even though access is required only a couple of times per year);
  • cannot readily emulate the behaviour required for certain types of test cases;
  • are shared throughout different development teams, negatively influencing resource availability.

Service virtualization tools
Currently, four commercial service virtualization tools are available on the market:

Furthermore, several open source service virtualization projects have emerged in recent years, such as WireMock and Betamax. These offer significantly less features, obviously, but they just might fit your project requirements nonetheless, making them worthy of an evaluation.

Personally, I have extensive experience using Parasoft Virtualize and have been able to successfully implement it in a number of projects for our customers. Results have been excellent, as can be seen in the following case study.

A case study
The central order management application at one of our customers relied heavily on an external resource for the successful provisioning of orders. This external resource requires manual configuration for each order that is created, resulting in the test team having to file requests for configuration changes for each test case. The delay for this configuration could be as much as a week, resulting in severe delays in testing and possible test coverage (as testers could only create a small amount of orders per test cycle).

Using service virtualization to simulate the behaviour of this external resource, this dependency has been removed altogether. The virtual asset implementing the external resource behaviour processes new orders in a matter of minutes, as opposed to weeks in case the ‘real’ external resource is required. Using SV, testers can provision orders much faster and are able to achieve much higher test coverage as a result. This has resulted in a significant increase in software quality. Also, SV has been a key factor in the switch to an Agile development process. Without SV, the short development and testing cycles associated with the Agile software development method would not have been possible.

Service virtualization on Wikipedia

Using the Page Object Design pattern in Selenium Webdriver

In a previous post, we have seen how using an object map significantly reduces the amount of maintenance needed on your Selenium scripts when your application under test is updated. Using this object map principle minimizes duplication of code on an object level. In this post, I will introduce an additional optimization pattern that minimizes code maintenance required on a higher level of abstraction.

Even though we have successfully stored object properties in a SPOM (a Single Point Of Maintenance), we still have to write code that handles these objects every time our script processes a given page including that object in our set of test scripts. If our set of test scripts requires processing a login form five times throughout the execution, we will need to include the code that handles the objects required to log in – a username field, a password field and a submit button, for example – five times as well. If the login page changes but the objects defined previously remain the same – for example, an extra checkbox is included to have a user agree to certain terms and conditions – we still need to update our scripts five times to include the processing of the checkbox.

To eliminate this code redundancy and maintenance burden, we are going to use a different approach known as the Page Object design pattern. This pattern uses page objects that represent a web page (or a form within a page, if applicable) to separate test code (validations and test flow logic, for example) from page specific code. It does so by making all actions that can be performed on a page available as methods of the page object representing that page.

So, assuming our test scripts needs to login twice (with different credentials), instead of this code:

public static void main(String args[]) {
	
	// start testing
	WebDriver driver = new HtmlUnitDriver();
		
	// first login
	driver.get("http://ourloginpage");
	driver.findElement(objMap.getLocator("loginUsername")).sendKeys("user1");
	driver.findElement(objMap.getLocator("loginPassword")).sendKeys("pass1");
	driver.findElement(objMap.getLocator("loginSubmitbutton")).click();
		
	// do stuff
		
	// second login
	driver.get("http://ourloginpage");
	driver.findElement(objMap.getLocator("loginUsername")).sendKeys("user2");
	driver.findElement(objMap.getLocator("loginPassword")).sendKeys("pass2");
	driver.findElement(objMap.getLocator("loginSubmitbutton")).click();
		
	// do more stuff
	
	// stop testing
	driver.close();
}

we would get

public static void main(String args[]) {
		
	// start testing
	WebDriver driver = new HtmlUnitDriver();
		
	// first login
	LoginPage lp = new LoginPage(driver);
	HomePage hp = lp.login("user1","pass1");
		
	// do stuff
		
	// second login
	LoginPage lp = new LoginPage(driver);
	HomePage hp = lp.login("user2","pass2");
		
	// do more stuff
		
	// stop testing
	driver.close();
}

Now, when we want to go to and handle our login page, we simply create a new instance of that page and call the login method to perform our login action. This method in turn returns a HomePage object, which is a representation of the page we get after a successful login action. A sample implementation of our LoginPage object could look as follows:

public class LoginPage {
	
	private final WebDriver driver;
	
	public LoginPage(WebDriver driver) {
		this.driver = driver;
		
		if(!driver.getTitle().equals("Login page")) {
			// we are not at the login page, go there
			driver.get("http://ourloginpage");
		}
	}
	
	public HomePage login(String username, String password) {
		driver.findElement(objMap.getLocator("loginUsername")).sendKeys("username");
		driver.findElement(objMap.getLocator("loginPassword")).sendKeys("password");
		driver.findElement(objMap.getLocator("loginSubmitbutton")).click();
		return new HomePage(driver);
	}	
}

It contains a constructor that opens the login page if it is not visible already. Alternatively, you could throw an exception and stop test execution whenever the login page is not the current page, depending on how you want your test to behave. Our LoginPage class also contains a login method that handles our login actions. If ever the login screen changes, we only need to update our test script once thanks to the proper use of page objects.

When the login action is completed successfully, our test returns a HomePage object. This class will be set up similar to the LoginPage class and provide methods specific to the page of our application under test it represents.

In case we also want to test an unsuccessful login, we simply add a method to our LoginPage class that executes the behaviour required:

public LoginPage incompleteLogin(String username) {
	driver.findElement(objMap.getLocator("loginUsername")).sendKeys("username");
	driver.findElement(objMap.getLocator("loginSubmitbutton")).click();
	return this;
}

This alternative login procedure does not enter a password. As a result, the user is not logged in and the login page remains visible, hence we return the current LoginPage object here instead of a HomePage object. If we want to test this type of incorrect login in our script, we simply call our new incorrectLogin method:

public static void main(String args[]) {
		
	// start testing
	WebDriver driver = new HtmlUnitDriver();
		
	// incorrect login
	LoginPage lp = new LoginPage(driver);
	lp = lp.incompleteLogin("user1");
	Assert.assertEquals("You forgot to type your password",lp.getError());
		
	//stop testing
	driver.quit();
}

The getError method is implemented in our LoginPage class as well:

public String getError() {
	return driver.findElement(objMap.getLocator("errorField")).getText();
}

This getError method is the result of another best practice. In order to keep your test code as much separated from your object code, always place your assertions outside of your page objects. If you need to validate specific values from a page, write methods that return them, as we did in the example above using the getError method.

To wrap things up, using the Page Object design pattern, we introduced another Single Point of Maintenance or SPOM in our Selenium test framework. This means even less maintenance required and higher ROI achieved!

An example Eclipse project using the pattern described above can be downloaded here.