A couple of presentations on service virtualization

Hey all!

First of all, apologies for the lack of recent postings on this blog. I have been quite busy with my day job and haven’t had time to write new posts in a while. I definitely intend to get working on some new stuff soon enough, so please do keep posted.

It’s been great to see that even though I haven’t added any new posts in a while, questions on existing posts are still coming in quite regularly. This is a great motivator for me to keep working on this site, and yet another reason to get my lazy behind into gear and start working on new material.

One of the reasons I’ve been quite busy is the fact that I’ve been doing a couple of presentations at conferences here in the Netherlands, and I thought it would be nice to share these with you as well.

The first presentation I held in November at the Dutch Testing Conference. The presentation shows the case study for a project I’ve been working on for the last 8 months or so. In this project, we have successfully introduced service virtualization as a means to get rid of some major blockers in our test environment. Using virtualized services that emulate the behaviour of the dependencies that were causing trouble, we have been able to speed up the development and testing process significantly. Introducing SV has also been an enabler for test automation as well – as in: we couldn’t do test automation without these virtualized services.

I gave the second presentation at the first Continuous Delivery Conference, also here in the Netherlands. As you can guess from the name, this conference was more about Continuous Delivery, rather than just about testing. However, as our case study showed some pretty significant improvements in the CD area as well, we decided to present it there as well. I did this talk at the request of Parasoft.

Both conferences were great to attend, and especially the CD conference gave me a lot of inspiration for future work and areas to explore. It really showed that testing isn’t just an activity in itself anymore (if it ever has been), but it has become an integral part of a much larger story, that of continuously delivering high quality software at ever increasing pace. Very interesting to hear and see the perspectives of some of the inspirators in the CD field on this topic..

Writing BDD tests using Selenium and Cucumber

In this article, I want to introduce another Behaviour Driven Development (BDD) tool to you. It’s similar to JBehave, about which I’ve been talking in a previous post, but with a couple of advantages:

  • It is being updated regularly, in contrast to JBehave, which has not seen any updates for quite some time now
  • It works completely separately from your Selenium WebDriver implementation, which means you can simply add it as another dependency to your existing and new Selenium projects. Again, this is unlike JBehave, which uses its own Selenium browser driver implementation, and because these aren’t updated regularly, it’s likely you’ll run into compatibility issues when trying to run Selenium + JBehave tests on newer browser versions.

Installing and configuring Cucumber
The Cucumber Java implementation (Cucumber JVM) can be downloaded from the Cucumber homepage. Once you’ve downloaded the binaries, just add the required .jar files as dependencies to your Selenium project and you’re good to go.

Defining the user story to be tested
Like with JBehave, Cucumber BDD tests are written as user stories in the Given-When-Then format. I am going to use the following login scenario as an example user story in this post:

Feature: Login action
	As a user of ParaBank
	I want to be able to log in
	So I can use the features provided
	
	Scenario: An authorized user logs in successfully
	Given the ParaBank home page is displayed
	When user "john" logs in using password "demo"
	Then the login is successful

ParaBank is a demo application from the guys at Parasoft that I use (or abuse) a lot in my demos.

In order for Cucumber to automatically detect the stories (or features, as they’re known in Cucumber), you need to make sure that they carry the .feature file extension. For example, in this case, I’ve named my user story login.feature.

Translating the test steps to code
As with JBehave, the next step is to translate the steps from the user story into runnable code. First, we are going to define the actual class to be run, which looks like this:

import org.junit.runner.RunWith;
import cucumber.api.CucumberOptions;
import cucumber.api.junit.Cucumber;

@RunWith(Cucumber.class)
@CucumberOptions(format = "pretty")
public class SeleniumCucumber {
	
}

I’ve configured my tests to run as JUnit tests, because this will make it easy to incorporate them into a CI configuration, and it auto-generates reporting in Eclipse. The @RunWith annotation defines the tests to run as Cucumber tests, whereas the @CucumberOptions annotation is used to set options for your tests (more info here). Cucumber automatically looks for step definitions and user stories in the same package, so there’s no need for any additional configuration here. Note that in Cucumber, the class to be run cannot contain any actual implementation of the test steps. These are defined in a different class:

public class SeleniumCucumberSteps {

	WebDriver driver;
	
	@Before
	public void beforeTest() {
		driver = new FirefoxDriver();
	}

	@Given("^the ParaBank home page is displayed$")
	public void theParaBankHomepageIsDisplayed() {

		driver.get("http://parabank.parasoft.com");
	}

	@When("^user john logs in using password demo$")
	public void userJohnLogsInUsingPasswordDemo() {

		driver.findElement(By.name("username")).sendKeys("john");
		driver.findElement(By.name("password")).sendKeys("demo");
		driver.findElement(By.cssSelector("input[value='Log In']")).click();
	}

	@Then("^the login is successful$")
	public void theLoginIsSuccessful() {
		Assert.assertEquals("ParaBank | Accounts Overview",driver.getTitle());
	}
	
	@After
	public void afterTest() {
		driver.quit();
	}
}

Similar to what we’ve seen with JBehave, each step in our user story is translated to runnable code. As the user story is quite straightforward, so is its implementation, as you can see.

I’ve used the JUnit @Before and @After annotations for set-up and tear-down test steps. These are steps to set up your test environment before tests are run and to restore the test environment to its original setup after test execution is finished. In our case, the only set-up step we take is the creation of a new browser instance, and the only tear-down action is closing that same browser instance. All other steps are part of the actual user story to be executed.

I’ve used the JUnit Assert feature to execute the verification from the Then part of the user story, so it shows up nicely in the reports.

That’s all there is to creating a first Cucumber test.

Running the test
Now, if we run the tests we defined above by running the SeleniumCucumber class, Cucumber automatically detects the user story to be executed (as it is in the same directory) and the associated step implementations (as they’re in the same package too). The result is that the user story is executed successfully and a nicely readable JUnit-format report is generated and displayed:
Cucumber test resultsAs you can see, the test result even displays the name of the feature and the scenario that has been executed, which makes it all the more readable and usable.

Using parameters in test steps
In Cucumber, we can use parameters to make our step definitions reusable. This is done by replacing the values in the step definitions with regular expressions and adding parameters to the methods that implement the steps:

@When("^user \"(.*)\" logs in using password \"(.*)\"$")
public void userJohnLogsInUsingPasswordDemo(String username, String password) {

	driver.findElement(By.name("username")).sendKeys(username);
	driver.findElement(By.name("password")).sendKeys(password);
	driver.findElement(By.cssSelector("input[value='Log In']")).click();
}

When running the tests, Cucumber will replace the regular expressions with the values defined in the user story and pass these as parameters to the method that is executed. Easy, right? You can do the same for integer, double and other parameter types just as easily if you know your way around regular expressions.

Overall, I’ve found Cucumber to be really easy to set up and use so far, even for someone who is not a true developer (I’m definitely not!). In a future post, I’ll go into more detail on adding these tests to a Continuous Integration solution.

An Eclipse project containing the example code and the user story / feature I’ve used in this post can be downloaded here.

Defining test steps and object properties in an Excel Data Source for Selenium

Recently, one of the readers of this blog sent me an email asking whether it was possible to define not only input and validation parameters in an external data source (such as an Excel file), but also the test steps to be taken and the object properties on which these steps need to be performed by Selenium WebDriver. As it happens, I have been playing around with this idea for a while in the past, but until now I’ve never gotten round to writing a blog post on it.

The Excel Data Source
As an example, we are going to perform a search query on Google and validate the number of results we get from this search query. This scripts consists of the following steps:

  1. Open a new browser instance
  2. Navigate to the Google homepage
  3. Type the search query in the text box
  4. Click on the search button
  5. Validate the contents of the web element displaying the number of search results against our expected value
  6. Close the browser instance

For each step, we are going to define (if applicable):

  • A keyword identifying the action to be taken
  • A parameter value that is used in the action (note that for example a type action takes a parameter, a click action does not)
  • The attribute type that uniquely identifies the object on which the action is performed
  • The corresponding attribute value

For our example test script, the Excel Data Source to be used could look something like this:
Excel Data Source
Reading the Data Source
Similar to the concept explained in a previous post, first we need to read the values from the Excel sheet in order to execute the test steps defined. We need to be a little extra careful here as some cells might be empty and the methods we use do not particularly like reading values from cells that do not exist.

public static void main (String args[]) {

		String action = "";
		String value = "";
		String attribute = "";
		String attrval = "";
		
		try {
			// Open the Excel file for reading
			FileInputStream fis = new FileInputStream("C:\\Tools\\testscript.xls");
			// Open it for writing too
			FileOutputStream fos = new FileOutputStream("C:\\Tools\\testscript.xls");
			// Access the required test data sheet
			HSSFWorkbook wb = new HSSFWorkbook(fis);
			HSSFSheet sheet = wb.getSheet("steps");
			// Loop through all rows in the sheet
			// Start at row 1 as row 0 is our header row
			for(int count = 1;count<=sheet.getLastRowNum();count++){
				HSSFRow row = sheet.getRow(count);
				System.out.println("Running test step " + row.getCell(0).toString());

				// Run the test step for the current test data row
				if(!(row.getCell(1) == null || row.getCell(1).equals(Cell.CELL_TYPE_BLANK))) {
					action = row.getCell(1).toString();
				} else {
					action = "";
				}

				if(!(row.getCell(2) == null || row.getCell(2).equals(Cell.CELL_TYPE_BLANK))) {
					value = row.getCell(2).toString();
				} else {
					value = "";
				}

				if(!(row.getCell(3) == null || row.getCell(3).equals(Cell.CELL_TYPE_BLANK))) {
					attribute = row.getCell(3).toString();
				} else {
					attribute = "";
				}

				if(!(row.getCell(4) == null || row.getCell(4).equals(Cell.CELL_TYPE_BLANK))) {
					attrval = row.getCell(4).toString();
				} else {
					attrval = "";
				}

				System.out.println("Test action: " + action);
				System.out.println("Parameter value: " + value);
				System.out.println("Attribute: " + attribute);
				System.out.println("Attribute value: " + attrval);
				
				String result = runTestStep(action,value,attribute,attrval);
				
				// Write the result back to the Excel sheet
				row.createCell(5).setCellValue(result);
				
			}
			
			// Save the Excel sheet and close the file streams
			wb.write(fos);
			fis.close();
			fos.close();
			
		} catch (Exception e) {
			System.out.println(e.toString());
		}
}

We do not read the value from the Result column, but rather we are going to execute the test steps using the values from the other columns, determine the result and write this back to the Excel sheet. In this way, we have a rudimentary logging function built in directly into our framework. Neat, right?

Defining and executing test steps
Next, we need to implement the generic runTestStep method we use to execute the test steps we have defined in our Data Source. This can be done pretty straightforward by looking at the current Action keyword and then executing the necessary steps for that keyword.

public static String runTestStep(String action, String value, String attribute, String attrval) throws Exception {

		switch(action.toLowerCase()) {
		case "openbrowser":
			switch(value.toLowerCase()) {
			case "firefox":
				driver = new FirefoxDriver();
				driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
				return "OK";
			case "htmlunit":
				driver = new HtmlUnitDriver();
				driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
				return "OK";
			default:
				return "NOK";
			}
		case "navigate":
			driver.get(value);
			return "OK";
		case "type":
			try {
				WebElement element = findMyElement(attribute,attrval);
				element.sendKeys(value);
				return "OK";
			} catch (Exception e) {
				System.out.println(e.toString());
				return "NOK";
			}
		case "click":
			try {
				WebElement element = findMyElement(attribute,attrval);
				element.click();
				return "OK";
			} catch (Exception e) {
				System.out.println(e.toString());
				return "NOK";
			}
		case "validate":
			try {
				WebElement element = findMyElement(attribute,attrval);
				if (element.getText().equals(value)) {
					return "OK";
				} else {
					System.out.println("Actual value: " + element.getText() + ", expected value: " + value);
					return "NOK";
				}
			} catch (Exception e) {
				System.out.println(e.toString());
			}
		case "closebrowser":
			driver.quit();
			return "OK";
		default:
			throw new Exception("Unknown keyword " + action);
		}
}

This setup makes it very easy to add other keywords (e.g., for selecting a value from a dropdown list), both for standardized browser actions as well as for custom keywords. Every step returns a result (here either OK or NOK) to indicate whether the action has been executed successfully. This is written back to the Excel sheet in our main method, so we can see whether our test steps have been executed correctly.

Finding the required object
I’ve also used a helper method findMyElement that, given an attribute and an attribute value, returns a WebElement that corresponds to these values. Its implementation is quite rudimentary, but it should be easy for you to extend it and make it more fail safe. I’m lazy sometimes. I haven’t even implemented all types of selectors that Selenium can handle!

public static WebElement findMyElement(String attribute, String attrval) throws Exception {

		switch(attribute.toLowerCase()) {
		case "id":
			return driver.findElement(By.id(attrval));
		case "name":
			return driver.findElement(By.name(attrval));
		case "xpath":
			return driver.findElement(By.xpath(attrval));
		case "css-select":
			return driver.findElement(By.cssSelector(attrval));
		default:
			throw new Exception("Unknown selector type " + attribute);
		}
}

Now, when I run the test, my code nicely reads all rows from Excel, determines what test action needs to be performed, executes it and reports the result back to Excel. After running the test, the Excel sheet looks like this:
Excel Data Source including test results
The only test step that fails does so because it Google doesn’t always return the same number of test results in the same amount of time, obviously. Nevertheless, it illustrates the added value of writing back the result of every step to the Excel sheet. Of course, you can very easily extend this to include a useful error message and even a link to a screenshot.

The Eclipse project containing the code I’ve used to create this example can be downloaded here.