Writing BDD tests using Selenium and Cucumber

In this article, I want to introduce another Behaviour Driven Development (BDD) tool to you. It’s similar to JBehave, about which I’ve been talking in a previous post, but with a couple of advantages:

  • It is being updated regularly, in contrast to JBehave, which has not seen any updates for quite some time now
  • It works completely separately from your Selenium WebDriver implementation, which means you can simply add it as another dependency to your existing and new Selenium projects. Again, this is unlike JBehave, which uses its own Selenium browser driver implementation, and because these aren’t updated regularly, it’s likely you’ll run into compatibility issues when trying to run Selenium + JBehave tests on newer browser versions.

Installing and configuring Cucumber
The Cucumber Java implementation (Cucumber JVM) can be downloaded from the Cucumber homepage. Once you’ve downloaded the binaries, just add the required .jar files as dependencies to your Selenium project and you’re good to go.

Defining the user story to be tested
Like with JBehave, Cucumber BDD tests are written as user stories in the Given-When-Then format. I am going to use the following login scenario as an example user story in this post:

Feature: Login action
	As a user of ParaBank
	I want to be able to log in
	So I can use the features provided
	
	Scenario: An authorized user logs in successfully
	Given the ParaBank home page is displayed
	When user "john" logs in using password "demo"
	Then the login is successful

ParaBank is a demo application from the guys at Parasoft that I use (or abuse) a lot in my demos.

In order for Cucumber to automatically detect the stories (or features, as they’re known in Cucumber), you need to make sure that they carry the .feature file extension. For example, in this case, I’ve named my user story login.feature.

Translating the test steps to code
As with JBehave, the next step is to translate the steps from the user story into runnable code. First, we are going to define the actual class to be run, which looks like this:

import org.junit.runner.RunWith;
import cucumber.api.CucumberOptions;
import cucumber.api.junit.Cucumber;

@RunWith(Cucumber.class)
@CucumberOptions(format = "pretty")
public class SeleniumCucumber {
	
}

I’ve configured my tests to run as JUnit tests, because this will make it easy to incorporate them into a CI configuration, and it auto-generates reporting in Eclipse. The @RunWith annotation defines the tests to run as Cucumber tests, whereas the @CucumberOptions annotation is used to set options for your tests (more info here). Cucumber automatically looks for step definitions and user stories in the same package, so there’s no need for any additional configuration here. Note that in Cucumber, the class to be run cannot contain any actual implementation of the test steps. These are defined in a different class:

public class SeleniumCucumberSteps {

	WebDriver driver;
	
	@Before
	public void beforeTest() {
		driver = new FirefoxDriver();
	}

	@Given("^the ParaBank home page is displayed$")
	public void theParaBankHomepageIsDisplayed() {

		driver.get("http://parabank.parasoft.com");
	}

	@When("^user john logs in using password demo$")
	public void userJohnLogsInUsingPasswordDemo() {

		driver.findElement(By.name("username")).sendKeys("john");
		driver.findElement(By.name("password")).sendKeys("demo");
		driver.findElement(By.cssSelector("input[value='Log In']")).click();
	}

	@Then("^the login is successful$")
	public void theLoginIsSuccessful() {
		Assert.assertEquals("ParaBank | Accounts Overview",driver.getTitle());
	}
	
	@After
	public void afterTest() {
		driver.quit();
	}
}

Similar to what we’ve seen with JBehave, each step in our user story is translated to runnable code. As the user story is quite straightforward, so is its implementation, as you can see.

I’ve used the JUnit @Before and @After annotations for set-up and tear-down test steps. These are steps to set up your test environment before tests are run and to restore the test environment to its original setup after test execution is finished. In our case, the only set-up step we take is the creation of a new browser instance, and the only tear-down action is closing that same browser instance. All other steps are part of the actual user story to be executed.

I’ve used the JUnit Assert feature to execute the verification from the Then part of the user story, so it shows up nicely in the reports.

That’s all there is to creating a first Cucumber test.

Running the test
Now, if we run the tests we defined above by running the SeleniumCucumber class, Cucumber automatically detects the user story to be executed (as it is in the same directory) and the associated step implementations (as they’re in the same package too). The result is that the user story is executed successfully and a nicely readable JUnit-format report is generated and displayed:
Cucumber test resultsAs you can see, the test result even displays the name of the feature and the scenario that has been executed, which makes it all the more readable and usable.

Using parameters in test steps
In Cucumber, we can use parameters to make our step definitions reusable. This is done by replacing the values in the step definitions with regular expressions and adding parameters to the methods that implement the steps:

@When("^user \"(.*)\" logs in using password \"(.*)\"$")
public void userJohnLogsInUsingPasswordDemo(String username, String password) {

	driver.findElement(By.name("username")).sendKeys(username);
	driver.findElement(By.name("password")).sendKeys(password);
	driver.findElement(By.cssSelector("input[value='Log In']")).click();
}

When running the tests, Cucumber will replace the regular expressions with the values defined in the user story and pass these as parameters to the method that is executed. Easy, right? You can do the same for integer, double and other parameter types just as easily if you know your way around regular expressions.

Overall, I’ve found Cucumber to be really easy to set up and use so far, even for someone who is not a true developer (I’m definitely not!). In a future post, I’ll go into more detail on adding these tests to a Continuous Integration solution.

An Eclipse project containing the example code and the user story / feature I’ve used in this post can be downloaded here.

Up and running with: JUnit

This is the first article in a new series on tools used in test automation. Each of the articles in it will introduce a specific test tool and will show you how to get up and running with it. The focus will be on free and / or open source test tools as this allows everyone with an interest in the tool presented to get started using it right away.

What is JUnit?
JUnit is a unit testing framework for Java. Is it part of a family of unit testing frameworks for a variety of programming languages, known collectively as xUnit. With JUnit, you can quickly develop and run unit tests for Java classes to verify their correctness.

Where can I get JUnit?
JUnit can be downloaded from here. However, when you use an IDE such as Eclipse or IntelliJ, JUnit comes with the installation, making getting started with JUnit test development even easier.

How do I install and configure JUnit?
As I prefer using Eclipse, the instructions below show you how to start developing and running JUnit tests in Eclipse. Those using IntelliJ are referred to the IntelliJ homepage.

To start using JUnit to run tests on your code, all you need to do is to add the JUnit library to your existing Java project. To do this, right-click on your project, select Properties, go to the Libraries tab and press the ‘Add Library’ button. Select JUnit from the list and click ‘Next’. Then, select ‘JUnit 4’ as the library version and click ‘Finish’. JUnit has now been added to your project libraries and you’re ready to go.

junit_add_library

Creating a first test script
Now, let’s create a first JUnit test script. First of all, as JUnit tests are Java classes in themselves, we create a new source folder in our project to prevent test code getting mixed up with application code. Right-click on your project, select Properties and in the Source tab, select ‘Add Folder’. Name this folder ‘test’ for instance and add it to your project.

It’s a good idea to structure the test code in your test folder similar to the application code to be tested. So, if the Java class you are writing tests for is in package x.y.z in the ‘src’ folder, your test code for this class is placed in package x.y.z in the ‘test’ folder. Create this package in the ‘test’ folder by right-clicking it and selecting ‘New > Package’.

One of the features of Eclipse is that it can automatically generate JUnit test skeletons for you based on the definition of the class you’re writing tests for. To do so, right-click on the package you’ll place the test code in and select ‘New > JUnit Test Case’. Name your test class – I prefer adding ‘Test’ to the name of the class to be tested, so tests for class Apple.java are placed in AppleTest.java – and select the class under test using the button ‘Select’ next to ‘Class under test:’. Click ‘Next’ and select the methods for which you want to generate JUnit tests.

junit_select_methods

Click ‘Finish’ to generate JUnit test skeletons for the selected methods. The code generated should be similar to this:

package com.ontestautomation.selenium.objectmap;

import static org.junit.Assert.*;

import org.junit.Test;

public class ObjectMapTest {

	@Test
	public void testObjectMap() {
		fail("Not yet implemented");
	}

	@Test
	public void testGetLocator() {
		fail("Not yet implemented");
	}

}

The annnotation @Test is used by JUnit to indicate the start of a test case. The generated test code is neatly placed in the right package in your Java project:

junit_package_explorer

Running your test
To run your JUnit tests, simply right-click the test code file and select ‘Run As > JUnit Test’. Your test methods will be executed and a new tab opens displaying the test results:

junit_test_results_1

Both tests fail for now as they have not yet been implemented. That is, no actual tests are being executed so far as we haven’t written the tests itself yet. Below you see a simple test for the getLocator method that tests whether the object retrieved from the object map using this method is equal to the object that is expected.

@Test
public void testGetLocator() {
	ObjectMap objMap = new ObjectMap("objectmap.properties");
	try {
		By testLocator = objMap.getLocator("bing.homepage.textbox");
		assertEquals("Check testLocator object",By.id("sb_form_q"),testLocator);
	} catch (Exception e) {
		System.out.println("Error during JUnit test execution:\n" + e.toString());
	}
		
}

If we rerun the test code, we now see that the test for the getLocator method passes, meaning we have successfully implemented and run our first JUnit test!

junit_test_results_2

Useful features
To write more advanced and better maintainable tests, JUnit provides some nice features. The most important of these features are:

  • The ability to test for expected exceptions. If you want to validate that the method you developed throws the right exception under the right circumstances, you can easily verify this with JUnit, using the expected=NameOfException.class notation:
@Test(expected=Exception.class)
public void testGetLocatorException() throws Exception {
	ObjectMap objMap = new ObjectMap("objectmap.properties");
	By testLocator = objMap.getLocator("unknownobject");
}
  • The ability to create test suites using the @Suite annotation. Using this you can run a set of test classes by calling a single test suite class.
  • The ability to parameterize tests with test data, using the @Parameterized annotation. Using this you can make your unit tests data driven and run the same tests using multiple sets of test data without the need for duplicate test code.
  • The ability to easily export test results to continuous integration frameworks, such as Jenkins. JUnit generates reports in an XML format that can easily be interpreted by Jenkins and similar CI frameworks. This results in a very readable graphical representation of your JUnit test results:

build_result_test_detail

Further reading
For more information on JUnit, you can visit the sites below:

An Eclipse project including the tests I’ve demonstrated above and the reports that have been generated can be downloaded here.

Happy unit testing!