Getting started with: JGiven

This is the eighth article in our series on new, popular or otherwise interesting tools used in test automation. You can read all posts within this series by clicking here.

What is JGiven?
From the JGiven.org website: JGiven is a developer-friendly and pragmatic BDD tool for Java. Developers write scenarios in plain Java using a fluent, domain-specific API, JGiven generates reports that are readable by domain experts.

In short, JGiven can be used as an alternative to other BDD frameworks, such as JBehave and Cucumber. Where the latter two separate the features from the test code, JGiven does not and might therefore be considered more suitable to unit and integration tests rather than automated system and regression tests.

Where can I get JGiven?
The JGiven sources can be downloaded from the JGiven GitHub site.

How do I install and configure JGiven?
The easiest way to get started with JGiven is by creating a new Maven project and including the following dependency:

<dependency>
	<groupId>com.tngtech.jgiven</groupId>
	<artifactId>jgiven-testng</artifactId>
	<version>0.9.5</version>
	<scope>test</scope>
</dependency>

This is for use in combination with TestNG, which I will do in the rest of this post. If you prefer to user JUnit, use jgiven-junit as your artifactId. Please note that this dependency does not include TestNG (or JUnit) itself, so be sure to include it separately in your pom.xml.

Creating and running a first JGiven test
Taking my trusted ParaBank application as an example, I want to perform a test determining whether I can login successfully using a JGiven scenario. I.e., I want to execute the following scenario:

Given I am at the login page
When I login as john with password demo
Then the login action will be successful

The implementation of this scenario as a TestNG-based test in JGiven is pretty straightforward:

public class LoginTest extends ScenarioTest<GivenIAmAtTheLoginPage, WhenILoginAsJohnWithPasswordDemo, ThenTheLoginActionWillBeSuccessful> {
	
	@Test
	public void aFirstLoginTest() {
		
		given().I_am_at_the_login_page();
		when().I_login_as_john_with_password_demo();
		then().the_login_action_will_be_successful();
	}
}

The ScenarioTest class requires three parameters, each representing a stage in the Given-When-Then scenario. To make this compile, we also need to implement each of the three classes that are used as a parameter. As an example, this is what the implementation of the GivenIAmAtTheLoginPage class looks like:

public class GivenIAmAtTheLoginPage extends Stage<GivenIAmAtTheLoginPage>{
	public GivenIAmAtTheLoginPage I_am_at_the_login_page() {
		
		return self();
	}
}

The other two classes (stages) are implemented in a similar way. Now that this is done, we can run our test, which lads to the following output in the console:

Test Class: com.ontestautomation.jgiven.tests.LoginTest

 Scenario: A first login test

   Given I am at the login page
    When I login as john with password demo
    Then the login action will be successful

PASSED: aFirstLoginTest

We can see that our test passes, so we have something to build upon. No actual test actions are performed yet, so we are going to add these next. Since we are performing a Selenium WebDriver test, it’s required that we pass along the browser instance as a parameter for each of the steps. This also means we can initialize it before the test is run (using @BeforeTest) and destroy it afterwards (using @AfterTest):

public class LoginTest extends ScenarioTest<GivenIAmAtTheLoginPage, WhenILoginAsJohnWithPasswordDemo, ThenTheLoginActionWillBeSuccessful> {
	
	WebDriver driver;
	
	@BeforeTest
	public void initBrowser() {

		driver = new FirefoxDriver();
		driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
	}
	
	@Test
	public void aFirstLoginTest() {
		
		given().I_am_at_the_login_page(driver);
		when().I_login_as_john_with_password_demo(driver);
		then().the_login_action_will_be_successful(driver);
	}
	
	@AfterTest
	public void tearDown() {
		
		driver.quit();
	}
}

The WebDriver actions associated with the Given, When and Then steps can now be added to the respective classes as well, for example for GivenIAmAtTheLoginPage this results in:

public class GivenIAmAtTheLoginPage extends Stage<GivenIAmAtTheLoginPage>{
	
	WebDriver _driver;
	
	public GivenIAmAtTheLoginPage I_am_at_the_login_page(WebDriver driver) {
		
		_driver = driver;
		
		_driver.get("http://parabank.parasoft.com");
		
		return self();
	}
}

If we implement the other two Stage classes in a similar vein – where the Then stage should include an actual TestNG assertion to make it a proper test case – and rerun our test, we can see that it passes again:

Test Class: com.ontestautomation.jgiven.tests.LoginTest

 Scenario: A first login test

   Given I am at the login page FirefoxDriver: firefox on WINDOWS (6bccf261-5ce0-4378-8118-b545b6c82eca)
    When I login as john with password demo FirefoxDriver: firefox on WINDOWS (6bccf261-5ce0-4378-8118-b545b6c82eca)
    Then the login action will be successful FirefoxDriver: firefox on WINDOWS (6bccf261-5ce0-4378-8118-b545b6c82eca)

PASSED: aFirstLoginTest

Note that JGiven automatically adds a bit of information on the browser instance used to the console output.

Useful features
Some useful additional features of JGiven are:

  • Several types of test execution and results reporting, including JSON and HTML
  • Support for parameterized steps (in the above example, you could for instance parameterize the username and password in the When step)
  • Support for tags to organize scenarios

Further reading
Apart from the features mentioned above, JGiven provides some more useful features for creating useful BDD-style Java-based tests. A complete reference guide can be found on the JGiven website.

An Eclipse Maven project including the tests and test classes I’ve used in this post can be downloaded here.

Getting started with: WireMock

This is the seventh article in our series on new, popular or otherwise interesting tools used in test automation. You can read all posts within this series by clicking here.

What is WireMock?
From the WireMock.org website: WireMock is a flexible library for stubbing and mocking web services. Unlike general purpose mocking tools it works by creating an actual HTTP server that your code under test can connect to as it would a real web service. It supports HTTP response stubbing, request verification, proxy/intercept, record/playback of stubs and fault injection, and can be used from within a unit test or deployed into a test environment. Although it’s written in Java, there’s also a JSON API so you can use it with pretty much any language out there.

In short, WireMock is a very handy tool for all those situations where you need to set up a mock version of a web service, for example for testing or development purposes.

Where can I get WireMock?
WireMock can be downloaded from the WireMock Maven repository.

How do I install and configure WireMock?
Installing WireMock is as simple as downloading the latest standalone version from the WireMock Maven repository and adding it as a dependency to your Java project.

Creating a first WireMock mock service
Let’s start by creating a very simple mock service that returns a plain text string and a HTTP status code.

public void setupStub() {
		
	stubFor(get(urlEqualTo("/an/endpoint"))
            .willReturn(aResponse()
                .withHeader("Content-Type", "text/plain")
                .withStatus(200)
                .withBody("You've reached a valid WireMock endpoint")));
}

This code snippet creates a simple mock service that runs at the /an/endpoint endpoint on the WireMock server (your localhost most of the time, but more about that in a bit). It returns a response with content type text/plain, HTTP status 200 and body text “You’ve reached a valid WireMock endpoint“.

Running and testing your mock service
Of course, we would also like to validate whether the mock service we created does what we told it to do. In this post, I will use JUnit-based REST Assured tests to validate the behaviour of our mocks. But first, we need to get our mock service up and running before we start testing. This can be done very easily with WireMock using a JUnit @Rule:

@Rule
public WireMockRule wireMockRule = new WireMockRule(8090);

This rule starts and stops the mocks defined in your JUnit test class for every test in it. Next, we can define a couple of REST Assured tests to see whether the mock service is doing what we expect:

@Test
public void testStatusCodePositive() {
		
	setupStub();
		
	given().
	when().
		get("http://localhost:8090/an/endpoint").
	then().
		assertThat().statusCode(200);
}
	
@Test
public void testStatusCodeNegative() {
	
	setupStub();
		
	given().
	when().
		get("http://localhost:8090/another/endpoint").
	then().
		assertThat().statusCode(404);
}
	
@Test
public void testResponseContents() {
		
	setupStub();
	
	String response = get("http://localhost:8090/an/endpoint").asString();
	Assert.assertEquals("You've reached a valid WireMock endpoint", response);
}

When we run these tests, they will all pass, indicating our mock service behaves as expected:
WireMock JUnit test results

Request-response matching
WireMock also allows you to define a stub that returns a particular answer based on (part of) the request message using the matching matcher method:

public void setupStub() {
		
	stubFor(post(urlEqualTo("/pingpong"))
			.withRequestBody(matching("<input>PING</input>"))
            .willReturn(aResponse()
                .withStatus(200)
                .withHeader("Content-Type", "application/xml")
                .withBody("<output>PONG</output>")));
}

This service only returns a status code 200 and a response XML message when the request body equals a given value. Again, we can validate this easily using REST Assured:

@Test
public void testPingPongPositive() {
		
	setupStub();
		
	given().
		body("<input>PING</input>").
	when().
		post("http://localhost:8090/pingpong").
	then().
		assertThat().
		statusCode(200).
		and().
		assertThat().body("output", org.hamcrest.Matchers.equalTo("PONG"));
}

WireMock also allows you to use regular expressions for more flexible request matching.

Creating stateful mocks
As a final example, I’ll show how to create a stateful mock. This is done in WireMock using scenarios:

public void setupStub() {
		
	stubFor(get(urlEqualTo("/todolist"))
			.inScenario("addItem")
			.whenScenarioStateIs(Scenario.STARTED)
			.willReturn(aResponse()
                .withStatus(200)
                .withHeader("Content-Type", "application/xml")
                .withBody("<list>Empty</list>")));
		
	stubFor(post(urlEqualTo("/todolist"))
			.inScenario("addItem")
			.whenScenarioStateIs(Scenario.STARTED)
			.willSetStateTo("itemAdded")
			.willReturn(aResponse()
				.withHeader("Content-Type", "application/xml")
                .withStatus(201)));
		
	stubFor(get(urlEqualTo("/todolist"))
			.inScenario("addItem")
			.whenScenarioStateIs("itemAdded")
			.willReturn(aResponse()
                .withStatus(200)
                .withHeader("Content-Type", "application/xml")
                .withBody("<list><item>Item added to list</item></list>")));	
}

So, when we first perform a GET on this mock, an empty to-do list is returned. Then, when we do a POST, the mock service state is changed from Scenario.STARTED (a default in WireMock) to “itemAdded“. Then, when we do another GET, we get a to-do list with a single item on it:

@Test
public void testStatefulMock() {
		
	setupStub();
		
	given().
	when().
		get("http://localhost:8090/todolist").
	then().
		assertThat().
		statusCode(200).
		and().
		assertThat().body("list", org.hamcrest.Matchers.equalTo("Empty"));
		
	given().
	when().
		post("http://localhost:8090/todolist").
	then().
		assertThat().
		statusCode(201);
		
	given().
	when().
		get("http://localhost:8090/todolist").
	then().
		assertThat().
		statusCode(200).
		and().
		assertThat().body("list", org.hamcrest.Matchers.not("Empty")).
		and().
		assertThat().body("list.item", org.hamcrest.Matchers.equalTo("Item added to list"));
}

A trivial example, perhaps, but nevertheless it shows you how can create stateful mocks easily using WireMock.

Further reading
WireMock provides many more useful features for creating just the mock you want. A complete reference guide can be found on the WireMock website.

An Eclipse project including the mock services and REST Assured tests I’ve used in this post can be downloaded here.

Up and running with: XMLUnit

This is the sixth article in our series on new, popular or otherwise interesting tools used in test automation. You can read all posts within this series by clicking here.

What is XMLUnit?
From the XMLUnit.org website: XMLUnit provides you with the tools to verify the XML you emit is the one you want to create. It provides helpers to validate against an XML Schema, assert the values of XPath queries or compare XML documents against expected outcomes. The most important part is a diff-engine that provides you with full control over what kind of difference is important to you and which part of the generated document to compare with which part of your reference document.

Where can I get XMLUnit?
XMLUnit can be downloaded from the XMLUnit GitHub site. Snapshot binaries for XMLUnit 2.0 (the version that is covered in this post) can also be obtained from the Maven repository.

How do I install and configure XMLUnit?
Installing XMLUnit is as simple as downloading the latest xmlunit-core snapshot version and adding it as a dependency to your Java project.

Creating a first XMLUnittest
Let’s start with a very simple test that validates the value of a specific element of a predefined XML message:

@Test
public void aFirstTest() {
		
	Source source = Input.fromString("<foo>bar</foo>").build();
	XPathEngine xpath = new JAXPXPathEngine();
	String content = xpath.evaluate("/foo/text()", source);
	Assert.assertEquals(content, "bar");
}

Note that I have created this XMLUnit test as a TestNG test to make running the tests easier and to have TestNG automatically generate a test report.

Useful features
Using XMLUnit, XML validations can not only be done based on predefined strings containing XML, but also directly on the XML result of a call to a REST service. This is done in a very similar way:

@Test
public void restCallTest() {
		
	Source source = Input.fromURI("http://parabank.parasoft.com/parabank/services/bank/customers/12212").build();
	XPathEngine xpath = new JAXPXPathEngine();
	String content = xpath.evaluate("//city/text()", source);
	Assert.assertEquals(content, "Beverly Hills");
}

The only thing that is changed is the use of Input.fromURI instead of Input.fromString. Similar methods are available to perform XPath-based validations directly from an InputStream, a File, or a Node object, amongst others.

XMLUnit can not only perform element-based validation, it can also do a full compare of two XML messages. In this example, the XML returned by the same REST service call used in the previous example is compared against a predefined XML messages stored on the local file system (as response.xml in the messages subdirectory):

@Test
public void compareTest() {
		
	DiffBuilder db = DiffBuilder.compare(Input.fromFile("messages/response.xml")).withTest(Input.fromURI("http://parabank.parasoft.com/parabank/services/bank/customers/12212").build());
	Diff d = db.build();
		
	if(d.hasDifferences()) {
		Iterator<Difference> diffs = d.getDifferences().iterator();
		while(diffs.hasNext()) {
			Reporter.log(diffs.next().toString());
		}
	}
		
	Assert.assertEquals(d.hasDifferences(), false);
}

Any differences between the two XML files are written to the TestNG report file using the default Reporter:

XMLUnit differences written to a TestNG report

Finally, XMLUnit can also validate XML messages against an XSD document:

@Test
public void validationTest() {
		
	Validator v = Validator.forLanguage(Languages.W3C_XML_SCHEMA_NS_URI);
	v.setSchemaSources(Input.fromFile("messages/orderschema.xsd").build());
	ValidationResult result = v.validateInstance(Input.fromFile("messages/order.xml").build());
		
	Iterator<ValidationProblem> problems = result.getProblems().iterator();
		
	while (problems.hasNext()) {
		Reporter.log(problems.next().toString());
	}
		
	Assert.assertEquals(result.isValid(), true);
}

Again, any issues that occur during schema validation can be written to the TestNG report using the default Reporter:

XMLUnit schema issues in TestNG report

Further reading
An Eclipse project including the tests I’ve demonstrated above and the reports that have been generated can be downloaded here.