Selecting response elements with GPath in REST Assured

Hey, another post on REST Assured! This time I’d like to take a closer look at how you can select elements from a JSON or XML response to check their values. For this, REST Assured uses GPath, a path expression language integrated into the Groovy language. It is similar to XPath for XML, but GPath can handle both XML and JSON. This makes it an excellent fit for checking responses of RESTful web services. Read more on GPath here or here.

To see how GPath works and how you can use it effectively in REST Assured tests, let’s consider the following JSON response from the Ergast MRD API. This response lists all drivers for the 2016 Formula 1 season:

JSON response containing all Formula 1 drivers for the 2016 season

You can see the full response by selecting (or otherwise performing a GET call to) this link.

Extracting a single element value
Say we want to extract the driverId of the last driver and check that it is equal to wehrlein. Like with XPath, you can simply specify the path to the element starting with the root node and navigating through the tree until the required node is reached. Note that the index [-1] is used by GPath to denote the last matching element.

@Test
public void extractAndCheckSingleValue() {
		
	given().
	when().
		get("http://ergast.com/api/f1/2016/drivers.json").
	then().
		assertThat().
		body("MRData.DriverTable.Drivers.driverId[-1]",equalTo("wehrlein"));
}

Extracting a set of element values
Another check we might want to perform is that the collection of driverId values contains some specific values. This is done using a GPath expression very similar to the previous example:

@Test
public void extractAndCheckMultipleValues() {
		
	given().
	when().
		get("http://ergast.com/api/f1/2016/drivers.json").
	then().
		assertThat().
		body("MRData.DriverTable.Drivers.driverId",hasItems("alonso","button"));
}

Using the REST Assured base path
By now, you might have noticed that in every GPath expression we needed to start navigating from the root node MRData downwards. This results in quite long GPath expressions. REST Assured has a nifty little feature that can simplify these expressions by having you define a BasePath:

@BeforeTest
public void initPath() {
		
	RestAssured.rootPath = "MRData.DriverTable.Drivers";
}

This cleans up our tests and has a positive effect on maintainability:

@Test
public void extractAndCheckMultipleValues() {
		
	given().
	when().
		get("http://ergast.com/api/f1/2016/drivers.json").
	then().
		assertThat().
		body("driverId",hasItems("alonso","button"));
}

Extracting a specific subset of values
GPath also supports array slicing to retrieve specific subsets of a collection of values. In the example below, we check that the collection of elements with index [0] through [2] has exactly three items (the ones at index [0], [1] and [2]):

@Test
public void extractAndCheckArraySliceSize() {
		
	given().
	when().
		get("http://ergast.com/api/f1/2016/drivers.json").
	then().
		assertThat().
		body("driverId[0..2]",hasSize(3));
}

Filtering values
Using GPath, you can also filter values to return an even more specific subset of values. For example, if we only want to return the collection of permanentNumber values in use this year that are between 20 and 30 inclusive, we can do this:

@Test
public void extractAndCheckRange() {
		
	given().
	when().
		get("http://ergast.com/api/f1/2016/drivers.json").
	then().
		assertThat().
		body("findAll{Drivers->Drivers.permanentNumber >= \"20\" && Drivers.permanentNumber <= \"30\"}.permanentNumber",hasItem("22")).
		and().
		body("findAll{Drivers->Drivers.permanentNumber >= \"20\" && Drivers.permanentNumber <= \"30\"}.permanentNumber",not(hasItem("33")));
}

The quotes around the lower and upper boundary are required since the API call returns the permanentNumber values as strings instead of integers.

Use parameters
To make the previous example a little more flexible, we can parameterize both the lower and upper boundaries as well as the values that are expected to be (or not to be) in the resulting collection. Let’s specify these in a TestNG DataProvider first:

@DataProvider(name = "rangesAndValues")
public String[][] createTestDataObject() {
		
	return new String[][] {
		{"20","30","22","25"},
		{"30","40","33","31"},
		{"1","9","9","4"}
	};
}

Now we can apply these in our test:

@Test(dataProvider = "rangesAndValues")
public void extractAndCheckRangeParameterized(String lowerLimit, String upperLimit, String inCollection, String notInCollection) {
		
	given().
	when().
		get(getDriverListFor2016).
	then().		
		assertThat().
		body("findAll{Drivers->Drivers.permanentNumber >= \"" + lowerLimit + "\" && Drivers.permanentNumber <= \"" + upperLimit + "\"}.permanentNumber",hasItem(inCollection)).
		and().
		body("findAll{Drivers->Drivers.permanentNumber >= \"" + lowerLimit + "\" && Drivers.permanentNumber <= \"" + upperLimit + "\"}.permanentNumber",not(hasItem(notInCollection)));
}

Note that the syntax gets a little messy here, especially since we have to keep the escaped double quotes in the GPath expression. There might be an easier way to do this, but I haven’t found one that still supports the given()/when()/then() format. Any pointers on this are well appreciated!

Sample code
You can download a Maven project containing all the code examples presented in this blog post here.

Creating data driven API tests with REST Assured and TestNG

As you’ve probably read in previous posts on this site, I am a big fan of the REST Assured library for writing tests for RESTful web services. In this post, I will demonstrate how to create even more powerful tests with REST Assured by applying the data driven principle, with some help from TestNG.

Combining REST Assured and TestNG
REST Assured itself is a Domain Specific Language (DSL) for writing tests for RESTful web services and does not offer a mechanism for writing data driven tests (i.e., the ability to write a single test that can be executed multiple times with different sets of input and validation parameters). However, REST Assured tests are often combined with JUnit or TestNG, and the latter offers an easy to use mechanism to create data driven tests through the use of the DataProvider mechanism.

Creating a TestNG DataProvider
A TestNG DataProvider is a method that returns an object containing test data that can then be fed to the actual tests (REST Assured tests in this case). This data can be hardcoded, but it can also be read from a database or a JSON specification, for example. It’s simply a matter of implementing the DataProvider in the desired way. The following example DataProvider creates a test data object (labeled md5hashes) that contains some example text strings and their md5 hash:

@DataProvider(name = "md5hashes")
public String[][] createMD5TestData() {
		
	return new String[][] {
			{"testcaseOne", "4ff1c9b1d1f23c6def53f957b1ed827f"},
			{"testcaseTwo", "39738347fb533d798aca9ae0f56ca126"},
			{"testcaseThree", "db6b151bb4bde46fddb361043bc3e2d9"}
	};
}

Using the DataProvider in REST Assured tests (with query parameters)
Using the data specified in the DataProvider in REST Assured tests is fairly straightforward. In the example below, we’re using it in combination with a RESTful web service that uses a single query parameter:

@Test(dataProvider = "md5hashes")
public void md5JsonTest(String originalText, String md5Hash) {
		
	given().
		parameters("text", originalText).
	when().
		get("http://md5.jsontest.com").
	then().
		assertThat().
		body("md5", equalTo(md5Hash));
}

Notice how the data in the DataProvider can be used for specification of expected results just as easy as for specifying input parameters. When we run this test, we can see in the output that the test is run three times, once for every set of input and validation parameters:

Console output for our data driven REST Assured test

Using the DataProvider in REST Assured tests (with query parameters)
The other parameter approach commonly used with RESTful web services is the use of path parameters. Let’s create another DataProvider, this one specifying Formula 1 circuits and the country they are situated in:

@DataProvider(name = "circuitLocations")
public String[][] createCircuitTestData() {
		
	return new String[][] {
			{"adelaide","Australia"},
			{"detroit","USA"},
			{"george","South Africa"}
	};
}

The Ergast Developer API provides motor racing data through a public API (check it out, it’s really cool). Here’s how to parameterize the call to get specific circuit data and check the country for each circuit with REST Assured, again using our DataProvider:

@Test(dataProvider = "circuitLocations")
public void circuitLocationTest(String circuitId, String location) {
		
	given().
		pathParameters("circuitId",circuitId).
	when().
		get("http://ergast.com/api/f1/circuits/{circuitId}.json").
	then().
		assertThat().
		body("MRData.CircuitTable.Circuits[0].Location.country",equalTo(location));
}

The only real difference is in how to set up path parameter use in REST Assured (see also here), otherwise making your tests data driven is just as easy as with query parameters.

When we run this second test, we can see again that it’s run three times, once for every circuit specified:

Console output for our data driven REST Assured test (with path parameters_

A Maven Eclipse project containing the code demonstrated in this blog post can be downloaded here.

Running your tests in a specific order

General consensus within the test automation community is that your automated tests should be able to run independently. That is, tests should be runnable in any given order and the result of a test should not depend on the outcome of one or more previous tests. This is generally a good practice and one I try and adhere to as much as possible. Nothing worse than a complete test run being wasted because some initial test data setup actions failed, causing all other tests to fail as well. Especially when you’re talking about end-to-end user interface tests that only run overnight because they’re taking so awfully long to finish.

However, in some cases, having your tests run in a specific order can be the most pragmatic or even the only (practical) option. For example, in my current project I am automating of a number of regression tests that require specific test data to be present before the actual checks can be performed. And since:

  • I don’t want to rely on test data that’s already being present in the database
  • Restoring a database snapshot is not a feasible option (at least not at the moment)
  • Creation of suitable test data takes at least a minute but closer to two for most tests. This has to be done through the user interface since a lot of data is stored in blobs, making SQL updates a challenging strategy to say the least..

the only viable option is to make the creation of test data the first step in a test run. Creating the necessary test data before each and every individual test would just take too long.

Since we’re using SpecFlow, we’re creating test data in the first scenario in every feature. All other scenarios in the feature rely on this test data. This means that the test data creation scenario needs to be run first, otherwise the subsequent scenarios will fail. Using Background is not an option, because those steps are run before each individual scenario, whereas we want to run the test data creation steps once every feature.

The above situation is just one example of a case where being able to control the execution order of your tests can come in very useful. Luckily, most testing frameworks support this in one or more ways. In the remainder of this post, we’re going to have a look at how you can define test execution order in JUnit, TestNG and NUnit.

JUnit
Before version 4.11, JUnit did not support controlling the test execution order. However, newer versions of the test framework allow you to annotate your test classes using @FixMethodOrder, which enables you to select one of various MethodSorters. For example, the tests in this class are run in ascending alphabetical order, sorted by test method name:

@FixMethodOrder(MethodSorters.NAME_ASCENDING)
public class JUnitOrderedTests {
	
	@Test
	public void thirdMethod() {		
	}
	
	@Test
	public void secondMethod() {		
	}
	
	@Test
	public void firstMethod() {		
	}
}

Running these tests show that they are executed in the specified order:
JUnit FixMethodOrder result

TestNG
TestNG offers no less than three ways to order your tests:

Using preserve-order in the testng.xml file
You can use the preserve-order attribute in the testng.xml file (where you specify which tests will be run) to have TestNG run the tests in the order they appear in the XML file:

<test name="OrderedTestNGTests" preserve-order="true">
	<classes>
		<class name="TestNGTestClass">
			<methods>
				<include name="testOne" />
				<include name="testTwo" />
			</methods>
		</class>
	</classes>
</test>

Using the priority attribute
You can also use the priority attribute in your @Test annotation to prioritize your test methods and determine the order in which they are run:

public class TestNGPrioritized {
	
	@Test(priority = 3)
	public void testThree() {		
	}
	
	@Test(priority = 1)
	public void testOne() {		
	}
	
	@Test(priority = 2)
	public void testTwo() {		
	}	
}

Using dependencies
In TestNG, you can have tests and test suites depend on other tests / test suites. This also implicitly defines the order in which the tests are executed: when test A depends on test B, test B will automatically be run before test A. These dependencies can be defined in code:

public class TestNGOrderedTests {
	
	@Test(dependsOnMethods = {"parentTest"})
	public void childTest() {		
	}
	
	@Test
	public void parentTest() {		
	}
}

This works on method level (using dependsOnMethods) as well as on group level (using dependsOnGroups). Alternatively, you can define dependencies on group level in the testng.xml file:

<test name="TestNGOrderedTests">
	<groups>
		<dependencies>
			<group name="parenttests" />
			<group name="childtests" depends-on="parenttests" />
		</dependencies>
	</groups>
</test>

NUnit
NUnit (the .NET version of the xUnit test frameworks) does not offer an explicit way to order tests, but you can control the order in which they are executed by naming your methods appropriately. Tests are run in alphabetical order, based on their method name. Note that this is an undocumented feature that may be altered or removed at will, but it still works in NUnit 3, which was recently released, and I happily abuse it in my current project..

At the beginning of this post, I mentioned that in my current project, we use SpecFlow to specify our regression tests. We then execute our SpecFlow scenarios using NUnit as a test runner, so we can leverage this alphabetical test order ‘trick’ by naming our SpecFlow scenarios alphabetically inside a specific feature. This gives us a way to control the order in which our scenarios are executed:

Scenario: 01 Create test data
	Given ...
	When ...
	Then ...
	
Scenario: 02 Modify data
	Given ...
	When ...
	Then ...

Scenario: 03 Remove modified data
	Given ...
	When ...
	Then ...

Again, it is always best to create your tests in such a way that they can be run independently. However, sometimes this just isn’t possible or practical. In those cases, you can employ one of the strategies listed in this post to control your test order execution.