Testing web services using in-memory web containers

Last week, I read an interesting blog post on how to test Java web services inside an in-memory web container. I thought it was a fast and elegant solution to overcome a well-known unit testing problem: unit tests for web services become cumbersome when your web services uses the services provided by the container and you need to mock these container services. On the other hand, if you don’t mock these, your unit tests are virtually worthless. The blog post, originally written by Antonio Goncalves, provides an elegant solution to this problem: use an in-memory container. In this case, he uses Sun’s implementation of Java SE 6, which includes a light-weight HTTP server API and implementation: com.sun.net.httpserver.

In this post, I will show a slightly improved version of his solution using my own web service implementation. In order to get the example below to work, all I needed to do was include a recent version of the javax.jws library, which you can find for example here.

I have constructed a simple service that, given a composer, returns his name, his best known work, his country of birth and his age of death. The interface for this service looks like this:
Continue reading

Testing REST services with REST Assured

For those of you wanting to add the possibility to validate RESTful web services to your test automation framework, REST Assured can be a very useful way to do just that. In this post, I want to show you how to perform some basic tests on both JSON and XML-based REST services.

Installing and configuring REST Assured is easy, just download the latest version from the website and add the relevant .jar files to your testing project and you’re good to go.

Testing a REST service that returns XML
First, let’s try validating a REST service that returns a response in XML format. As an example, I use the ParaBank REST service to get the customer details for a customer with ID 12212 (click here to invoke the service and see the response in your browser). For this response, I want to check that the returned customer ID is equal to 12212, that the first name is equal to John, and that the last name is equal to Doe. The following REST Assured-test does exactly that. Note that the test consists of just a single statement, but I’ve added some line breaks to make it more readable.

package com.ontestautomation.restassured.test;

import static com.jayway.restassured.RestAssured.*;
import static org.hamcrest.Matchers.*;

public class RestServiceTest {
	public static void main (String args[]) {
			assertThat().body("customer.id", equalTo("12212")).
			assertThat().body("customer.firstName", equalTo("John")).
			assertThat().body("customer.lastName", equalTo("Doe"));		

Pretty straightforward, right? All you need to do is invoke the service using get() and then perform the required checks using assertThat() and the equalTo() matcher (which is part of Hamcrest by the way, not of REST Assured). Identifying the response elements to be checked is done using a dot notation. For example, customer.id identifies the id element that is a child of the customer element, which is the root element of the response message.

When we run this test, we see the results displayed neatly in our IDE:
REST Assured test resultsOne of the assertions fails, since the actual last name for customer 12212 is Smith, not Doe.

Testing a REST service that returns JSON
REST Assured can be used just as easily to perform checks on REST services that return JSON instead of XML. In the next example, I use a REST service that takes a text string and returns the md5 checksum for that string, together with the original string.
REST service with JSON responseThe following code example checks both the original string value and the md5 checksum value for this REST service:

package com.ontestautomation.restassured.test;

import static com.jayway.restassured.RestAssured.*;
import static org.hamcrest.Matchers.*;

public class RestServiceTestGherkin {
	public static void main(String args[]) {
			parameters("text", "test").
			body("original", equalTo("incorrect"));

As you can see, checking values from JSON responses is just as straightforward as is the case for XML responses. Two additional things do have changed in this example though:

  • As this web service call takes parameters – in this case a single parameter containing the string for which the md5 value is to be calculated – we need to define the value of this parameter before calling the service. This is done using the parameters() method, which takes parameter key-value pairs as its argument. Alternatively, we could have omitted the parameters call and just perform a get(“http://md5.jsontest.com?text=test”) instead, but this is both more elegant and more maintainable.
  • Also, as you can see from this example, in REST Assured, you can specify your tests in a BDD format using the Given-When-Then construction that is also used (for example) in Cucumber.

When we run this test, again, the test results show up nicely in our IDE:
Test results for JSON REST service testI’ve added another defect on purpose to show how they are displayed: the value of the original element in the response should of course be test instead of incorrect.

Combining REST Assured and TestNG
Finally, we can also combine REST Assured with TestNG to create an even more powerful REST service testing framework. In that way, you can for example use the reporting capabilities of TestNG for your REST service tests, as well as integrate REST Assured tests into an existing test framework. Doing so is as easy as replacing the default Hamcrest assertions with TestNG assertions and performing the required checks:

package com.ontestautomation.restassured.test;

import static com.jayway.restassured.RestAssured.*;

import org.testng.Assert;
import org.testng.annotations.Test;

import com.jayway.restassured.path.xml.XmlPath;

public class RestServiceTestNG {
	public void testng() {
		// Get the response XML from the REST service and store it as a String
		String xml = get("http://parabank.parasoft.com/parabank/services/bank/customers/12212/").andReturn().asString();
		// Retrieve the values to be checked from the XML as a String
		XmlPath xmlPath = new XmlPath(xml).setRoot("customer");
		String customerId = xmlPath.getString("id");
		String firstName = xmlPath.getString("firstName");
		String lastName = xmlPath.getString("lastName");
		// Perform the required checks
		Assert.assertEquals(customerId, "12212");
		Assert.assertEquals(firstName, "John");
		Assert.assertEquals(lastName, "Doe");

When using TestNG, we first store the response of the web service as a String, after which we can check whatever we want to check. It does make the test code a little longer, and in my personal opinion a little less elegant. On the upside, we can now seamlessly integrate our REST Assured-tests in a larger TestNG-based test framework and profit from the automatic report generation that TestNG provides.

Running this test results in the following output in our IDE:
Test results for TestNG-based testsand the following HTML report:
HTML results for the TestNG-based REST Assured testsAdditional features
Apart from the options shown above, REST Assured provides some other features that can come in very handy when testing REST-based web services. For instance, you can also test services that require authentication in order to be invoked. For basic authentication, you can simply add the credentials to your test statement as follows:

given().auth().basic("username", "password").when().

It is also possible to invoke services that require OAuth authorization (this requires the help of the Scribe library):


If you want to explicitly validate that a web service returns the correct content type (JSON, in this case), you can do this:


For a complete overview of all REST Assured features, you can refer to the online documentation.

Happy REST testing!

Data driven testing using a test data database

In a previous post I explained how to set up a data driven test in Selenium Webdriver using data from an Excel worksheet. However, you might have your potential test data stored in a database rather than in Excel. In this post, I will show you how to set up and run a data driven test using data from a database. In this example, I will use a REST webservice as the object to be tested and therefore won’t use Selenium Webdriver, but you can easily apply this approach to your Selenium tests as well.

First, we need a table containing our test data. For this example, I have created a simple table in a local MySQL installation, containing the following data:

The table containing our test data

As our test object, I am going to use a public REST service API that returns, amongst other data, the city and state corresponding to a US zip code in JSON format (click here for an example).

Now, let’s create a test that calls this API for all zipcodes in our test data table and verify whether the city and state returned by the service match the expected values stored in our table.

To do that, we first need to create a connection to our database table, retrieve the test data from it and call our test method for every row in the table. This is done using the following piece of code:

public static void runTest() {
	try {
		// Retrieve database connection properties from the properties file
		String driver = DBDrivenProperties.getProperty("db.driver");
		String dburl = DBDrivenProperties.getProperty("db.url");
		String dbname = DBDrivenProperties.getProperty("db.dbname");
		String dbquery = DBDrivenProperties.getProperty("db.query");
		String dbuser = DBDrivenProperties.getProperty("db.username");
		String dbpassword = DBDrivenProperties.getProperty("db.password");
		// Load the MySQL JDBC driver
		// Create a connection to the MySQL database
		Connection conn = DriverManager.getConnection(dburl + dbname, dbuser, dbpassword);
		// Create a statement to be executed
		Statement stmt = conn.createStatement();
		// Execute the query
		ResultSet rs = stmt.executeQuery(dbquery);
		// Loop through the query results and run the REST service test for every row
		while (rs.next()) {
			String zipcode = rs.getString("zipcode");
			String city = rs.getString("city");
			String state = rs.getString("state");
			try {
			} catch (IOException | JSONException e) {
		// Close the database connection
	} catch (ClassNotFoundException | SQLException e) {

For this method to run, we need to add a MySQL JDBC driver to the classpath of our project, otherwise we get an error when we try to load the driver. You can get yours here.

For clarity, I have put all the configuration data that is needed to connect to the database and get the results from it in a separate properties file that looks like this:

The properties file

You can find the code that I use to retrieve the property values from this file in the project files. See the end of this post for a link to it.

Now that we have retrieved our test data from the database, let’s write the actual test method. This is very similar to the one I used in a previous post:

public static void testService(String zipcode, String city, String state) throws IOException, JSONException {
	System.out.println("Validating response for " + DBDrivenProperties.getProperty("rest.url") + zipcode + "...");
	// Retrieve the base URL for the REST service and append the zipcode parameter
	String restURL = DBDrivenProperties.getProperty("rest.url") + zipcode;
	// Call the REST service and store the response
	HttpUriRequest request = new HttpGet(restURL);
	HttpResponse httpResponse = HttpClientBuilder.create().build().execute(request);

	// Convert the response to a String format
	String result = EntityUtils.toString(httpResponse.getEntity());

	// Convert the result as a String to a JSON object
	JSONObject jo = new JSONObject(result);
	// Get the array containing the places that correspond to the requested zipcode
	JSONArray ja = jo.getJSONArray("places");
	// Assert that the values returned by the REST service match the expected values in our database
	Assert.assertEquals(city, ja.getJSONObject(0).getString("place name"));
	Assert.assertEquals(state, ja.getJSONObject(0).getString("state"));	

The only thing that is new compared to the code in my previous post on testing REST webservices is that in this response, the elements I am interested in (being the city and state corresponding to the zip code in the request) are stored within a result array called places. To retrieve these, I need to dig one level deeper into my JSON response object using the JSONArray object. Other than that, the test method is pretty straightforward.

One warning I need to address before you go ahead and create your own tests using test data from a database is that I used a very broad query in this example (a simple SELECT * FROM table). This potentially generates a lot of results and subsequently a lot of test iterations. Even though this gives great test coverage, it also takes more time to execute all test cases. Especially when you use this approach in combination with Selenium Webdriver, you might want to use a narrower query (or limit the number of results returned) to prevent your test from taking too long to finish.

The Eclipse project files including all code needed to get this to work can be downloaded here.