Testing RESTful webservices

With the world of IT systems and applications becoming more and more distributed, testing the interfaces between system components becomes ever more important. In a previous post I have introduced a very basic way to test a SOAP web service interface. However, nowadays, in the world of APIs, the REpresentational State Transfer or REST standard becomes increasingly popular due to its lightweight nature. This article introduces a way to test RESTful APIs in a simple yet effective manner, and in a way that integrates smoothly with your existing Java-based testing frameworks, such as those based on Selenium Webdriver.

RESTful APIs come in several varieties. In this post, we are going to see two of them: one returning XML responses (I used this one) and one returning JSON responses (I used this one). You’re welcome to click on either link to see the difference in response format.

Also, I have used some external libraries to get the code to work:

Verify the HTTP status code
The first thing we are going to test for our services is whether they respond to our requests at all. This means we are going to validate whether the HTTP response code for the REST service is correct. There way to do this is equal for both XML and JSON REST APIs:

public static void testStatusCode(String restURL) throws ClientProtocolException, IOException {

	HttpUriRequest request = new HttpGet(restURL);
	HttpResponse httpResponse = HttpClientBuilder.create().build().execute(request);
		
	Assert.assertEquals(httpResponse.getStatusLine().getStatusCode(),HttpStatus.SC_OK);
}

Pretty simple, right? As you probably know, calling a RESTful API is done by either sending a HTTP POST or a HTTP GET to a specific URL. This in contrast with SOAP web services, where you send a specific XML message to an endpoint. Then, retrieve the HTTP status code from the response and check whether this is OK (i.e., equal to HTTP 200).

Verify the response content type
Next, we are going to verify whether our web service sends us the content type we expect (either XML or JSON). Again, this can be done with just a couple of lines of code:

public static void testMimeType(String restURL, String expectedMimeType) throws ClientProtocolException, IOException {
		
	HttpUriRequest request = new HttpGet(restURL);
	HttpResponse httpResponse = HttpClientBuilder.create().build().execute(request);
		
	Assert.assertEquals(expectedMimeType,ContentType.getOrDefault(httpResponse.getEntity()).getMimeType());
}

Again, just call the RESTful API URL and retrieve the information you need from the response. In this case, we are interested in the MIME type, which should be either ‘application/xml’ or ‘application/json’, depending on the URL we use.

Verify the response content
Now that we have verified whether our services return correct responses with the correct MIME type, it’s time to look at the actual content of the response. This is where testing XML RESTful APIs differs from testing JSON RESTful APIs, since we need to parse the response to extract the elements and element values we’re interested in.

For XML, we can do this using the following piece of code:

public static void testContent(String restURL, String element, String expectedValue) throws ClientProtocolException, IOException, SAXException, ParserConfigurationException {
		
	Document doc = DocumentBuilderFactory.newInstance().newDocumentBuilder().parse(restURL);
	NodeList nodelist = doc.getElementsByTagName(element);
		
	Assert.assertEquals(expectedValue,nodelist.item(0).getTextContent());		
}

Note that we use a different way to retrieve the result from the REST URL as we need to parse it as actual XML. After we’ve done that, we simply look for the XML element we’re interested in and compare its value to the value we expect.

For JSON, we can verify element values as follows:

public static void testContentJSON(String restURL, String element, String expectedValue) throws ClientProtocolException, IOException, SAXException, ParserConfigurationException, JSONException {

	HttpUriRequest request = new HttpGet(restURL);
	HttpResponse httpResponse = HttpClientBuilder.create().build().execute(request);

	// Convert the response to a String format
	String result = EntityUtils.toString(httpResponse.getEntity());

	// Convert the result as a String to a JSON object
	JSONObject jo = new JSONObject(result);

	Assert.assertEquals(expectedValue, jo.getString(element));
}

Here, we need to take an intermediate step to parse the response to a sensible format. First, we convert the HttpResponse to a String, then we convert this String to a JSONObject. From this JSONObject we can then extract the value for a specific element and compare it to an expected value.

The above examples should get you started nicely when you’re asked to include testing RESTful web services in your automated testing solutions.

Again, happy testing!

An example Eclipse project where I use the code above can be downloaded here.

An introduction to service virtualization

One of the concepts that is rapidly gaining popularity within the testing world – and in IT in general – is service virtualization (SV). This post provides a quick introduction to the SV concept and the way it leverages automated and manual testing efforts and thereby software development in general.

What is service virtualization?
SV is the concept of simulating the behaviour of an application or resource that is required in order to perform certain types of tests, in case this resource is not readily available or the availability or use of this resource is too expensive. In other words, SV is aimed at the removal of traditional dependencies in software development when it comes to the availability of systems and environments. In this way, SV is complementary to other forms of virtualization, such as hardware (VPS) or operating system (VMware and similar solutions) virtualization.

Behaviour simulation is carried out using virtual assets, which are pieces of software that mimic application behaviour in some way. Although SV started out with the simulation of web service behaviour, modern SV tools can simulate all kinds of communication that is performed over common message protocols. In this way, SV can be used to simulate database transactions, mainframe interaction, etc.

http://www.w3.org/WAI/intro/people-use-web/Overview.html

From a ‘live’ test environment to a virtual test environment

What are the benefits of using service virtualization?
As mentioned in the first paragraph of this post, SV can significantly speed up the development process in case required resources:

  • are not available during (part of) the test cycle, thereby delaying tests or negatively influencing test coverage;
  • are too expensive to keep alive (e.g., test environments need to be maintained or rented continuously even though access is required only a couple of times per year);
  • cannot readily emulate the behaviour required for certain types of test cases;
  • are shared throughout different development teams, negatively influencing resource availability.

Service virtualization tools
Currently, four commercial service virtualization tools are available on the market:

Furthermore, several open source service virtualization projects have emerged in recent years, such as WireMock and Betamax. These offer significantly less features, obviously, but they just might fit your project requirements nonetheless, making them worthy of an evaluation.

Personally, I have extensive experience using Parasoft Virtualize and have been able to successfully implement it in a number of projects for our customers. Results have been excellent, as can be seen in the following case study.

A case study
The central order management application at one of our customers relied heavily on an external resource for the successful provisioning of orders. This external resource requires manual configuration for each order that is created, resulting in the test team having to file requests for configuration changes for each test case. The delay for this configuration could be as much as a week, resulting in severe delays in testing and possible test coverage (as testers could only create a small amount of orders per test cycle).

Using service virtualization to simulate the behaviour of this external resource, this dependency has been removed altogether. The virtual asset implementing the external resource behaviour processes new orders in a matter of minutes, as opposed to weeks in case the ‘real’ external resource is required. Using SV, testers can provision orders much faster and are able to achieve much higher test coverage as a result. This has resulted in a significant increase in software quality. Also, SV has been a key factor in the switch to an Agile development process. Without SV, the short development and testing cycles associated with the Agile software development method would not have been possible.

Service virtualization on Wikipedia

A very basic web service test tool

For those of us involved in testing web services and SOA-based applications, there are lots of different test tools on the market that can help speed up the testing and make them repeatable and easy to maintain. Some examples of these tools are SoapUI from SmartBear and SOAtest from Parasoft. SoapUI is available both as a freeware product and as an enterprise edition offering additional functionality. SOAtest is only available in a paid version.

One of the biggest downsides of the freeware version of SoapUI is the lack of data driven test support. This feature is only available in the paid version. In a previous post I introduced a simple way of implementing data driven testing in Selenium Webdriver tests, using Microsoft Excel sheets as the test data container format. As I am very regularly involved in web service testing myself, and often want to use data driven testing to create maintainable and flexible test suites, I thought it would be a good idea to see whether it’s possible to quickly create a very basic web service testing tool that supports data driven testing.

The test data source
As in the previous example, let’s create a test data source first. I chose to base the solution presented here on predefined XML request message files as this saves us the trouble of creating XML objects in our code. There are obvious downsides to this, of course, but for now we will focus on the ability to call a web service and subsequently capture and validate the result.

Our test data source looks like this:

webservice_test_data_source

It contains columns to identify the current test case, the path to the XML file containing the SOAP message to be sent and the endpoint to which the message should be sent. The last two columns contain the name of an element in the web service response we would like to validate and its expected value, respectively.

Reading our test data source
In our very basic web service test tool, we will process this Excel sheet just as we did in the Selenium example posted earlier. For every test data row, we will then execute the following steps:

  • Create a SOAP request from the XML file
  • Send the SOAP request to the right web service endpoint and capture the response
  • Extract the value from a response message element and compare it to the expected value

Create a SOAP request from the XML file
This is an easy step, as all we need to do is open the file, read all of its contents and transform it into a SOAP message object:

private static SOAPMessage createSOAPRequest(String strPath) throws Exception {
        
    // Create a SOAP message from the XML file located at the given path
    FileInputStream fis = new FileInputStream(new File(strPath));
    MessageFactory factory = MessageFactory.newInstance();
    SOAPMessage message = factory.createMessage(new MimeHeaders(), fis);
    return message;
}

Send the SOAP request to the right web service and capture the response
This is fairly straightforward as well and can be done using standard Java methods:

private static SOAPMessage getSOAPResponse(SOAPMessage soapRequest, String strEndpoint) throws Exception, SOAPException {
    	
    // Send the SOAP request to the given endpoint and return the corresponding response
    SOAPConnectionFactory soapConnectionFactory = SOAPConnectionFactory.newInstance();
    SOAPConnection soapConnection = soapConnectionFactory.createConnection();
    SOAPMessage soapResponse = soapConnection.call(soapRequest, strEndpoint);
    return soapResponse;	
}

Validating elements from the response message
Finally, we will parse the response message and validate the value of one of its elements. The validation result is sent to the stdout:

private static void validateValue(SOAPMessage soapMsg, String strEl, String strExpected) throws Exception {
    	
    // Get all elements with the requested element tag from the SOAP message
    SOAPBody soapBody = soapMsg.getSOAPBody();
    NodeList elements = soapBody.getElementsByTagName(strEl);
        
    // Check whether there is exactly one element with the given tag
    if (elements.getLength() != 1){
        System.out.println("Expected exactly one element " + strEl + "in message, but found " + Integer.toString(elements.getLength()));
    } else {
        // Validate the element value against the expected value
        String strActual = elements.item(0).getTextContent();
        if (strActual.equals(strExpected)) {
        	System.out.println("Actual value " + strActual + " for element " + strEl + " matches expected value");
        } else {
        	System.out.println("Expected value " + strExpected + " for element " + strEl + ", but found " + strActual + " instead");
        }
    }
}

Running the test
When we run the test, we see that the web service to be tested is called three times. Two tests succeed, the last test case fails (on purpose, to show that the validation is executed properly):

webservice_console_output

Extensions to our tool
As stated earlier in this post, there are a lot of improvements to be made to this very basic web service test tool. For instance, we could add:

  • Dynamic request message generation based on a template and data source values
  • XSD validation for the response messages
  • Support for non-SOAP-based (or plain XML) web services or even for transport protocols other than HTTP

Some of these improvements will probably be featured in later articles at ontestautomation.com. For questions or suggestions on topics to be covered on this blog, please do not hesitate to contact me at bas AT ontestautomation.com or leave a reply through the comment form below.

An example Eclipse project using the pattern described above can be downloaded here.