Using the TestNG ITestContext to create smarter REST Assured tests

In this post, I would like to demonstrate two different concepts that I think work very well together:

  • How to store and retrieve data objects using the TestNG ITestContext for better code maintainability
  • How to communicate with RESTful web services that use basic or OAuth2 authorization using REST Assured

Using the PayPal sandbox API as an example, I will show you how you can create readable and maintainable tests for secured APIs using TestNG and REST Assured.

The TestNG ITestContext
If you have a suffficiently large test suite, chances are high that you want to be able to share objects between individual tests to make your tests shorter and easier to maintain. For example, if you are calling a web service multiple times throughout your test suite and that web service requires an authentication token in order to be able to consume it, you might want to request and store that authentication token in the setup phase of your test suite, then retrieve and use it in all subsequent tests where this web service is invoked. This is exactly the scenario we’ll see in this blog post.

TestNG offers a means of storing and retrieving objects between tests through the ITestContext interface. This interface allows you to store (using the inherited setAttribute() method) and retrieve (using getAttribute()) objects. Since the ITestContext is created once and remains active for the duration of your test run, this is the perfect way to implement object sharing in your test suite. Making the ITestContext available in your test methods is easy: just pass it as a parameter to your test method (we’ll see an example further down).

REST Assured authentication options
As you might have read in one of my previous blog posts, REST Assured is a Java library that allows you to write and execute readable tests for RESTful web services. Since we’re talking about secured APIs here, it’s good to know that REST Assured supports the following authentication mechanisms:

  • Basic
  • Digest
  • OAuth (version 1 and 2)
  • Form

In the examples in this post, we’ll take a closer look at both Basic authentication (for requesting an OAuth token) and OAuth2 authentication (for invoking secured web service operations) in REST Assured.

The PayPal sandbox API
To illustrate the concepts introduced above I chose to use the PayPal sandbox API. This is a sandbox version of the ‘live’ PayPal API that can be used to test applications that integrate with PayPal, as well as to goof around. It’s free to use for anybody that has an active PayPal account. You can find all documentation on the API here.

Retrieving an Oauth2 access token
The first step – after creating the necessary test accounts in the sandbox environment – is to construct a call in REST Assured that retrieves an OAuth2 authentication token from the PayPal web service. This request uses basic authentication and looks like this:

@BeforeSuite
public void requestToken(ITestContext context) {

	String response =
			given().
				parameters("grant_type","client_credentials").
				auth().
				preemptive().
				basic("client_id","secret").
			when().
				post("https://api.sandbox.paypal.com/v1/oauth2/token").
				asString();
}

The actual values for client_id and secret are specific to the PayPal sandbox account. Note that we have stored the JSON response as a string. This makes it easier to parse it, as we will see in a moment. The response to this request contains our OAuth2 authentication token:

Our OAuth2 access token

In order to store this token for use in our actual tests, we need to extract it from the response and store it in the TestNG ITestContext:

JsonPath jsonPath = new JsonPath(response);

String accessToken = jsonPath.getString("access_token");
		
context.setAttribute("accessToken", accessToken);

System.out.println("Access token: " + context.getAttribute("accessToken"));

The System.out.println output shows us we have successfully stored the OAuth2 access token in the ITestContext:

Access token has been stored in the ITestContext

Using the OAuth2 access token in your tests
Next, we want to use the previously stored token in subsequent API calls that require OAuth2 authentication. This is fairly straightforward: see for example this test that verifies that no payments have been made for the current test account:

@Test
public void checkNumberOfAssociatedPaymentsIsEqualToZero(ITestContext context) {

	given().
		contentType("application/json").
		auth().
		oauth2(context.getAttribute("accessToken").toString()).
	when().
		get("https://api.sandbox.paypal.com/v1/payments/payment/").
	then().
		assertThat().
		body("count", equalTo(0));
}

Note the use of context.getAttribute() to retrieve the token from the ITestContext. This test passes, which not only tells us that no payments have yet been made by this account, but also that our authentication worked as expected (otherwise, we would have received an authentication error).

Download an example project
The Maven project containing all code from this post can be downloaded here.

API testing best practices

This is the second post in a three-part series on API testing. The first post, which can be found here, provided a brief introduction on APIs, API testing and its relevance to the testing world. This post will feature some best practices for everybody involved in API testing. The third and final post will contain some useful code example for those of you looking to build your own automated API testing framework.

As was mentioned in the first post in this mini-series, API test execution differs from user interface-based testing since APIs are designed for communication between systems or system components rather than between a system or system component and a human being. This introduces some challenges to testing APIs, which I will try to tackle here.

API communication
Whereas a lot of testing on the user interface level is still done by hand (and rightfully so), this is impossible for API testing; you need a tool to communicate with APIs. There are a lot of tools available on the market. Some of the best known tools that are specifically targeted towards API testing are:

I have extensive experience with SOAtest and limited experience with SoapUI and can vouch for their usefulness in API testing.

Structuring tests
An API usually consists of several methods or operations that can be tested individually as well as through the setup of test scenarios. These test scenarios are usually constructed by stringing together multiple API calls. I suggest a three step approach to testing any API:

  1. Perform syntax testing of individual methods or operations
  2. Perform functional testing of individual methods or operations
  3. Construct and execute test scenarios

Syntax testing
This type of testing is performed to check whether the method or operation accepts correct input and rejects incorrect input. For example, syntax testing determines whether:

  • Leaving mandatory fields empty results in an error
  • Optional fields are accepted as expected
  • Filling fields with incorrect data types (for example, putting a text value into an integer field) results in an error

Functional testing of individual operations or methods
This type of testing is performed to check whether the method or operations performs its intended action correctly. For example:

  • Is calculation X performed correctly when calling operation / method Y with parameters A, B and C?
  • Is data stored correctly for future use when calling a setter method?
  • Does calling a getter method retrieve the correct information?

Test scenarios
Finally, when individual methods or operations have been tested successfully, method calls can be strung together to emulate business processes, For example:
API test scenarios
You see that this approach is not unlike user interface-based testing, where you first test individual components for their correct behaviour before executing end-to-end test scenarios.

API virtualization
When testing systems of interconnected components, the availability of some of the components required for testing might be limited at the time of testing (or they might not be available at all). Reasons for limited availability of a component might be:

  • The component itself is not yet developed
  • The component features insufficient or otherwise unusable test data
  • The component is shared with other teams and therefore cannot be freely used

In any of these cases, virtualization of the API can be a valuable solution, enabling testing to continue as planned. Several levels of API virtualization exist:

  • Mocking – This is normally done for code objects using a framework such as Mockito
  • Stubbing – this is used to create a simple emulation of an API, mostly used for SOAP and REST web services
  • Virtualization – This is the most advanced technique of the three, enabling the simulation of behaviour of complex components, including back-end database connectivity and transport protocols other than HTTP

Non-functional testing
As with all software components, APIs can (and should!) be tested for characteristics other than functionality. Some of the most important nonfunctional API test types that should at least be considered are:

  • Security testing – is the API accessible to those who are allowed to use it and inaccessible to those without the correct permissions?
  • Performance – Especially for web services: are the response times acceptable, even under a high load?
  • Interoperability and connectivity – can be API be consumed in the agreed manner and does it connect to other components as expected?

Most of the high-end API testing tools offer solutions for execution of these (and many other types of) nonfunctional test types.

More useful API testing best practices can again be found in the API Testing Dojo.

Do you have any additional API testing best practices you would like to share with the world?

API testing skills: why you need them

This is the first post in a three-part series on API testing. This post is an introduction on APIs, API testing and its relevance to the testing world. The second part of this series will feature some best practices for everybody involved in API testing. The third and final post will contain some useful code example for those of you looking to build your own automated API testing framework.

As information systems become more and more distributed and systems and devices become ever more interconnected, the use of APIs has seen exponential growth in the past couple of years. Where traditional (or old-fashioned) computer systems were monolithic in nature, nowadays they are often made up of reusable components that communicate and exchange information with one another through various APIs.

The figure below depicts the growth in number of publicly accessible APIs, as published by ProgrammableWeb:
Growth in publicly accessible APIs in recent years

Estimates for the number of publicly available APIs for the coming years range from 300.000 in 2016 to around a million by 2017.

With APIs becoming more and more common and important, proper testing of these APIs has become a hot issue with both API providers and consumers. As a provider, you wouldn’t want to be associated with an API of poor quality. As a consumer, you wouldn’t want your software system and/or your business to rely on a buggy API.

However, as APIs are designed for computer-to-computer interaction, rather than computer-to-user interaction, they do not have a user interface through which the tester can access the API. Moreover, to properly assess whether the output as given by the API is correct, a tester would need to know at least something about the internal workings of the API (i.e., perform white-box testing rather than traditional black-box testing). This might make API testing seem ‘hard’ or ‘difficult’ for some testers.

What makes API testing even more important is that in the current wave of layered information systems, business rules and business logic is often coded and enforced within the API layer (and not in the user interface or the database layer, for example). This is yet another reason for every development project that features the development or consumption of APIs to pay sufficient attention to API testing.

In the next part of this series, I will present some pointers for those of you who are involved in API testing, are looking to do so, or have been asked to do so. Many of my tips are also featured on the API Testing Dojo. There, you can also find some exercises to test and sharpen your API testing skills. A highly recommended resource.