Three reasons to start improving your API test automation skills

Modern applications and software development methods have changed the requirements for testers and the skills they need to possess to add real value to their clients and projects. One of these emerging and sought after skills is the ability to design and execute automated tests for APIs. In this post, I will give you three reasons why it might be useful for you to start improving your API test automation skills.

API word cloud

APIs are everywhere
The first reason why you should invest in your API test automation skills is a simple question of demand and supply: APIs are becoming ever more present in current IT solutions. From mobile applications to the Internet of Things, many modern systems and applications expose their data, their business logic or both through APIs. Whether you’re building an application that uses APIs to expose data or logic to the outside world, or you’re on the other side as an API consumer, you need to be able to perform tests on APIs. Otherwise, how are you going to ensure that an API and its integration with the outside world function as expected?

Oh, and if you’re testing an application that consumes a third-party API, please don’t fall into the trap of assuming that this API you’re using works perfectly as designed, or that integration with your own application will be seamless. Can anyone really assure you that that business-critical third party API you’re relying on has been tested for your specific situation and requirements? Thought not. Make sure it does and do some proper testing on it!

API test automation hits the sweet spot between speed and coverage
The second reason why API test automation can be very useful is that automated checks on the API level hit the sweet spot between speed of execution and coverage of application features. Compared to the two other types of automated tests in the test automation pyramid, API-level tests tend to:

  • execute faster than user interface-driven tests. User interface-driven automated tests, such as those written in Selenium WebDriver, need to fire up a browser and render several web pages every time a test is executed. When your tests go through a lot of different pages, execution time skyrockets. API-level tests, on the other hand, have to wait for a server responding to HTTP calls only. The only client-side processing that needs to be done is parsing the response and performing validation checks, for example on specific elements in the response. This is a lot faster than sending (possibly many) HTTP requests to a web server to fetch all objects required for a web page and then waiting until your browser has finished rendering the page.
  • cover more business logic than unit tests. Yes, you should have unit tests. And they should cover as much of the internal workings of your application that make up the business logic. However, there’s only so much unit tests can do. For example, unit tests can check whether the salary of a given employee is calculated correctly in the back end. They cannot guarantee that the same salary is correctly sent out to the front end layer of your application (or to the IRS) upon request, though. For this, you will need to perform tests at a higher level in your application, and API tests are usually perfect for that.

Automated API tests tend to be more reliable
Apart from having the right mixture of speed of execution and coverage of functional aspects, API-level automated checks have another big advantage over user interface-driven automated tests: they’re usually far more reliable. User interface-driven tests constantly have to walk the wobbly rope of synchronization, ever changing (or ‘improving’) user interface designs, dynamic element identification methods, etc. API definitions and interfaces on the other hand are amongst the most stable parts of an application: they follow standardized specification formats (such as WSDL for SOAP or WADL, RAML or Swagger for REST) and once agreed upon, an API does not usually change all that much. This applies especially to outward-facing APIs. For example, Google can easily change its Gmail user interface without this impacting end users (apart from annoyance, maybe, but that’s a different story altogether), but sudden radical changes to its Gmail API would render a lot of third-party applications that have Gmail integration useless. Therefore, changes to an API will usually be a lot fewer and further between, resulting in less maintenance required for your API-level automated tests.

A sample API test in REST Assured

Further reading
I hope this blog post has given you some insight into why I think API test automation skills are a valuable asset for any tester with an interest in test automation. If you want to read more on API testing and test automation, I highly recommend the API Testing Dojo on the SoapUI website. Additionally, you can also check out my other posts on API testing here.

Using the TestNG ITestContext to create smarter REST Assured tests

In this post, I would like to demonstrate two different concepts that I think work very well together:

  • How to store and retrieve data objects using the TestNG ITestContext for better code maintainability
  • How to communicate with RESTful web services that use basic or OAuth2 authorization using REST Assured

Using the PayPal sandbox API as an example, I will show you how you can create readable and maintainable tests for secured APIs using TestNG and REST Assured.

The TestNG ITestContext
If you have a suffficiently large test suite, chances are high that you want to be able to share objects between individual tests to make your tests shorter and easier to maintain. For example, if you are calling a web service multiple times throughout your test suite and that web service requires an authentication token in order to be able to consume it, you might want to request and store that authentication token in the setup phase of your test suite, then retrieve and use it in all subsequent tests where this web service is invoked. This is exactly the scenario we’ll see in this blog post.

TestNG offers a means of storing and retrieving objects between tests through the ITestContext interface. This interface allows you to store (using the inherited setAttribute() method) and retrieve (using getAttribute()) objects. Since the ITestContext is created once and remains active for the duration of your test run, this is the perfect way to implement object sharing in your test suite. Making the ITestContext available in your test methods is easy: just pass it as a parameter to your test method (we’ll see an example further down).

REST Assured authentication options
As you might have read in one of my previous blog posts, REST Assured is a Java library that allows you to write and execute readable tests for RESTful web services. Since we’re talking about secured APIs here, it’s good to know that REST Assured supports the following authentication mechanisms:

  • Basic
  • Digest
  • OAuth (version 1 and 2)
  • Form

In the examples in this post, we’ll take a closer look at both Basic authentication (for requesting an OAuth token) and OAuth2 authentication (for invoking secured web service operations) in REST Assured.

The PayPal sandbox API
To illustrate the concepts introduced above I chose to use the PayPal sandbox API. This is a sandbox version of the ‘live’ PayPal API that can be used to test applications that integrate with PayPal, as well as to goof around. It’s free to use for anybody that has an active PayPal account. You can find all documentation on the API here.

Retrieving an Oauth2 access token
The first step – after creating the necessary test accounts in the sandbox environment – is to construct a call in REST Assured that retrieves an OAuth2 authentication token from the PayPal web service. This request uses basic authentication and looks like this:

@BeforeSuite
public void requestToken(ITestContext context) {

	String response =
			given().
				parameters("grant_type","client_credentials").
				auth().
				preemptive().
				basic("client_id","secret").
			when().
				post("https://api.sandbox.paypal.com/v1/oauth2/token").
				asString();
}

The actual values for client_id and secret are specific to the PayPal sandbox account. Note that we have stored the JSON response as a string. This makes it easier to parse it, as we will see in a moment. The response to this request contains our OAuth2 authentication token:

Our OAuth2 access token

In order to store this token for use in our actual tests, we need to extract it from the response and store it in the TestNG ITestContext:

JsonPath jsonPath = new JsonPath(response);

String accessToken = jsonPath.getString("access_token");
		
context.setAttribute("accessToken", accessToken);

System.out.println("Access token: " + context.getAttribute("accessToken"));

The System.out.println output shows us we have successfully stored the OAuth2 access token in the ITestContext:

Access token has been stored in the ITestContext

Using the OAuth2 access token in your tests
Next, we want to use the previously stored token in subsequent API calls that require OAuth2 authentication. This is fairly straightforward: see for example this test that verifies that no payments have been made for the current test account:

@Test
public void checkNumberOfAssociatedPaymentsIsEqualToZero(ITestContext context) {

	given().
		contentType("application/json").
		auth().
		oauth2(context.getAttribute("accessToken").toString()).
	when().
		get("https://api.sandbox.paypal.com/v1/payments/payment/").
	then().
		assertThat().
		body("count", equalTo(0));
}

Note the use of context.getAttribute() to retrieve the token from the ITestContext. This test passes, which not only tells us that no payments have yet been made by this account, but also that our authentication worked as expected (otherwise, we would have received an authentication error).

Download an example project
The Maven project containing all code from this post can be downloaded here.

Creating mock RESTful APIs using Sandbox

While browsing through my Twitter feed a couple of days ago I saw someone mentioning Sandbox, a Software as a Service (SaaS) solution for quick creation and deployment of mock services for development and testing purposes. After starting to play around with it a bit, I was rather impressed by the ease with which one can create useful mocks from Swagger, RAML and WSDL API specifications.

As an example, I created a RAML API model for a simple API that shows information about test tools. Consumers of this API can create, read, update and delete entries in the list. The API contains six different operations:

  • Add a test tool to the list
  • Delete a test tool from the list
  • Get information about a specific test tool from the list
  • Update information for a specific test tool
  • Retrieve the complete list of test tools
  • Delete the complete list of test tools

You can view the complete RAML specification for this API here.

Creating a skeleton for the mock API in Sandbox is as easy as registering and then loading the RAML specification into Sandbox:
Loading the RAML specification in SandboxSandbox then generates a fully functioning API skeleton based on the RAML:
API operations from the RAMLSandbox also creates a routing routine for every operation:

var testtools = require("./routes/testtools.js")

/* Route definition styles:
 *
 *	define(path, method, function)
 *	soap(path, soapAction, function)
 *
 */
Sandbox.define("/testtools", "POST", testtools.postTesttools);
Sandbox.define("/testtools", "GET", testtools.getTesttools);
Sandbox.define("/testtools", "DELETE", testtools.deleteTesttools);
Sandbox.define("/testtools/{toolname}", "GET", testtools.getTesttools2);
Sandbox.define("/testtools/{toolname}", "PUT", testtools.putTesttools);
Sandbox.define("/testtools/{toolname}", "DELETE", testtools.deleteTesttools2);

and empty responses for every operation (these are the responses for the first two operations):

/*
 * POST /testtools
 *
 * Parameters (body params accessible on req.body for JSON, req.xmlDoc for XML):
 *
 */
exports.postTesttools = function(req, res) {
	res.status(200);

	// set response body and send
	res.send('');
};

/*
 * GET /testtools
 *
 * Parameters (named path params accessible on req.params and query params on req.query):
 *
 * q(type: string) - query parameter - Search phrase to look for test tools
 */
exports.getTesttools = function(req, res) {
	res.status(200);

	// set response body and send
	res.type('json');
	res.render('testtools_getTesttools');
};

Sandbox also creates template responses (these are rendered using res.render(‘testtools_getTesttools’) in the example above). These are mostly useful when dealing with either very large JSON responses or with XML responses, though, and as our example API has neither, we won’t use them here.

To show that the generated API skeleton is fully working, we can simply send a GET to the mock URL and verify that we get a response:
Testing the generated API skeletonNow that we’ve seen our API in action, it’s time to implement the operations to have the mock return more meaningful responses. We also want to add some state to be able to store new entries to our test tool list for later reference. For example, to add a test tool submitted using a POST to our list – after verifying that all parameters have been assigned a value – we use the following implemenation for the export.postTesttools method:

/*
 * POST /testtools
 *
 */
exports.postTesttools = function(req, res) {
    
    if (req.body.name === undefined) {
        return res.json(400, { status: "error", details: "missing tool name" });
    }
    
    if (req.body.description === undefined) {
        return res.json(400, { status: "error", details: "missing tool description" });
    }
    
    if (req.body.url === undefined) {
        return res.json(400, { status: "error", details: "missing tool website" });
    }
    
    if (req.body.opensource === undefined) {
        return res.json(400, { status: "error", details: "missing tool opensource indicator" });
    }

    // add tool to list of tools
    state.tools.push(req.body);
    
    return res.json(200, { status: "ok", details: req.body.name + " successfully added to list" });
};

Likewise, I’ve added meaningful implementations for all other methods. The complete code for our mock API implementation can be found here.

Finally, to prove that the mock API works as desired, I wrote a couple of quick tests using REST Assured. Here’s a couple of them:

// base URL for our Sandbox testtools API
static String baseUrl = "http://testtools.getsandbox.com/testtools";
	
// this is added to the URL to perform actions on a specific item in the list
static String testtoolParameter = "/awesometool";
	
// original JSON body for adding a test tool to the list 
static String testtoolJson = "{\"name\":\"awesometool\",\"description\":\"This is an awesome test tool.\",\"url\":\"http://awesometool.com\",\"opensource\":\"true\"}";
	
// JSON body used to update an existing test tool
static String updatedTesttoolJson = "{\"name\":\"awesometool\",\"description\":\"This is an awesome test tool.\",\"url\":\"http://awesometool.com\",\"opensource\":\"false\"}";

@BeforeMethod
public void clearList() {
		
	// clear the list of test tools
	delete(baseUrl);
		
	// add an initial test tool to the list
	given().
		contentType("application/json").
		body(testtoolJson).
	when().			
		post(baseUrl).
	then();
}

@Test
public static void testGetAll() {
		
	// verify that a test tool is in the list of test tools
	given().
	when().
		get(baseUrl).
	then().
		body("name", hasItem("awesometool"));
}

@Test
public static void testDeleteAll() {
		
	// verify that the list is empty after a HTTP DELETE		
	given().
	when().
		delete(baseUrl).
	then().
		statusCode(200);
	
	given().
	when().
		get(baseUrl).
	then().
		body(equalTo("[]"));
}

An Eclipse project containing all of the tests I’ve written (there really aren’t that many, by the way, but I can think of a lot more) can be downloaded here.

One final note: Sandbox is a commercial SaaS solution, and therefore requires you to fork over some money if you want to use it in a serious manner. For demonstration and learning purposes, however, their free plan works fine.

Overall, I’ve found Sandbox to be a great platform for rapidly creating useful mock API implementation, especially when you want to simulate RESTful services. I’m not sure whether it works as well when you’re working with XML services, because it seems a little more cmplicated to construct meaningful responses without creating just a bunch of prepared semi-fixed responses. Having said that, I’m pretty impressed with what Sandbox does and I’ll surely play around with it some more in the future.