On handling processing time in your integration tests with Awaitility, and on using the Gmail API

For someone that writes and talks about REST Assured a lot, it has taken me a lot of time to find an opportunity to use it in an actual client project. Thankfully, my current project finally gives me the opportunity to use it on a scale broader than the examples and exercises from my workshop. Being able to do so has made me realize that some of the concepts and features I teach in the workshop deserve a more prominent spot, while others can be taught later on, or more briefly.

But that’s not what I wanted to talk about today. No, now that I actually use REST Assured for integration testing, as in: the integration between a system and its adjacent components, there’s something I have to deal with that I haven’t yet had to tackle before: processing time, or the time it takes for a message triggered using the REST API of system A to reach and be processed by system B, and being able to verify its result through consuming the REST API of system B.

Unlike writing tests for a single API, which is how I have been using and demonstrating REST Assured until now, I need a mechanism that helps me wait exactly long enough for the processing to be finished. Similar to user interface-driven automation, I could theoretically solve this by using calls to Thread.sleep(), but that’s ugly and vile and… just no.

Instead, I needed a mechanism that allowed me to poll an API until its response indicated a certain state change had occurred before performing the checks I needed to perform. In this case, and this is the example I’ll use in the remainder of this blog post, I invoked the REST API of system A to trigger a password reset for a given user account, and wanted to check if that resulted in a ‘password reset’ email message arriving in system B, system B being a specific Gmail inbox here.

Triggering the password reset
Triggering the password reset is done by means of a simple API call, which I perform (as expected) using REST Assured:

given().
    spec(identityApiRequestSpecification)
and().
    body(new PasswordForgottenRequestBody()).
when().
    post("/passwordreset").
then().
    assertThat().
    statusCode(204);

Waiting until the password reset email arrives
As stated above, I could just include a fixed waiting period of, say, 10 seconds before checking Gmail and seeing whether the email message arrived as expected. But again, Thread.sleep() is evil and dirty and… and should be avoided at all times. No, I wanted a better approach. Preferably one that didn’t result in unreadable code, both because I use my code in demos and I’d like to spend as little time as possible explaining my tests to others, and therefore want to keep it as readable as possible. Looking for a suitable library (why reinvent the wheel.. ) I was pointed to a solution that was created by Johan Haleby (not coincidentally also the creator of REST Assured), called Awaitility. From the website:

Awaitility is a DSL that allows you to express expectations of an asynchronous system in a concise and easy to read manner.

I’m not going to write about all of the features provided by Awaitility here (the usage guide does that way better than I ever could), but to demonstrate its expression power, here’s how I used it in my test:

await().
    atMost(10, TimeUnit.SECONDS).
with().
    pollInterval(1, TimeUnit.SECONDS).
    until(() -> this.getNumberOfEmails() == 1);

This does exactly what it says on the tin: it executes a method called getNumberOfEmails() once per second for a duration of 10 seconds, until the result returned by that method equals 1 (in which case my test execution continues) or until the 10 second timeout period has been exceeded, resulting in an exception being thrown. All with a single line of readable code. That’s how powerful it is.

In this example, the getNumberOfEmails() is a method that retrieves the contents for a specific Gmail mailbox and returns the number of messages in it. Before the test starts, I empty the mailbox completely to make sure that no old messages remain there and cause false positives in my test. Here’s how it looks:

private int getNumberOfEmails() {

    return given().
        spec(gmailApiRequestSpec).
    when().
        get("/messages").
    then().
        extract().
        path("resultSizeEstimate");
}

This method retrieves the number of emails in a Gmail inbox (the required OAuth2 authentication details, base URL and base path are specified in the gmailApiRequestSpec RequestSpecification) by means of a GET call to /messages and extracting and returning the value of the resultSizeEstimate field of the JSON response returned by the API. If you want to know more about the Gmail API, its documentation can be found here, by the way.

Checking the content of the password reset message
So, now that we know that an email message has arrived in the Gmail inbox, all that’s left for us to do is check whether it is a password reset message and not any other type of email message that might have arrived during the execution of our test. All we need to do is to once more retrieve the contents of the mailbox, extract the message ID of the one email message in it, use that to retrieve the details for that message and check whatever we want to check (in this case, whether it has landed in the inbox and whether the subject line has the correct value):

String messageID =
    given().
        spec(gmailApiRequestSpec).
    when().
        get("/messages").
    then().
        assertThat().
        body("resultSizeEstimate", equalTo(1)).
    and().
        extract().
        path("messages.id[0]");

    // Retrieve email and check its contents
    given().
        spec(gmailApiRequestSpec).
    and().
        pathParam("messageID", messageID).
    when().
        get("/messages/{messageID}").
    then().
        assertThat().
        body("labelIds",hasItem("INBOX")).
    and().
        body("payload.headers.find{it.name=='Subject'}.value",equalTo("Password reset"));

Gmail authentication
It took me a little while to figure out how to consume the Gmail API. In the end, this proved to be quite simple but I spent a couple of hours fiddling with OAuth2, authentication codes, OAuth2 access and refresh tokens and the way Google has implemented this. Describing how all this goes is beyond the scope of this blog post, but you can find instructions here. Once you’ve obtained a refresh token, store it, because that’s the code you can use to generate a new access token through the API, without having to deal with the pesky authentication user interface. For those of you more experienced with OAuth2, this might sound obvious, but it took me a while to figure it out. Still, it’s far better than writing automation against the Gmail user interface, though (seriously, DON’T DO THAT).

So, to wrap things up, there are two lessons here.

One, if you’re looking for a library that helps you deal with processing times in integration tests in a flexible and readable manner, and you’re writing your tests in Java, I highly recommend taking a look at Awaitility. I’ve only recently discovered it but I’m sure this one won’t leave my tool belt anytime soon.

Two, if you want to include checking email into your integration or possibly even your end-to-end tests, skip the user interface and go the API route instead. Alternatively, you could try an approach like Angie Jones presents in a recent blog post, leveraging the JavaMail API, instead. Did I say don’t use the Gmail (or Outlook, or Yahoo, or whatever) user interface?

Using JsonPath and XmlPath in REST Assured

While preparing my REST Assured workshop for the Romanian Testing Conference next month, I ran into a subject I feel I didn’t cover enough in the previous times I hosted the workshop: how to effectively use JsonPath and XmlPath to extract specific elements and element groups in RESTful API responses before verifying them. Here are a couple of tricks I learned since and worked into the exercises that’ll be part of the workshop from now on.

The examples in this post are all based on the following XML response:

<?xml version="1.0" encoding="UTF-8" ?>
<cars>
	<car make="Alfa Romeo" model="Giulia">
		<country>Italy</country>
		<year>2016</year>
	</car>
	<car make="Aston Martin" model="DB11">
		<country>UK</country>
		<year>1949</year>
	</car>
	<car make="Toyota" model="Auris">
		<country>Japan</country>
		<year>2012</year>
	</car>
</cars>

The syntax for JsonPath is very, very similar, except for the obvious lack of support for attributes in JsonPath (JSON does not have attributes).

Extracting a single element based on its index
Let’s get started with an easy example. Say I want to check that the first car in the list is made in Italy. To do this, we can simply traverse the XML tree until we get to the right element, using the index [0] to select the first car in the list:

@Test
public void checkCountryForFirstCar() {
						
	given().
	when().
		get("http://path.to/cars").
	then().
		assertThat().
		body("cars.car[0].country", equalTo("Italy"));
}

Similarly, we can check that the last car came on the market in 2012, using the [-1] index (this points us to the last item in a list):

@Test
public void checkYearForLastCar() {
						
	given().
	when().
		get("http://path.to/cars").
	then().
		assertThat().
		body("cars.car[-1].year", equalTo("2012"));
}

Extracting an attribute value
Just as easily, you can extract and check the value of an attribute in an XML document. If we want to check that the model of the second car in the list is ‘DB11’, we can do so using the ‘@’ notation:

@Test
public void checkModelForSecondCar() {
						
	given().
	when().
		get("http://path.to/cars").
	then().
		assertThat().
		body("cars.car[1].@model", equalTo("DB11"));
}

Counting the number of occurrences of a specific value
Now for something a little more complex: let’s assume we want to check that there’s only one car in the list that is made in Japan. To do this, we’ll need to apply a findAll filter to the country element, and subsequently count the number of items in the list using size():

@Test
public void checkThereIsOneJapaneseCar() {
		
	given().
	when().
		get("http://path.to/cars").
	then().
		assertThat().
		body("cars.car.findAll{it.country=='Japan'}.size()", equalTo(1));
}

Likewise, we can also check that there are two cars that are made either in Italy or in the UK, using the in operator:

@Test
public void checkThereAreTwoCarsThatAreMadeEitherInItalyOrInTheUK() {
		
	given().
	when().
		get("http://path.to/cars").
	then().
		assertThat().
		body("cars.car.findAll{it.country in ['Italy','UK']}.size()", equalTo(2));
}

Performing a search for a specific string of characters
Finally, instead of looking for exact attribute or element value matches, we can also filter on substrings. This is done using the grep() method (very similar to the Unix command). If we want to check the number of cars in the list whose make starts with an ‘A’, we can do so like this:

@Test
public void checkThereAreTwoCarsWhoseMakeStartsWithAnA() {
		
	given().
	when().
		get("http://localhost:9876/xml/cars").
	then().
		assertThat().
		body("cars.car.@make.grep(~/A.*/).size()", equalTo(2));
}

If you know of more examples, or if I missed another example of how to use JsonPath / XmlPath, do let me know!

Open sourcing my workshop on WireMock

For those of you that want to jump to the good stuff directly, you can find the workshop slides, exercises and everything related here.

A couple of weeks ago I was given the opportunity to deliver another workshop before TestNet, the Dutch software testing community, as part of their fall conference. Half a year ago, I did a similar workshop for their spring conference. That workshop was on RESTful API testing using REST Assured, which I decided to make open source a little later on. I’ve received some positive feedback on that, so why not do it again?

This time, the conference was centered around test automation. The subject of my workshop this time was closely related to the main theme: stubbing test environment dependencies using WireMock.

The title slide for my workshop

Delivery
As with most workshops that day, mine was set in a classroom-style space. I had somewhere between 15 and 20 participants, which I think is pretty much the ideal group size for a hands-on workshop that involves writing code. Not so many that I can’t give proper attention to all questions asked, but not a group so small you start to doubt whether the upfront investment has been worth it. As those of you who have prepared and delivered workshops before, you know that preparing them takes a lot of time. Spending all those hours and then only having two people turn up, one of whom is a coworker and the other one seems a little lost, is a bit of a bummer. Fortunately, this wasn’t the case for me, at least not this time..

The levels of experience of the participants (semi-pro tip: know your audience) ranged from people having some prior experience with stub and mock development to people that had never in their life written a single line of code before. An interesting mix, to say the least!

Workshop contents
As said, the main subject of the workshop was WireMock. I started out by telling people a little about the difficulties with regards to keeping test environments up and running and properly configured in these times of parallel development, Continuous Delivery and Testing and distributed applications. I then introduced WireMock, which I’ve done here on this site before as well. Then came four cycles of me presenting a WireMock feature, followed by exercises where the participants could try this feature out for themselves. I chose to highlight the following WireMock features in the workshop:

  • Writing a first, basic stub
  • Request matching options
  • Fault simulation
  • Creating stateful mocks

For each set of exercises, I prepared REST Assured tests that the participants could run to see if their stub implementation was correct. Call it test driven stub development if you like. This approach worked remarkably well, it definitely saved me a lot of time answering questions in the ‘is this correct?’ vein. Test green = stub good, test red = stub needs improvement. It doesn’t get any clearer than that.

Here’s an example of an exercise and the test that determines the correctness of the solution:

public void setupStubExercise101() {

	/************************************************
	 * Create a stub that listens at path
	 * /exercise101
	 * and responds to all GET requests with HTTP status code 200
	 ************************************************/
}

@Test
public void testExercise101() {
        
    wme.setupStubExercise101();
	         
    given().
    when().
        get("http://localhost:9876/exercise101").
    then().
        assertThat().
        statusCode(200);
}

How it turned out
Fine, I think. Time flew by and I didn’t experience any major faults, missing or incorrect slides or things like that. The experience I’m slowly gathering by doing this more often is starting to pay off, I think. I received almost exclusively positive feedback on the workshop, so I’m a happy guy. Also, everybody seemed to have learned at least something new and enjoyed the process too, no matter whether they had prior stubbing or even programming experience or not, and that has been the most important result of the morning to me. I’m looking forward to the next opportunity for delivering this workshop.

Having said that…
All workshop contents, that’s:

  • The complete set of slides
  • All workshop exercises and their answers
  • Matching REST Assured tests to verify the stubs created

can be found on my GitHub page. As with the previous workshop I’ve published in this manner, feel free to download, adapt, extend and then deliver the workshop to your liking. I look forward to hearing your experiences.

And in case you’re interested in following a WireMock workshop, but do not want to deliver this yourself, don’t hesitate to contact me. I’ll be happy to discuss options. Also, this workshop can easily be combined with a workshop on REST Assured for a full day of API testing and stubbing goodness.