Choose wisely

In a recent article that was published on TechBeacon, I argued that writing tests at the API level often hits the sweet spot between speed of execution, stability (both in terms of execution and maintenance required) and test coverage. What I didn’t write about in this article is what motivated me to write the article in the first place, so I thought it might be a good idea to dedicate a blog post to the reason behind the piece.

There really was only a single reason for me to suggest the topic to the people at TechBeacon: because I see things go wrong too often when people start creating automated tests. I’m currently working with a number of clients in two separate projects, and what a lot of them seem to have in common is that they revert instantly to end-to-end tests (often using a tool like Selenium or Protractor) to create the checks that go beyond the scope of unit tests.

As an example, I’m working on a project where we are going to create automated checks for a web shop that sells electronic cigarettes and related accessories in the United States. There are several product categories involved, several customer age groups to be considered (some products can be purchased if you’re over 18, some over 21, some are fit for all ages, etc.), and, this being the US, fifty different states, each with their own rules and regulations. In short, there’s a massive amount of possible combinations (I didn’t do the math yet, but it’s easily in the hundreds). Also, due to the strict US regulations, and more importantly the fines associated with violating these rules, they want all relevant combinations included in the automated test.

Fair enough, but the problem started when they suggested we write an automated end-to-end test case for each of the possible combinations. That means creating an order for every combination of product group, age group and state, and every order involves filling out three or four separate forms and some additional more straightforward web page navigation. In other words, this would result in a test that would be slow to execute (we’re talking hours here) and possibly quite hard to maintain as well.

Instead, I used Fiddler to analyze what exactly it was that the web application did in order to determine if a customer could order a given product. Lo and behold.. it simply called an API that exposed the business logic used to make this decision. So, instead of creating hundreds of UI-driven test cases, I suggested to create API-level tests that would verify the business logic configuration, and add a couple of end-to-end tests to verify that a user can indeed place an order successfully, as well as receive an error message in case he or she tries to order a product that’s not available for a specific reason.

We’re still working on this, but I think this case illustrates my point fairly well: it often pays off big time to look beyond the user interface when you’re creating automated tests for web applications:

  • Only use end-to-end tests to verify whether a user of your web application can perform certain sequences of actions (such as ordering and paying for a product in your web shop).
  • See (ask!) whether business logic hidden behind the user interface can be accessed, and therefore tested, at a lower (API or unit) level, thereby increasing both stability and speed of execution.

For those of you familiar with the test automation pyramid, this might sound an awful lot like a stock example of the application of this model. And it is. However, in light of a couple of recent blog posts I read (this one from John Ferguson Smart being a prime example) I think it might not be such a good idea to relate everything to this pyramid anymore. Instead, I agree that what it comes down to (as John says) is to get clear WHAT it is that you’re trying to test and then write tests on the right level. If that leads to an ice cream cone, so be it. If only because I like ice cream..

This slightly off-topic remark about the test automation pyramid notwithstanding, I think the above case illustrates the key point I’m trying to get across fairly well. As I’ve said before, creating the most effective automated tests comes down to:

  • First, determining why you want to automate those tests in the first place. Although that’s not really the subject of this blog post, it IS the first question that should be asked. In the example in this post, the why is simple: because the risk and impact of fines imposed in case of the sale of items to groups of people that should not be allowed to is high enough to warrant thorough testing.
  • Then, deciding what to test. In this case, it’s the business logic that determines whether or not a customer is allowed to purchase a given item, based on state of residence, product ID and date of birth.
  • Finally, we get to the topic of this blog post, the question of how to test a specific application or component. In this case, the business logic that’s the subject of our tests is exposed at the API level, so it makes sense to write tests at that level too. I for one don’t feel like writing scenarios for hundreds of UI-level tests, let alone run, monitor and maintain them..

I’m sure there are a lot of situations in your own daily work where reconsidering the approach taken in your automated tests might prove to be beneficial. It doesn’t have to be a shift from UI to API (although that’s the situation I most often encounter), it could also be writing unit tests instead of end-to-end user interface-driven tests. Or maybe in some cases replacing a large number of irrelevant unit tests with a smaller number of more powerful API-level integration tests. Again, as John explained in his LinkedIn post, you’re not required to end up with an actual pyramid, as long as you end up with what’s right for your situation. That could be a pyramid. But it could also not be a pyramid. Choose (and automate) wisely.

(De)serializing POJOs in REST Assured

In this post, I’d like to demonstrate how you can leverage the ability of REST Assured to (de-)serialize Plain Old Java Objects (better known as POJOs) for more powerful testing.

As an example, we’ll use the following POJO representing a car and some of its properties:

public class Car {
	
	String make;
	String model;
	int year;
	
	public Car() {		
	}
	
	public Car(String make, String model, int year) {
		
		this.make = make;
		this.model = model;
		this.year = year;
	}
	
	public String getMake() {
		return this.make;
	}

	public void setMake(String make) {
		this.make = make;
	}
	
	public String toString() {
		return "My car is a " + this.year + " " + this.make + " " + this.model;
	}
}

Please note that I’ve removed the getters and setters for the other properties for brevity. Now, let’s create an instance of the car:

Car myCar = new Car("Aston Martin","DB9",2004);

Say we want to transmit the information stored in this object instance to a RESTful API, without having to map each individual property of our car to a corresponding field in the request. REST Assured supports this by allowing you to serialize the myCar object instance as follows:

@Test
public void testCarSerialization() {
				
	given().
		contentType("application/json").
		body(myCar).
	when().
		post("http://localhost:9876/carstub").
	then().
		assertThat().
		body(equalTo("Car has been stored"));
}

So, all you have to do is pass the object using body(). REST Assured will automatically translate this to the following request body:

{
    "make": "Aston Martin",
    "model": "DB9",
    "year": 2004
}

Neat, right? In this example, we serialized the car object to a JSON request, but REST Assured also allows you to serialize it to XML or HTML. Additionally, you can create custom object mappings as well. See this page in the REST Assured documentation for more information.

REST Assured also supports deserialization, meaning that we can easily transform a suitably formatted API response to a POJO instance:

@Test
public void testCarDeserialization() {
		
	Car myDeserializedCar = get("http://localhost:9876/carstub").as(Car.class);
		
	System.out.println(myDeserializedCar.toString());
	
	Assert.assertEquals("Check the car make", myDeserializedCar.getMake(), "Aston Martin");		
}

Note that http://localhost:9876/carstub points to a WireMock stub I’ve created to illustrate this example. The fact that our assertion is passing and that the console shows the following output when running the test tells us that deserialization has been successful:

My car is a 2004 Aston Martin DB9

You can download a Maven project containing all of the code I’ve used in the examples in this blog post here.

How (not) to test RESTful APIs with Selenium WebDriver

I’ve seen the question of how to do RESTful API testing with Selenium WebDriver come up a lot in various places. I’ve seen it mainly on LinkedIn, but a couple of weeks ago I was asked this question in person as well. So, as your ever helpful consultant, I thought I’d be a good idea to show you how it’s done.

First, let’s create a new browser driver object. For the sake of simplicity, I’ll use Firefox, because it doesn’t require setting up a separate driver. And yes, I know implicit waits aren’t the best waiting strategy, but I didn’t feel like writing an ExpectedCondition.

driver = new FirefoxDriver();
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);

Now, instead of browsing to a web page, we’ll simply browse to the RESTful API endpoint for which we want to check the response. I’ll use api.zippopotamus.us as an example here to look up some data for the US zip code 90210:

driver.get("http://api.zippopotam.us/us/90210");

This gives us the following output:

Response from the zippopotam.us API in JSON format

Let’s grab the text we see on screen and convert it to a JSON object (using the org.json package):

WebElement element = driver.findElement(By.xpath("//pre"));
JSONObject jsonObject = new JSONObject(element.getText());

Now that we have a JSON object containing the API response, we can extract specific elements and check their value:

String valueToCheck = jsonObject.get("country").toString();
Assert.assertEquals(valueToCheck, "United States");

As a final step, throw everything away, never do this again and please forget everything you’ve seen so far in this post. Selenium is not an API testing tool. It has never been, and it will never be. So please don’t try and force it to be.

Please stick to simulating UI interaction when using Selenium and use a dedicated, fit for purpose tool for API testing. As an example, REST Assured can perform the exact same check as the above. With a single line of code. Which is far better readable to boot:

@Test
public void doRestTestProperly() {
		
	given().
	when().
		get("http://api.zippopotam.us/us/90210").
	then().
		assertThat().
		body("country", equalTo("United States"));
}

So, here’s to hoping I did my part in exterminating questions that feature ‘Selenium’ and ‘API testing’ in the same sentence.