How (not) to test RESTful APIs with Selenium WebDriver

I’ve seen the question of how to do RESTful API testing with Selenium WebDriver come up a lot in various places. I’ve seen it mainly on LinkedIn, but a couple of weeks ago I was asked this question in person as well. So, as your ever helpful consultant, I thought I’d be a good idea to show you how it’s done.

First, let’s create a new browser driver object. For the sake of simplicity, I’ll use Firefox, because it doesn’t require setting up a separate driver. And yes, I know implicit waits aren’t the best waiting strategy, but I didn’t feel like writing an ExpectedCondition.

driver = new FirefoxDriver();
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);

Now, instead of browsing to a web page, we’ll simply browse to the RESTful API endpoint for which we want to check the response. I’ll use as an example here to look up some data for the US zip code 90210:


This gives us the following output:

Response from the API in JSON format

Let’s grab the text we see on screen and convert it to a JSON object (using the org.json package):

WebElement element = driver.findElement(By.xpath("//pre"));
JSONObject jsonObject = new JSONObject(element.getText());

Now that we have a JSON object containing the API response, we can extract specific elements and check their value:

String valueToCheck = jsonObject.get("country").toString();
Assert.assertEquals(valueToCheck, "United States");

As a final step, throw everything away, never do this again and please forget everything you’ve seen so far in this post. Selenium is not an API testing tool. It has never been, and it will never be. So please don’t try and force it to be.

Please stick to simulating UI interaction when using Selenium and use a dedicated, fit for purpose tool for API testing. As an example, REST Assured can perform the exact same check as the above. With a single line of code. Which is far better readable to boot:

public void doRestTestProperly() {
		body("country", equalTo("United States"));

So, here’s to hoping I did my part in exterminating questions that feature ‘Selenium’ and ‘API testing’ in the same sentence.

Review: Specification by Example workshop with Gojko Adzic

For a consultant wanting to stay on top of their game (or at the very least hireable) it’s a good idea to take a step back from the daily work of consulting and writing every once in a while and get inspired by a conference or a course or workshop. When the WannaFlex team announced they invited Gojko Adzic to bring his Specification by Example workshop to Amsterdam, it didn’t take me long to decide to sign up. I have come across the approach in previous projects, but found myself lacking experience and background information, and this workshop looked to be an excellent opportunity to get some of both.

What is Specification by Example?
In the booklet we received as a handout, Specification by Example (SbE) is defined as:

A set of process patterns that facilitate change in software products to ensure the right product is delivered efficiently

In my own words, SbE is a set of techniques that help software development teams to reach a shared understanding of the software about to be built, in order to eliminate (or at least greatly reduce) the possibility of mismatched expectations leading to incorrect software or software that will not be accepted and used to satisfaction by the end user.

Organization and setup
The SbE workshop is a two-day workshop where theory and practice around the creation and validation of software specifications take turns. First some theory, then a practical exercise, mostly conducted in small groups of four or five people. This group size is large enough to not come to an agreement on specifications too soon, but small enough to avoid the risk of endless discussions.

Specification by Example workshop announcement

Day one: creating specifications
The first day of the workshop focused mainly on the importance of creating clear specifications and how to get to that point. For example, we were given a set of ‘traditional’ requirements for a piece of software and 20 minutes to select a subset of the requirements and a pen and paper implementation. In other words, we simulated what could be sprint 1 of a development process, based on requirements that turned out to be vague, ambiguous and often plain incorrect. This is a learning technique that I have quickly come to appreciate now that I’ve experienced it first-hand: tell people just enough to get them started, let them make mistakes and discuss these afterwards. This is far more effective than simply showing and telling how something needs to be done!

We were also introduced to the process of how to get to good (or at least significantly better) specifications. In short (for the full story I highly recommend you take the workshop yourself if given the opportunity):

  1. Come up with examples of what the system should do. Any example will do as long as its valid, since every example tells you something about the desired behavior of the system you’re writing specifications for.
  2. Try and detect the pattern that’s behind the examples. What are the input parameters? What is the output? And so on.. This will help you see the bigger picture and get it clear what the system is supposed to do.
  3. Work on the boundary cases. This is where the interesting stuff happens, where questions mostly arise and where defects occur most often. Therefore, it’s a good idea to give them enough attention in the specification process.

Day two: application
Day two started with ways to apply the specifications in practice. Patterns such as Given / When / Then were introduced and the link to test automation was made. As a test automation consultant, this was especially valuable to me. I’ve seen and worked with most of the tools that were discussed (such as Cucumber, SpecFlow and FitNesse), but the link to SbE theory was very useful information. We also spent some time analysing a number of specifications from real-world projects and seeing where they could be improved (if they weren’t beyond saving..). Very useful as well.

Unfortunately, due to other commitments, I couldn’t stay until the end of the workshop so I don’t know exactly what has been discussed in the afternoon of the second day. Still, the first day and a half provided me with lots of useful information and tips on how to apply SbE in practice and how to combine this with test automation in my projects.

Lessons learned and recommendation
Too many to mention here! I’ve learned a lot in this workshop, both with regards to SbE as a technique as well as on how to facilitate a workshop. If you’re faced with SbE in your project or organization, or if you just want to know more about the technique and how it can help you to create better specifications and therefore better software, I highly recommend attending the SbE workshop. And if you like to learn from the best, you simply can’t go wrong with Gojko.

Thanks too to the people at Wannaflex for making this a very smooth and pleasant workshop.

No post this week

The last week and a half I’ve been very busy in both my professional and my private life. I hope to write about the outcomes of the former in separate blog posts soon enough. Also, I’m changing projects again, which takes its time and energy too. Unfortunately, as a result, there won’t be a blog post from me this week. I’ll be back next Wednesday, promised.

In the meantime, please check out the links in my blog roll on the right. They are fantastic sources that are updated weekly by people working very hard to gather everything interesting there is to read in the testing and test automation space.

See you next week.

— Bas