New open source workshop: writing tests for REST APIs in Python with requests

Just a quick update to let you all know that I’ve just released the first version of a brand new open source workshop.

If you’re looking to learn how to write tests for RESTful APIs in Python using the requests library, head on over to my GitHub page to find a free and open source workshop on this very topic.

So far, it contains five series of examples, exercises and the corresponding answers for you to try out. As with all the other open source workshops, you’re absolutely free to use it in any way you want. Share it, teach it to others, discuss it with coworkers, whatever you like.

The only thing I’m asking you is to share your experiences with me. Is there anything missing? How did you use this workshop and what did you think?

Have fun and happy learning.

Do you really need that Cucumber with your Selenium?

Note: this blog post is NOT meant to discredit the value of Cucumber, SpecFlow and similar tools. Quite the contrary. I think these are fantastic tools, created and maintained by great people.

Somewhere last week I watched the recording of ‘Is Cucumber Automation Killing Your Project?‘, a SauceLabs webinar presented by Nikolay Advolodkin. In this webinar, Nikolay showed some interesting figures: 68% of the participants indicated that they don’t collaborate with others to create business specs in three amigos sessions. However, 54% of the participants said they used Cucumber.

That means that there’s a significant amount of participants that do use Cucumber without actively collaborating on the creation of specifications through practices like three amigos sessions, Specification by Example and Example Mapping. That’s not the strong point of a tool like Cucumber, though. These tools really shine when they’re used to support collaboration, as discussed in this blog post from Aslak Hellesøy, creator of and core contributor to the Cucumber project.

I must say that the above statistics don’t surprise me. Many clients that I work with use Cucumber (or SpecFlow) in the same way, including my current one. Their reasoning?

“We want everybody in our team to understand what we’re testing with our tests”

And for a long time, I supported this. I, too, thought that using Cucumber on top of your test automation code could be a good idea, even if you’re not practicing Behaviour Driven Development. I’ve even written an article on the Cucumber.io blog that says something to that extent. Yes, I’ve put in some pitfalls to avoid and things to consider, but I don’t think that blog post covers my current point of view well enough.

That’s where this blog post comes in. I’ve come to think that in a lot of projects where Cucumber is used solely as another layer in the automation stack, it does more harm than good. The only people that really read the Given-When-Then specifications are the people who create them (the automation engineers, most of the time), without regard for the additional time and effort it requires to implement and maintain this abstraction layer. There’s no discussion, no validation, no Example Mapping, just an automation engineer writing scenarios and implementing them, because readability.

That, though, is not the point of this blog post. What I do want to show here are a couple of techniques you can employ to make your test methods read (almost) like prose, without resorting to adding another abstraction layer like Cucumber.

Our application under test, once again, is ParaBank, the world’s least safe online bank (or rather, a demo web application from Parasoft. In this demo application, you can perform a variety of different scenarios related to online banking, such as opening a new checking or savings account.

With Cucumber, an example scenario that describes part of the behaviour of ParaBank around opening new accounts might look something like this:

Given John is an existing ParaBank customer
And he has an existing checking account with a balance of 5000 dollars
When he opens a new savings account
Then a confirmation message containing the new account number is shown

Not too bad, right? It’s readable, plain English, and (when you know that the initial balance is required for the deposit into the new savings account) describes the intended behaviour in a clear and unambiguous manner.

But here’s the thing: unless this specification has been conjured up before the software was written, by the three amigos, using techniques like Specification by Example and Example Mapping, you don’t need it. It’s perfectly possible to write test code that is nearly just as readable without the additional abstraction layer and dependency that a tool like Cucumber is.

I mean, if the automation engineer is the only person to read the specifications, why even bother creating them? This only presents a maintenance burden that a lot of projects could do without.

As an example, this is what the same test could look like without the Cucumber layer, but with some design decisions that are included for readability (an important aspect of test code, if you’d ask me) and which I’ll describe in more detail below:

private WebDriver driver;

@Before
public void initializeDatabaseAndLogin() {

    ApiHelpers.initializeDatabaseBeforeTest();

    driver = DriverHelpers.createADriverOfType(DriverType.CHROME);

    Credentials johnsCredentials = Credentials.builder().username("john").password("demo").build();

    new LoginPage(driver).
        load().
        loginUsing(johnsCredentials);
}

@Test
public void openAccount_withSufficientFunds_shouldSucceed() {

    Account aNewCheckingAccount =
        Account.builder().type(AccountType.CHECKING).build();

    Account depositingFromAccount =
        Account.builder().id(12345).build();

    new OpenAccountPage(driver).
        load().
        open(aNewCheckingAccount, depositingFromAccount);

    boolean newAccountIdIsDisplayed = new OpenAccountResultPage(driver).newAccountIdIsDisplayed();

    assertThat(newAccountIdIsDisplayed).isTrue();
}

Now, I don’t know about you, but to me, that’s almost as readable as the Cucumber scenario we’ve seen earlier. And remember: if we opted to use Cucumber instead, we would have had to write the same code anyway. So if there’s no upfront communication happening around these scenarios (or in this case, I’d rather just call them tests) anyway, why bother including the Cucumber layer in the first place?

Let’s look at some of the things I’ve implemented to make this code as readable as possible:

Short tests
This is probably the most important one of them all, and that’s why I mention it first. Your tests should be short, sweet and to the point. Ideally, they should check one thing only. Need specific data to be set up prior to the actual test? Try and do that using an API or directly in a database.

In this example, I’m calling a method initializeDatabaseBeforeTest() to reset the database to a known state via an API. There’s plenty of reading material out there on why your tests should be short, so I’m not going to dive into this too deeply here.

Model business concepts as types in your code
If you want to write tests that are human readable, it really helps to model business concepts that mean something to humans as object types in your code. For example, in the test above, we’re creating a new account. An account, in the context of an online banking system, is an entity that has specific properties. In this case, an account has a type, a unique id and a balance:

@Data
@Builder
@AllArgsConstructor
public class Account {

    private AccountType type;
    private int id;
    private double balance;

    public Account(){}
}

I’m using Lombok here to generate getters and setters as well as a builder to allow for fluid object creation in my test method.

It’s important that everybody understands and agrees on the definition of these POJOs (Plain Old Java Objects), such as the Account object here. This massively helps people that are not as familiar with the code as the person who wrote it to understand what’s happening. Not using Cucumber doesn’t absolve you from communicating with your amigos!

Another tip: if a property of a business object can only have specific values, use an enum, like we did here using AccountType:

public enum AccountType {
    CHECKING,
    SAVINGS
}

This prevents objects and properties to accidentally being assigned a wrong value and it increases readability. Winner!

Think hard about the methods exposed by your Page Objects
To further improve test readability, your Page Objects should (only) expose methods that have business meaning. Looking at the example above, the meat of the test happens on the OpenAccount page, where the new account is created. Next to a load() method used to navigate to the page directly (only use these for pages that you can load directly), it has an open() method that takes two arguments, both of type Account, the POJO we’ve seen before. The first one represents the new account, the second represents the account from which the initial deposit into the new account is made.

If you look at the page where you can open an account in the ParaBank application, you’ll see that there’s not much else to do than opening an account, so it makes sense to expose this action to the test methods that use the OpenAccount Page Object.

Choose good names, then choose better ones
You’ve hopefully seen by now that I tried to choose the names I use in my code very carefully, so as to maximize readability. This is hard. I changed the names of my variables and methods many times when I created this example, and I feel that there’s still more room for improvement.

Long variable and method names aren’t inherently bad, as long as they stick to the point. That’s why, for example, I chose to name the method that opens a new account on the OpenAccount page as open() instead of openAccount().

From the context, it’s clear that we’re opening an account here. It’s a method of the OpenAccount page, and its arguments are of type Account. No need to mention it again in the method name, as I did in an earlier iteration. By the way, I learned this from the Clean Code book, which I think is a very valuable read for automation engineers. Lots of good stuff in there.

Use libraries that help you with readability
Apart from Lombok, I also used the AssertJ library to help me write more readable assertions. So, instead of using the default JUnit assertTrue() method, I can now write

assertThat(newAccountIdIsDisplayed).isTrue();

which I think is easier to read. AssertJ has a lot of methods that can help you write more readable assertions, and I think it’s worth checking out for everybody writing Java test code.

So, all in all, I hope that the example above has shown you that it is possible to write (automation) code that is human readable without adding another layer of abstraction in the form of a tool like Cucumber or SpecFlow. This GitHub repository contains the examples I’ve shown here, plus a couple more tests to show some more example of readable (Selenium) test code.

I’m sure there’s still more room for improvement, and I’d love to hear your suggestions on how to further improve the readability of the test code shown here. My main point, though, is to show you that you don’t need Cucumber to make your tests readable to humans.

On no longer wanting to depend on ExpectedConditions in my C# Selenium WebDriver tests

Those of you that have been writing tests with Selenium in C# for a while might have noticed that as of version 3.11, some often-used things have been marked as deprecated, most notably the PageFactory implementation and the ExpectedConditions class. For those of you that have not read the motivation behind this, here is the rationale behind it, as written by Jim Evans, core contributor to the Selenium C# bindings.

These implementations have been moved to a new location and have been made available as separate NuGet packages (DotNetSeleniumExtras.PageObjects and DotNetSeleniumExtras.WaitHelpers, respectively). As of the time of writing this blog post, however, there’s still no new maintainer for this GitHub repository, meaning that there’s no support available and no certainty as to its future.

Notice of ExpectedConditions class being deprecated, as visible in Visual Studio

And while that is no problem yet, since you can still use the deprecated classes from the Selenium bindings or use the DotNetSeleniumExtras packages, I think it’s a risk to keep on relying on code that is effectively unsupported.

In my case, for PageFactory, that’s not a problem, since I don’t typically use it anyway. For ExpectedConditions, however, it’s a different story. When I’m writing and teaching Selenium automation, I’m pretty much still following the general approach I described a couple of years ago in this post. It still works, it’s readable, easy to maintain and explain and has proven to be effective in many different projects I’ve worked on over the years.

However, it also relies on the use of ExpectedConditions, and that’s something that is bothering me more and more often.

So, while preparing for a training session I delivered recently, I started looking for an alternative way of writing my wrapper methods in C#. And it turns out that C# offers an elegant way of doing so, out of the box, in the form of lambda expressions.

Lambda expressions (also known as anonymous functions) is a function that’s not bound to an identifier, but instead can be passed as an argument to a higher-order function. The concept stems from functional programming (it has been a long while since I’ve done any of that, by the way) but has been part of C# for a while. By the way, lambda expressions are available in Java too, from Java 8 onwards, for those of you not working with C#.

Let’s look at an example. Here’s my implementation for a wrapper function around the default Selenium SendKeys() method that takes care of synchronization and exception handling using ExpectedConditions:

public void SendKeys(By by, string valueToType)
{
    try
    {
      new WebDriverWait(_driver, TimeSpan.FromSeconds(Constants.DEFAULT_TIMEOUT)).Until(ExpectedConditions.ElementToBeClickable(by));
        _driver.FindElement(by).Clear();
        _driver.FindElement(by).SendKeys(valueToType);
    }
    catch (Exception ex) when (ex is NoSuchElementException || ex is WebDriverTimeoutException)
    {
         Assert.Fail($"Exception occurred in SeleniumHelper.SendKeys(): element located by {by.ToString()} could not be located within {Constants.DEFAULT_TIMEOUT} seconds.");
    }
}

Here’s the same wrapper, but now passing a lambda expression of our own as an argument to the Until() method instead of a method in ExpectedConditions:

public void SendKeys(By by, string valueToType)
{
    WebDriverWait wait = new WebDriverWait(_driver, TimeSpan.FromSeconds(Constants.DEFAULT_TIMEOUT));
            
    try
    {
        IWebElement myElement = wait.Until<IWebElement>(driver =>
        {
            IWebElement tempElement = _driver.FindElement(by);
            return (tempElement.Displayed && tempElement.Enabled) ? tempElement : null;
        }
        );
        myElement.Clear();
        myElement.SendKeys(valueToType);
    }
    catch (WebDriverTimeoutException)
    {
        Assert.Fail($"Exception in SendKeys(): element located by {by.ToString()} not visible and enabled within {Constants.DEFAULT_TIMEOUT} seconds.");
    }
}

From the Selenium docs, we learn that the Until() method repeatedly (with a default but configurable polling interval of 500 milliseconds) evaluates the lambda expression, until one of these conditions applies:

  • The expression returns neither null nor false;
  • The function throws an exception that is not in the list of ignored expressions;
  • The specified timeout expires

This explains why I created the lambda expression like I did. Before clicking on an element, I want to wait until it is clickable, which I define as it being both visible (expressed through the Displayed property) and enabled (the Enabled property). If both are true, I return the element, otherwise null. If this happens within the timeout set, I’ll clear the text field and use the standard SendKeys() method to type the specified value. If a WebDriverTimeOutException occurs, I’ll handle it accordingly, in this case by failing the test with a readable message.

Yes, implementing my helper methods like this makes my code a little more verbose, but you could argue that this is a small price to pay for not having to rely on (part of) a library that has been deprecated and of which the future is uncertain. Plus, because I define all these helpers in a single class, there’s really only one place in my code that is affected. The Page Objects and my test code remain wholly unaffected.

This GitHub repository contains a couple of runnable example tests that are written against the ParaBank demo application. This is an updated version of the solution described here. An updated version of this last blog post is coming soon, by the way, so that it reflects some opinions and preferences of mine that have changed since I wrote that one. No longer having to rely on ExpectedConditions, as I discussed here, is but one of those changes.