Writing API tests in Python with Tavern

So far, most of the blog posts I’ve written that covered specific tools were focused on either Java or C#. Recently, though, I got a request for test automation training for a group of data science engineers, with the explicit requirement to use Python-based tools for the examples and exercises.

Since then, I’ve been slowly expanding my reading and learning to also include the Python ecosystem, and I’ve also included a couple of Python-based test automation courses in my training offerings. So far, I’m pretty impressed. There are plenty of powerful test tools available for Python, and in this post, I’d like to take a closer look at one of them, Tavern.

Tavern is an API testing framework running on top of pytest, one of the most popular Python unit testing frameworks. It offers a range of features to write and run API tests, and if there’s something you can’t do with Tavern, it claims to be easily extensible through Python or pytest hooks and features. I can’t vouch for its extensibility yet, thought, since all that I’ve been doing with Tavern so far was possible out of the box. Tavern has good documentation too, which is also nice.

Installing Tavern on your machine is easiest when done through pip, the Python package installer and manager using the command

pip install -U tavern

Tests in Tavern are written in YAML files. Now, you either love it or hate it, but it works. To get started, let’s write a test that retrieves location data for the US zip code 90210 from the Zippopotam.us API and checks whether the response HTTP status code is equal to 200. This is what that looks like in Tavern:

test_name: Get location for US zip code 90210 and check response status code

stages:
  - name: Check that HTTP status code equals 200
    request:
      url: http://api.zippopotam.us/us/90210
      method: GET
    response:
      status_code: 200

As I said, Tavern runs on top of pytest. So, to run this test, we need to invoke pytest and tell it that the tests we want to run are in the YAML file we created:

As you can see, the test passes.

Another thing you might be interested in is checking values for specific response headers. Let’s check that the response content type is equal to ‘application/json’, telling the API consumer that they need to interpret the response as JSON:

test_name: Get location for US zip code 90210 and check response content type

stages:
  - name: Check that content type equals application/json
    request:
      url: http://api.zippopotam.us/us/90210
      method: GET
    response:
      headers:
        content-type: application/json

Of course, you can also perform checks on the response body. Here’s an example that checks that the place name associated with the aforementioned US zip code 90210 is equal to ‘Beverly Hills’:

test_name: Get location for US zip code 90210 and check response body content

stages:
  - name: Check that place name equals Beverly Hills
    request:
      url: http://api.zippopotam.us/us/90210
      method: GET
    response:
      body:
        places:
          - place name: Beverly Hills

Since APIs are all about data, you might want to repeat the same test more than once, but with different values for input parameters and expected outputs (i.e., do data driven testing). Tavern supports this too by exposing the pytest parametrize marker:

test_name: Check place name for multiple combinations of country code and zip code

marks:
  - parametrize:
      key:
        - country_code
        - zip_code
        - place_name
      vals:
        - [us, 12345, Schenectady]
        - [ca, B2A, North Sydney South Central]
        - [nl, 3825, Vathorst]

stages:
  - name: Verify place name in response body
    request:
      url: http://api.zippopotam.us/{country_code}/{zip_code}
      method: GET
    response:
      body:
        places:
          - place name: "{place_name}"

Even though we specified only a single test with a single stage, because we used the parametrize marker and supplied the test with three test data records, pytest effectively runs three tests (similar to what @DataProvider does in TestNG for Java, for example):

Tavern output for our data driven test, run from within PyCharm

So far, we have only performed GET operations to retrieve data from an API provider, so we did not need to specify any request body contents. When, as an API consumer, you want to send data to an API provider, for example when you perform a POST or a PUT operation, you can do that like this using Tavern:

test_name: Check response status code for a very simple addition API

stages:
  - name: Verify that status code equals 200 when two integers are specified
    request:
      url: http://localhost:5000/add
      json:
        first_number: 5
        second_number: 6
      method: POST
    response:
      status_code: 200

This test will POST a JSON document

{'first_number': 5, 'second_number': 6}

to the API provider running on localhost port 5000. Please note that for obvious reasons this test will fail when you run it yourself, unless you built an API or a mock that behaves in a way that makes the test pass (great exercise, different subject …).

So, that’s it for a quick introduction to Tavern. I quite like the tool for its straightforwardness. What I’m still wondering is whether working with YAML will lead to maintainability and readability issues when you’re working with larger test suites and larger request or response bodies. I’ll keep working with Tavern in my training courses for now, so a follow-up blog post might see the light of day in a while!

All examples can be found on this GitHub page.

Announcing my API testing masterclass

Do you want to learn more about APIs and how to test them? Have you been looking for a comprehensive course that teaches you everything there is to know about testing APIs and testing software systems at the API level? If so, you might want to read on!

Many API testing courses out there focus on a specific tool or technique that you can leverage to improve your API testing efforts. While those courses definitely have their use, I feel there’s much more to it if you really want to become well-versed in testing APIs and testing systems at the API level.

That’s the reason I created a brand new, three day masterclass that will teach you, among other things, about:

  • APIs and their role in modern software systems
  • Why to test APIs, and why to test at the API level
  • What to look for when testing APIs
  • Exploring APIs for testing purposes
  • Using tools for API test automation
  • API performance and security testing
  • API specifications and contract testing
  • Mocking APIs for testing purposes

A much more detailed description of this API testing masterclass, including a day-to-day breakdown of course contents and learning goals, can be found on the course page.

Need to convince your manager?
Please send them this course flyer, highlighting all of the benefits and providing a summary of the training course, all neatly on a single page. And don’t forget to bring the Early Bird discount to their attention!

I’m looking forward to many successful deliveries of this API testing masterclass, and I hope to see you all at one of them.

Some words of thanks
I owe a lot of thank you’s to Maria Kedemo of Black Koi Consulting for putting the idea for this masterclass in my head at exactly the right time, as well as for further discussing it and for reviewing the content outline as it can be found on the course page.

I am also very grateful for the content outline review and valuable comments made by Elizabeth Zagroba, Angie Jones and Joe Colantonio.

RESTful API testing in C# with RestSharp

Since my last blog post that involved creating tests at the API level in C#, I’ve kept looking around for a library that would fit all my needs in that area. So far, I still haven’t found anything more suitable than RestSharp. Also, I’ve found out that RestSharp is more versatile than I initially thought it was, and that’s the reason I thought it would be a good idea to dedicate a blog post specifically to this tool.

The examples I show you in this blog post use the Zippopotam.us API, a publicly accessible API that resolves a combination of a country code and a zip code to related location data. For example, when you send an HTTP GET call to

http://api.zippopotam.us/us/90210

(where ‘us’ is a path parameter representing a country code, and ‘90210’ is a path parameter representing a zip code), you’ll receive this JSON document as a response:

{
	post code: "90210",
	country: "United States",
	country abbreviation: "US",
	places: [
		{
			place name: "Beverly Hills",
			longitude: "-118.4065",
			state: "California",
			state abbreviation: "CA",
			latitude: "34.0901"
		}
	]
}

Some really basic checks
RestSharp is available as a NuGet package, which makes it really easy to add to your C# project. So, what does an API test written using RestSharp look like? Let’s say that I want to check whether the previously mentioned HTTP GET call to http://api.zippopotam.us/us/90210 returns an HTTP status code 200 OK, this is what that looks like:

[Test]
public void StatusCodeTest()
{
    // arrange
    RestClient client = new RestClient("http://api.zippopotam.us");
    RestRequest request = new RestRequest("nl/3825", Method.GET);

    // act
    IRestResponse response = client.Execute(request);

    // assert
    Assert.That(response.StatusCode, Is.EqualTo(HttpStatusCode.OK));
}

If I wanted to check that the content type specified in the API response header is equal to “application/json”, I could do that like this:

[Test]
public void ContentTypeTest()
{
    // arrange
    RestClient client = new RestClient("http://api.zippopotam.us");
    RestRequest request = new RestRequest("nl/3825", Method.GET);

    // act
    IRestResponse response = client.Execute(request);

    // assert
    Assert.That(response.ContentType, Is.EqualTo("application/json"));
}

Creating data driven tests
As you can see, creating these basic checks is quite straightforward with RestSharp. Since APIs are all about sending and receiving data, it would be good to be able to make these tests data driven. NUnit supports data driven testing through the TestCase attribute, and using that together with passing the parameters to the test method is really all that it takes to create a data driven test:

[TestCase("nl", "3825", HttpStatusCode.OK, TestName = "Check status code for NL zip code 7411")]
[TestCase("lv", "1050", HttpStatusCode.NotFound, TestName = "Check status code for LV zip code 1050")]
public void StatusCodeTest(string countryCode, string zipCode, HttpStatusCode expectedHttpStatusCode)
{
    // arrange
    RestClient client = new RestClient("http://api.zippopotam.us");
    RestRequest request = new RestRequest($"{countryCode}/{zipCode}", Method.GET);

    // act
    IRestResponse response = client.Execute(request);

    // assert
    Assert.That(response.StatusCode, Is.EqualTo(expectedHttpStatusCode));
}

When you run the test method above, you’ll see that it will run two tests: one that checks that the NL zip code 3825 returns HTTP 200 OK, and one that checks that the Latvian zip code 1050 returns HTTP 404 Not Found (Latvian zip codes are not yet available in the Zippopotam.us API). In case you ever wanted to add a third test case, all you need to do is add another TestCase attribute with the required parameters and you’re set.

Working with response bodies
So far, we’ve only written assertions on the HTTP status code and the content type header value for the response. But what if we wanted to perform assertions on the contents of the response body?

Technically, we could parse the JSON response and navigate through the response document tree directly, but that would result in hard to read and hard to maintain code (see for an example this post again, where I convert a specific part of the response to a JArray after navigating to it and then do a count on the number of elements in it. Since you’re working with dynamic objects, you also don’t have the added luxury of autocomplete, because there’s no way your IDE knows the structure of the JSON document you expect in a test.

Instead, I highly prefer deserializing JSON responses to actual objects, or POCOs (Plain Old C# Objects) in this case. The JSON response you’ve seen earlier in this blog post can be represented by the following LocationResponse class:

public class LocationResponse
{
    [JsonProperty("post code")]
    public string PostCode { get; set; }
    [JsonProperty("country")]
    public string Country { get; set; }
    [JsonProperty("country abbreviation")]
    public string CountryAbbreviation { get; set; }
    [JsonProperty("places")]
    public List<Place> Places { get; set; }
}

and the Place class inside looks like this:

public class Place
{
    [JsonProperty("place name")]
    public string PlaceName { get; set; }
    [JsonProperty("longitude")]
    public string Longitude { get; set; }
    [JsonProperty("state")]
    public string State { get; set; }
    [JsonProperty("state abbreviation")]
    public string StateAbbreviation { get; set; }
    [JsonProperty("latitude")]
    public string Latitude { get; set; }
}

Using the JsonProperty attribute allows me to map POCO fields to JSON document elements without names having to match exactly, which in this case is especially useful since some of the element names contain spaces, which are impossible to use in POCO field names.

Now that we have modeled our API response as a C# class, we can convert an actual response to an instance of that class using the deserializer that’s built into RestSharp. After doing so, we can refer to the contents of the response by accessing the fields of the object, which makes for far easier test creation and maintenance:

[Test]
public void CountryAbbreviationSerializationTest()
{
    // arrange
    RestClient client = new RestClient("http://api.zippopotam.us");
    RestRequest request = new RestRequest("us/90210", Method.GET);

    // act
    IRestResponse response = client.Execute(request);

    LocationResponse locationResponse =
        new JsonDeserializer().
        Deserialize<LocationResponse>(response);

    // assert
    Assert.That(locationResponse.CountryAbbreviation, Is.EqualTo("US"));
}

[Test]
public void StateSerializationTest()
{
    // arrange
    RestClient client = new RestClient("http://api.zippopotam.us");
    RestRequest request = new RestRequest("us/12345", Method.GET);

    // act
    IRestResponse response = client.Execute(request);
    LocationResponse locationResponse =
        new JsonDeserializer().
        Deserialize<LocationResponse>(response);

    // assert
    Assert.That(locationResponse.Places[0].State, Is.EqualTo("New York"));
}

So, it looks like I’ll be sticking with RestSharp for a while when it comes to my basic C# API testing needs. That is, until I’ve found a better alternative.

All the code that I’ve included in this blog post is available on my Github page. Feel free to clone this project and run it on your own machine to see if RestSharp fits your API testing needs, too.