Writing tests for RESTful APIs in Python using requests – part 2: data driven tests

Recently, I’ve delivered my first ever three day ‘Python for testers’ training course. One of the topics that was covered in this course is writing tests for RESTful APIs using the Python requests library and the pytest unit testing framework.

In this short series of blog posts, I want to explore the Python requests library and how it can be used for writing tests for RESTful APIs. This is the second blog post in the series, in which we will cover writing data driven tests. The first blog post in the series can be found here.

About data driven testing
Before we get started, let’s quickly review what data driven tests are.

Often, when developing automated tests, you will find yourself wanting to test the same business logic, algorithm, computation or application flow multiple times with different input values and expected output values. Now, technically, you could achieve that by simply copying and pasting an existing test and changing the requires values.

From a maintainability perspective, however, that’s not a good idea. Instead, you might want to consider writing a data driven test: a test that gets its test data from a data source and iterates over the rows (or records) in that data source.

Creating a test data object
Most unit testing frameworks provide support for data driven testing, and pytest is no exception. Before we see how to create a data driven test in Python, let’s create our test data source first. In Python, this can be as easy as creating a list of tuples, where each tuple in the list corresponds to an iteration of the data driven test (a ‘test case’, if you will).

test_data_zip_codes = [
    ("us", "90210", "Beverly Hills"),
    ("ca", "B2A", "North Sydney South Central"),
    ("it", "50123", "Firenze")

We’re going to run three iterations of the same test: retrieving location data for a given country code and zip code (the first two elements in each tuple) and then asserting that the corresponding place name returned by the API is equal to the specified expected place name (the third tuple element).

Creating a data driven test in pytest
Now that we have our test data available, let’s see how we can convert an existing test from the first blog post into a data driven test.

@pytest.mark.parametrize("country_code, zip_code, expected_place_name", test_data_zip_codes)
def test_using_test_data_object_get_location_data_check_place_name(country_code, zip_code, expected_place_name):
    response = requests.get(f"http://api.zippopotam.us/{country_code}/{zip_code}")
    response_body = response.json()
    assert response_body["places"][0]["place name"] == expected_place_name

Pytest supports data driven testing through the built-in @pytest.mark.parametrize marker. This marker takes two arguments: the first tells pytest how (i.e., in which order) to map the elements in a tuple from the data source to the arguments of the test method, and the second argument is the test data object itself.

The test methods we have seen in the previous post did not have any arguments, but since we’re feeding test data to our tests from outside, we need to specify three arguments to the test method here: the country code, the zip code and the expected place name. We can then use these arguments in our test method body, the first two as path parameter values in the API call, the last one as the expected result value which is extracted from the JSON response body.

Running the test
When we run our data driven test, we see that even though we only have a single test method, pytest detects and runs three tests. Or better: it runs the same test three times, once for each tuple in the test data object.

Console output for a passing data driven test

This, to me, demonstrates the power of data driven testing. We can run as many iterations as required for a given test, without code duplication, given that we tell pytest where to find the test data. Need an additional test iteration with different test data values? Just add a record to the test data object. Want to update or remove a test case? You know the drill.

Another useful thing about data driven testing using pytest: when one of the test iterations fails, pytest will tell you which one did and what were the corresponding test data values used:

Console output for a failing data driven test

Creating an external data source
In the example above, our test data was still hardcoded into our test code. This might not be your preferred way of working. What if we could specify the test data in an external data source instead, and tell pytest to read it from there?

As an example, let’s create a .csv file that contains the same test data as the test data object we’ve seen earlier:

us,90210,Beverly Hills
ca,B2A,North Sydney South Central

To use this test data in our test, we need to write a Python method that reads the data from the file and returns it in a format that’s compatible with the pytest parametrize marker. Python offers solid support for handling .csv files in the built-in csv library:

import csv

def read_test_data_from_csv():
    test_data = []
    with open('test_data/test_data_zip_codes.csv', newline='') as csvfile:
        data = csv.reader(csvfile, delimiter=',')
        next(data)  # skip header row
        for row in data:
    return test_data

This method opens the .csv file in reading mode, skips the header row, adds all other lines to the list of test data values test_data one by one and returns the test data object.

The test method itself now needs to be updated to not use the hardcoded test data object anymore, but instead use the return value of the method that reads the data from the .csv file:

@pytest.mark.parametrize("country_code, zip_code, expected_place_name", read_test_data_from_csv())
def test_using_csv_get_location_data_check_place_name(country_code, zip_code, expected_place_name):
    response = requests.get(f"http://api.zippopotam.us/{country_code}/{zip_code}")
    response_body = response.json()
    assert response_body["places"][0]["place name"] == expected_place_name

Running this updated test code will show that this approach, too, results in three passing test iterations. Of course, you can use test data sources other than .csv too, such as database query results or XML or JSON files. As long as you’re able to write a method that returns a list of test data value tuples, you should be good to go.

In the next blog post, we’re going to further explore working with JSON and XML in API request and response bodies.

Using the examples for yourself
The code examples I have used in this blog post can be found on my GitHub page. If you download the project and (given you have installed Python properly) run

 pip install -r requirements.txt 

from the root of the python-requests project to install the required libraries, you should be able to run the tests for yourself. See you next time!

19 thoughts on “Writing tests for RESTful APIs in Python using requests – part 2: data driven tests

  1. import pytest
    import requests

    #creating zip code object

    test_data_zip_codes = [
    (“us”, “90210”, “Beverly Hills”),
    (“ca”, “B2A”, “North Sydney South Central”),
    (“it”, “50123”, “Firenze”)]

    #creating the data driven test
    @pytest.mark.parametrize(“country_code, zip_code, expected_place_name”, test_data_zip_codes)
    def validate_zip_code(country_code, zip_code, expected_place_name):
    response = requests.get(f’http://api.zippopotam.us/{country_code}/{zip_code}’)
    response_body = response.json()
    assert response_body[“places”][0][“place name”] == expected_place_name

    How to run the test? if i execute using the method name, it asking me three positional requirements to be passed.


      • Do you have any link for working with JSON ?

        I use SOAPUI for testing Restful APIs . So i can change the Payload say 100 times and i can write 100 test steps by using the json payload

        Similarly if i want to do the same using Requests in Pytest Framework , how do i do that for POST http method?

        • Hey Prasanth, let me see if I understand you correctly.. You want to:
          – Create a JSON payload in Python, and that payload is quite large
          – Modify some of the fields in that payload for different tests
          – Send that payload to an API using requests

          Am I right?

  2. Awesome work Bas. I was able to create a new automation project from scratch and set up everything in minutes. I am also able to use your examples project easily.

    Do you plan to extend this series and framework by adding a logger or a runner to it in the future? Does it need such extras or is it perfectly fine the way it is?

    • Hey Vladimir,

      that’s so good to hear! Well done 🙂

      I’ll probably come back to this series at some point, there are a couple more things I want to cover. I don’t think a test runner will be part of it (I think pytest or your unit testing framework of choice handles that well enough) but logging and providing additional information is a really good idea.

  3. Dear Bas,

    Hope you are doing good !!

    I like your post related to testing API using python requests module .

    Looking on data driven using csv data set , i have a question and i hope you don;t mind asking here .

    In DDT , if i have data in csv like
    key1 , key2, key3, key4
    v1, v2, v3,
    v1, ,v2,v4

    one request accept test data (key1, key2, key3)
    another request accept test data (key1, key2, key4)

    Can you help in handling the same using csv data set config ??

    Also is there any way let say i want to execute row1 data for one request and row2 data for different request .

    I hope my question is cleared .

    Awaiting for a solution from you . Thanks In Advance

    • I’d use two different data sets in that case.. I wouldn’t use the same data set for tests that essentially cover different flows. That would mean implementing more logic in the tests or the mechanism that reads the data sources and I don’t like that kind of complexity.

      • i was trying with above code Here is the snippets of my code for get request for rest url where i’m asserting if status code is 200. Not sure how to just pass url as parametrize

        import pytest
        import csv
        import requests
        import json

        @pytest.mark.parametrize(“test_data”, read_test_data_from_csv())
        def test_response_gpv(“test_data”, read_test_data_from_csv() ):
        response = requests.get(final_url, auth=(‘abc’, ‘xyz’), headers=header)
        assert response.status_code == 200

        def read_test_data_from_csv():
        test_data = []
        sub_url = ‘’
        header = {‘accept’: ‘application/json’, ‘Content-type’: ‘application/json’}
        f = open(‘./gpv.csv’, ‘r’)
        r = csv.reader(f)
        for row in r:
        a = str(row)[1:-1] #for removing the square bracket from list
        b = a.replace(“‘”, “”) # for removing the single quotes from list
        final_url = sub_url+b
        return test_data

        • here is my csv file format


          • So this:


            is a single endpoint that should be appended to the base URL?

            I’ll write you some code tomorrow, I think this can be done a lot simpler.

          • /restconf/config/network-topology:network-topology/topology/topology-netconf/node/0005B94238A0/yang-ext:mount/onecell:device/device-info

  4. Hello Bas,

    Right now i am using SOAP UI for testing restful Apis where i test Json Request Payload by modifying the inputs and it is huge. Could you please share a link on how to do this? I searched your blog and it says you are going to work on it

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.