Recently, I’ve delivered my first ever three day ‘Python for testers’ training course. One of the topics that was covered in this course is writing tests for RESTful APIs using the Python requests library and the pytest unit testing framework.

In this short series of blog posts, I want to explore the Python requests library and how it can be used for writing tests for RESTful APIs. This is the third blog post in the series, in which we will cover working with XML request and response bodies. Previous blog posts in this series talked about getting started with requests and pytest, and about creating data driven tests.

REST APIs and XML
While most REST APIs I encounter nowadays work with JSON as the preferred data format for request and response body bodies, from time to time you’ll encounter APIs that work with XML. And since XML is a little more cumbersome to work with XML in code compared to JSON (not just in Python, but in general), I thought it would be a good idea to show you some examples of how to create XML request bodies and how to parse and assert on XML response bodies when you’re working with the requests library.

For the examples in this blog post, I’ll be using an operation from the ParaBank REST API that can be used to submit bill payments. It’s available at

http://parabank.parasoft.com/parabank/services/bank/billpay

and takes, next to two query parameters specifying the source accountId and the bill amount, an XML request body containing specifics about the person to whom the payment is sent, i.e., the payee. Not surprisingly, this request body is sent to the API provider using an HTTP POST.

Creating XML request bodies using strings
I’d like to show you two distinct approaches to creating XML request bodies. The first one is the most straightforward one, but also the least flexible: creating a method that returns a string object containing the XML body:

def fixed_xml_body_as_string():
    return """
    <payee>
        <name>John Smith</name>
        <address>
            <street>My street</street>
            <city>My city</city>
            <state>My state</state>
            <zipCode>90210</zipCode>
        </address>
        <phoneNumber>0123456789</phoneNumber>
        <accountNumber>12345</accountNumber>
    </payee>
    """

Note the use of the triple double quotes to allow you to declare a multi-line string. Of course, instead of hard-coding our XML request body in our code, we could also read it from an XML (or text) file stored somewhere on our file system. The result is the same.

If we want to pass this XML request body to our API, we can do that like this:

def test_send_xml_body_from_string_check_status_code_and_content_type():
    response = requests.post(
        "http://parabank.parasoft.com/parabank/services/bank/billpay?accountId=12345&amount=500",
        headers={"Content-Type": "application/xml"},
        data=fixed_xml_body_as_string()
    )
    assert response.status_code == 200
    assert response.headers["Content-Type"] == "application/xml"

Note that we explicitly set the Content-Type header of the request to application/xml to make sure the provider understands that the request body should be interpreted as XML. Sending the XML request body is done by assigning the return value of our method returning the XML as a string to the data parameter of the requests post() method.

To check that our request has been received and processed successfully, we assert that the response status code equals 200 and that the response Content-Type header has a value of application/xml. We’ll take a closer look at the actual XML response body later on in this post.

Creating XML request bodies using ElementTree
The other approach to working with XML request bodies is to programmatically build them. Python contains a powerful library to do this, called ElementTree. We can import this into our module using

import xml.etree.ElementTree as et

Since an XML document is essentially a tree with a root node with child nodes attached to it, we start creating our XML request body by defining the payee root node:

payee = et.Element('payee')

We can then define an element name that is a child element of payee:

name = et.SubElement(payee, 'name')

We also need to assign an element value to the name element:

name.text = 'John Smith'

It’s not required for this example, but if you would have to add an attribute, say, type, with value fullName to the name element, you could do so like this:

name.set('type', 'fullName')

Creating the entire XML request body for our API call is a matter of repeating the above statements in the right order, with the right values:

def create_xml_body_using_elementtree():
    payee = et.Element('payee')
    name = et.SubElement(payee, 'name')
    name.text = 'John Smith'
    address = et.SubElement(payee, 'address')
    street = et.SubElement(address, 'street')
    street.text = 'My street'
    city = et.SubElement(address, 'city')
    city.text = 'My city'
    state = et.SubElement(address, 'state')
    state.text = 'My state'
    zip_code = et.SubElement(address, 'zipCode')
    zip_code.text = '90210'
    phone_number = et.SubElement(payee, 'phoneNumber')
    phone_number.text = '0123456789'
    account_number = et.SubElement(payee, 'accountNumber')
    account_number.text = '12345'
    return et.tostring(payee)

Note that we need to convert the element tree into a string before we can use it with the requests library. We can do this using the tostring() method.

While the approach using ElementTree might look a little more cumbersome than simply specifying our XML as a string, it gives us the option of creating more complex and flexible XML documents by creating loops to repeat XML blocks, working with data sources that are transformed into XML, and so on. I myself don’t really prefer one approach over the other, but I think it’s good to be aware of both and choose the one that best fits your requirements.

If we want to use the XML created using ElementTree above as a request body, we can do that in exactly the same way as when we used a string containing the XML:

def test_send_xml_body_from_elementtree_check_status_code_and_content_type():
    response = requests.post(
        "http://parabank.parasoft.com/parabank/services/bank/billpay?accountId=12345&amount=500",
        headers={"Content-Type": "application/xml"},
        data=create_xml_body_using_elementtree()
    )
    assert response.status_code == 200
    assert response.headers["Content-Type"] == "application/xml"

Parsing and working with XML response bodies
Now that we have covered creating XML request bodies, let’s see what we can do with XML responses. By far the most powerful way to create specific assertions is to convert the XML response body into an ElementTree and then asserting on its properties.

As an example, we’re going to perform an HTTP GET call to

http://parabank.parasoft.com/parabank/services/bank/accounts/12345

which returns details of the account with ID 12345. If we want to assert, for example, that the root node of the XML response is named account, and that it has neither any attributes nor a text value, we can do this as follows:

def test_check_root_of_xml_response():
    response = requests.get("http://parabank.parasoft.com/parabank/services/bank/accounts/12345")
    response_body_as_xml = et.fromstring(response.content)
    xml_tree = et.ElementTree(response_body_as_xml)
    root = xml_tree.getroot()
    assert root.tag == "account"
    assert len(root.attrib) == 0
    assert root.text is None

Note that we first have to convert the XML response body to an object of type Element using the fromstring() method, then create an ElementTree out of that using the ElementTree() constructor, which takes an Element as its argument.

If we’re interested in the text value of a specific subelement of the XML response, for example customerId which contains the ID of the customer to whom this account belongs, we can do that by finding it in the ElementTree using the find() method, then write an assertion on the text property of the found element:

def test_check_specific_element_of_xml_response():
    response = requests.get("http://parabank.parasoft.com/parabank/services/bank/accounts/12345")
    response_body_as_xml = et.fromstring(response.content)
    xml_tree = et.ElementTree(response_body_as_xml)
    first_name = xml_tree.find("customerId")
    assert first_name.text == "12212"

It’s good to know that the find() method returns the first occurrence of a specific element. If we want to return all elements that match a specific name, we need to use findall() instead:

def test_use_xpath_for_more_sophisticated_checks():
    response = requests.get("http://parabank.parasoft.com/parabank/services/bank/customers/12212/accounts")
    response_body_as_xml = et.fromstring(response.content)
    xml_tree = et.ElementTree(response_body_as_xml)
    savings_accounts = xml_tree.findall(".//account/type[.='SAVINGS']")
    assert len(savings_accounts) > 1

As you can see, next to passing element names directly, we can also use XPath expressions to perform more sophisticated selections. The expression

 .//account/type[.='SAVINGS']

in the example above selects all occurrences of the type element (a child element of account) that have SAVINGS as their element value.

Using the examples for yourself
The code examples I have used in this blog post can be found on my GitHub page. If you download the project and (given you have installed Python properly) run

 pip install -r requirements.txt

from the root of the python-requests project to install the required libraries, you should be able to run the tests for yourself. See you next time!

Recently, I’ve delivered my first ever three day ‘Python for testers’ training course. One of the topics that was covered in this course is writing tests for RESTful APIs using the Python requests library and the pytest unit testing framework.

In this short series of blog posts, I want to explore the Python requests library and how it can be used for writing tests for RESTful APIs. This is the second blog post in the series, in which we will cover writing data driven tests. The first blog post in the series can be found here.

About data driven testing
Before we get started, let’s quickly review what data driven tests are.

Often, when developing automated tests, you will find yourself wanting to test the same business logic, algorithm, computation or application flow multiple times with different input values and expected output values. Now, technically, you could achieve that by simply copying and pasting an existing test and changing the requires values.

From a maintainability perspective, however, that’s not a good idea. Instead, you might want to consider writing a data driven test: a test that gets its test data from a data source and iterates over the rows (or records) in that data source.

Creating a test data object
Most unit testing frameworks provide support for data driven testing, and pytest is no exception. Before we see how to create a data driven test in Python, let’s create our test data source first. In Python, this can be as easy as creating a list of tuples, where each tuple in the list corresponds to an iteration of the data driven test (a ‘test case’, if you will).

test_data_zip_codes = [
    ("us", "90210", "Beverly Hills"),
    ("ca", "B2A", "North Sydney South Central"),
    ("it", "50123", "Firenze")
]

We’re going to run three iterations of the same test: retrieving location data for a given country code and zip code (the first two elements in each tuple) and then asserting that the corresponding place name returned by the API is equal to the specified expected place name (the third tuple element).

Creating a data driven test in pytest
Now that we have our test data available, let’s see how we can convert an existing test from the first blog post into a data driven test.

@pytest.mark.parametrize("country_code, zip_code, expected_place_name", test_data_zip_codes)
def test_using_test_data_object_get_location_data_check_place_name(country_code, zip_code, expected_place_name):
    response = requests.get(f"http://api.zippopotam.us/{country_code}/{zip_code}")
    response_body = response.json()
    assert response_body["places"][0]["place name"] == expected_place_name

Pytest supports data driven testing through the built-in @pytest.mark.parametrize marker. This marker takes two arguments: the first tells pytest how (i.e., in which order) to map the elements in a tuple from the data source to the arguments of the test method, and the second argument is the test data object itself.

The test methods we have seen in the previous post did not have any arguments, but since we’re feeding test data to our tests from outside, we need to specify three arguments to the test method here: the country code, the zip code and the expected place name. We can then use these arguments in our test method body, the first two as path parameter values in the API call, the last one as the expected result value which is extracted from the JSON response body.

Running the test
When we run our data driven test, we see that even though we only have a single test method, pytest detects and runs three tests. Or better: it runs the same test three times, once for each tuple in the test data object.

This, to me, demonstrates the power of data driven testing. We can run as many iterations as required for a given test, without code duplication, given that we tell pytest where to find the test data. Need an additional test iteration with different test data values? Just add a record to the test data object. Want to update or remove a test case? You know the drill.

Another useful thing about data driven testing using pytest: when one of the test iterations fails, pytest will tell you which one did and what were the corresponding test data values used:

Creating an external data source
In the example above, our test data was still hardcoded into our test code. This might not be your preferred way of working. What if we could specify the test data in an external data source instead, and tell pytest to read it from there?

As an example, let’s create a .csv file that contains the same test data as the test data object we’ve seen earlier:

country_code,zip_code,expected_place_name
us,90210,Beverly Hills
ca,B2A,North Sydney South Central
it,50123,Firenze

To use this test data in our test, we need to write a Python method that reads the data from the file and returns it in a format that’s compatible with the pytest parametrize marker. Python offers solid support for handling .csv files in the built-in csv library:

import csv

def read_test_data_from_csv():
    test_data = []
    with open('test_data/test_data_zip_codes.csv', newline='') as csvfile:
        data = csv.reader(csvfile, delimiter=',')
        next(data)  # skip header row
        for row in data:
            test_data.append(row)
    return test_data

This method opens the .csv file in reading mode, skips the header row, adds all other lines to the list of test data values test_data one by one and returns the test data object.

The test method itself now needs to be updated to not use the hardcoded test data object anymore, but instead use the return value of the method that reads the data from the .csv file:

@pytest.mark.parametrize("country_code, zip_code, expected_place_name", read_test_data_from_csv())
def test_using_csv_get_location_data_check_place_name(country_code, zip_code, expected_place_name):
    response = requests.get(f"http://api.zippopotam.us/{country_code}/{zip_code}")
    response_body = response.json()
    assert response_body["places"][0]["place name"] == expected_place_name

Running this updated test code will show that this approach, too, results in three passing test iterations. Of course, you can use test data sources other than .csv too, such as database query results or XML or JSON files. As long as you’re able to write a method that returns a list of test data value tuples, you should be good to go.

In the next blog post, we’re going to further explore working with JSON and XML in API request and response bodies.

Using the examples for yourself
The code examples I have used in this blog post can be found on my GitHub page. If you download the project and (given you have installed Python properly) run

 pip install -r requirements.txt 

from the root of the python-requests project to install the required libraries, you should be able to run the tests for yourself. See you next time!