TDDing my way to a Python singleton implementation

So, I’ve been working on my first ever project as a software developer for a while now, and it’s crazy (but amazing) to see how much I have learned so far. Anything from writing readable code, to unit testing, to creating good documentation, to packaging and publishing my work, you name it, I’ve done it now. And I’m still learning, a lot, every day. So far, I’m loving it.

One of the challenges I encountered along the way was that I had to implement a singleton in Python. For those of you who don’t know what a singleton is, it’s a pattern that restricts the instantiation of a class to a single instance. I need this in my software because I need a single source of truth for a number of settings that are valid for the duration of a session.

Now, there are plenty of solid examples of how to do that in, for example, Java (see here for two examples as explained on the excellent Baeldung website), but for Python, not so much. It’s not that there aren’t any examples available online, the problem is that there are a lot of them, all implemented in different ways, and not all of them are equally readable and portable to what I needed. I’d prefer a native solution (no relying on third party libraries), since I’m developing a package for distribution myself and the fewer dependencies, the better.

So, I thought, why not try my hand at Test Driven Development (TDD) and see where that leads? It’s a practice I would like to have more experience in anyway.. In this blog post, I’ll try and explain my thought process and what I came up with in the end. It might not be the best implementation (hey, I’m still learning, and this is my first ever attempt at TDD), and I’d love any feedback and suggestions, but it did the trick for me in tackling this challenge.

Let’s go! (This is me talking to myself for the remainder of this blog post)

The first requirement for my singleton class implementation (let’s call it MySingleton) was that it could be instantiated. The simplest test that checks this requirement looks like this:

def test_singleton_can_be_instantiated():
    my_singleton = MySingleton()
    assert my_singleton is not None

and here’s the simplest version of the class code I could think of that makes this test pass:

class MySingleton:

    def __init__(self):
        pass

So far, so good. Our class is not a singleton yet, of course. Also, it’s not a very useful class… Before we move to the actual singleton part, let’s make sure that we can pass a parameter to the __init__() method (comparable to the class constructor in other languages), that it is stored in a field and that it can be read again. First the test:

def test_singleton_can_take_an_argument():
    my_singleton = MySingleton(fruit='banana')
    assert my_singleton.fruit == 'banana'

then the implementation:

class MySingleton:

    def __init__(self, fruit: str):
        self.fruit = fruit

Not a lot of refactoring that can be done here, I guess, so let’s move on to the next step. Now that we have warmed up, it’s time to take a shot at making this class a proper singleton. And here’s where things get interesting…

Typically, a singleton is created by ‘hiding’ the constructor (i.e., making it private), but Python does not allow us to do that. So, we need to find a way around it.

What we can do, though, is creating a class variable instance that contains the instance, and assign the current instance of the class to it whenever the __init__() function is called for the first time. A test for this would simply have to invoke the __init__() method by calling MySingleton() and then check that the instance class variable is not equal to None:

def test_singleton_instance_is_stored_as_a_class_variable():
    my_singleton = MySingleton(fruit='pear')
    assert my_singleton.instance is not None

We can implement this requirement like this:

class MySingleton:

    instance = None

    def __init__(self, fruit: str):
        MySingleton.instance = self

But hey… didn’t we forget something here? Yes, indeed, we also need to make sure once more that the fruit field of our single instance is set and can be read. Here’s the test:

def test_singleton_class_variable_exposes_properties():
    my_singleton = MySingleton(fruit='apple')
    assert my_singleton.instance.fruit == 'apple'

and here’s the implementation:

class MySingleton:

    instance = None

    def __init__(self, fruit: str):
        MySingleton.instance = self
        MySingleton.instance.fruit = fruit

Phew. Close call! Now would be a good time to check if there’s any refactoring we can do on our class.. Nope, so far so good I think!

Next, I’d like to add another requirement: because we can’t hide the __init__() function, I’d like to get notified whenever a process calls this function after the singleton has been created. You never know who’s going to consume your class.

I could choose to simply ignore any subsequent calls to __init__(), but instead, I’d like to be explicit and raise a RuntimeError whenever __init__() is called after our instance has been created.

Here’s what a test for that could look like (notice that testing for raised errors is straightforward with pytest):

def test_singleton_cannot_be_instantiated_twice():
    MySingleton(fruit='grape')
    with pytest.raises(RuntimeError) as re:
        MySingleton(fruit='pineapple')
    assert str(re.value) == 'Already instantiated!'

and here’s the implementation to make our test pass:

class MySingleton:

    instance = None

    def __init__(self, fruit: str):
        if self.instance is not None:
            raise RuntimeError('Already instantiated!')
        MySingleton.instance = self
        MySingleton.instance.fruit = fruit

As a final requirement, instead of allowing consuming classes to refer to the instance class variable directly, I’d like to encapsulate access to it in an instance() method, so that in the future, we can put additional logic or validations in there if required.

Because we might want to allow for class modification in this method in the future, it’s best to make this method a class method, as static methods are restricted in what data they can access.

We have to test for a couple of things here:

  1. The instance() method should return the initial instance
  2. The instance() method should allow access to the fruit field, and
  3. The instance class variable (renamed to __instance) should no longer be directly accessible (since that would break the encapsulation)

Here are the tests that check whether all the above conditions are satisfied:

def test_singleton_retains_initial_property_values_after_subsequent_init_calls():
    MySingleton(fruit='lychee')
    with pytest.raises(RuntimeError):
        MySingleton(fruit='orange')
    assert MySingleton.instance().fruit == 'lychee'

def test_singleton_instance_is_accessible_using_class_method():
    MySingleton(fruit='raspberry')
    my_singleton_instance = MySingleton.instance()
    assert my_singleton_instance.fruit == 'raspberry'

def test_singleton_instance_field_is_not_directly_accessible():
    with pytest.raises(AttributeError) as ae:
        MySingleton(fruit='peach').__instance
    assert str(ae.value) == "'MySingleton' object has no attribute '__instance'"

and here’s the final implementation of the MySingleton class:

class MySingleton:

    __instance = None

    def __init__(self, fruit: str):
        if MySingleton.__instance is not None:
            raise RuntimeError('Already instantiated!')
        MySingleton.__instance = self
        MySingleton.__instance.fruit = fruit

    @classmethod
    def instance(cls):
        return cls.__instance

Great! Our singleton is ready for use (again, I don’t think this needs a lot of refactoring at the moment), and we’ve got a number of tests to go with it, all thanks to a little bit of ‘thinking out loud’:

What do I want my code to do? How can I test for that?

For me, this has been a useful first go at applying TDD to write production code. I’ve certainly learned a lot in the process. I’m not sure if I’m going to use it all the time, but I am definitely going to practice it some more in those cases where I can’t exactly figure out how to implement something right away.

All code snippets and tests are available on my GitHub page, feel free to check them out here.

P.S. I’m looking for my next opportunity as a freelance Python developer (relatively new but learning fast), or as a freelance test automation trainer or engineer (experienced), from mid June 2020 onwards. I’m currently working on a Python development project and would definitely love to continue down this path. Feel free to get in touch or refer me to somebody you know who might be hiring. Thanks!

Writing tests for RESTful APIs in Python using requests – part 4: mocking responses

In this short series of blog posts, I want to explore the Python requests library and how it can be used for writing tests for RESTful APIs. This is the fourth blog post in the series, in which we will cover working mocking responses for unit testing purposes. Previous blog posts in this series talked about getting started with requests and pytest, about creating data driven tests and about working with XML-based APIs.

One thing that has been keeping me busy in the last couple of months is improving my software development skills. I’m currently working on a Python development project, and one of the tasks of a developer is writing good unit tests. When I was writing these tests, I ran into a challenge when I wanted to test a method that involves communicating with a REST API using the requests library.

Obviously, I don’t want to have to invoke the API itself in my unit tests, so I was looking for a way to mock out that dependency instead. One thing I considered was writing mocks for the API myself, until I stumbled upon the responses library (PyPI, GitHub). According to their homepage, this is ‘A utility library for mocking out the requests Python library’. Exactly what I was looking for.

So, what can you do with the responses library, and how can you use to your advantage when you’re writing unit tests? Let’s look at a couple of examples that involve creating mock responses for the Zippopotam.us API.

Creating a mock response
Let’s say that in our unit test, we want to test that our code handles an HTTP 404 returned by a REST API dependency as expected. This implies we need a way to ‘override’ the actual API response with a response that contains an HTTP 404 status code, and (maybe) a response body with an error message.

To use the responses library to create such a mock response, you’ll first have to add the @responses.activate decorator to your test method. In the test method body, you can then add a new mock response as follows:

@responses.activate
def test_simulate_data_cannot_be_found():
    responses.add(
        responses.GET,
        'http://api.zippopotam.us/us/90210',
        json={"error": "No data exists for US zip code 90210"},
        status=404
    )

When you use the requests library to perform an HTTP GET to http://api.zippopotam.us/us/90210, instead of the response from the live API (which will return an HTTP 200), you’ll receive the mock response, instead, which we can confirm like this:

response = requests.get('http://api.zippopotam.us/us/90210')
assert response.status_code == 404
response_body = response.json()
assert response_body['error'] == 'No data exists for US zip code 90210'

You can add any number of mock responses in this way.

Unmapped responses
If, during testing, you accidentally hit an endpoint that does not have an associated mock response, you’ll get a ConnectionError:

@responses.activate
def test_unmatched_endpoint_raises_connectionerror():
    with pytest.raises(ConnectionError):
        requests.get('http://api.zippopotam.us/us/12345')

Simulating an exception being thrown
If you want to test how your code handles an exception being thrown when you perform an API call using requests, you can do that using responses, too:

@responses.activate
def test_responses_can_raise_error_on_demand():
    responses.add(
        responses.GET,
        'http://api.zippopotam.us/us/99999',
        body=RuntimeError('A runtime error occurred')
    )

You can confirm that this works as expected by asserting on the behaviour in a test:

with pytest.raises(RuntimeError) as re:
    requests.get('http://api.zippopotam.us/us/99999')
assert str(re.value) == 'A runtime error occurred'

Creating dynamic responses
If you want to generate more complex and/or dynamic responses, you can do that by creating a callback and using that in your mock. This callback should return a tuple containing the response status code (an integer), the headers (a dictionary) and the response (in a string format).

In this example, I want to parse the request URL, extract the path parameters from it and then use those values in a message I return in the response body:

@responses.activate
def test_using_a_callback_for_dynamic_responses():

    def request_callback(request):
        request_url = request.url
        resp_body = {'value': generate_response_from(request_url)}
        return 200, {}, json.dumps(resp_body)

    responses.add_callback(
        responses.GET, 'http://api.zippopotam.us/us/55555',
        callback=request_callback,
        content_type='application/json',
    )

def generate_response_from(url):
    parsed_url = urlparse(url).path
    split_url = parsed_url.split('/')
    return f'You requested data for {split_url[1].upper()} zip code {split_url[2]}'

Again, writing a test confirms that this works as expected:

response = requests.get('http://api.zippopotam.us/us/55555')
assert response.json() == {'value': 'You requested data for US zip code 55555'}

Plus, responses retains all calls made to the callback and the responses it returned, which is very useful when you want to verify that your code made the correct (number of) calls:

assert len(responses.calls) == 1
assert responses.calls[0].request.url == 'http://api.zippopotam.us/us/55555'
assert responses.calls[0].response.text == '{"value": "You requested data for US zip code 55555"}'

Using the examples for yourself
The code examples I have used in this blog post can be found on my GitHub page. If you download the project and (given you have installed Python properly) run

pip install -r requirements.txt

from the root of the python-requests project to install the required libraries, you should be able to run the tests for yourself. See you next time!

New open source workshop: writing tests for REST APIs in Python with requests

Just a quick update to let you all know that I’ve just released the first version of a brand new open source workshop.

If you’re looking to learn how to write tests for RESTful APIs in Python using the requests library, head on over to my GitHub page to find a free and open source workshop on this very topic.

So far, it contains five series of examples, exercises and the corresponding answers for you to try out. As with all the other open source workshops, you’re absolutely free to use it in any way you want. Share it, teach it to others, discuss it with coworkers, whatever you like.

The only thing I’m asking you is to share your experiences with me. Is there anything missing? How did you use this workshop and what did you think?

Have fun and happy learning.