My thoughts on who should create automation, and why there might be a more urgent problem at hand

Recently, there has been quite a bit of discussion on Twitter on what role should be responsible for writing test automation code. From what I’ve read, there are roughly two camps:

  • The camp advocating that developers should be made responsible for writing most, if not all automation code. Their main reason: developers know best how to code, they also know best how the code is structured and works internally, providing them with the best insight into which hooks in the code to leverage for automation (as opposed to blindly doing everything through the user interface).
  • The camp advocating that test automation requires dedicated automation engineers, especially for anything above the unit testing level. Their main reason: it takes specific skills to write and maintain good automation, skills that extend further than ‘just’ development skills.

At the end of last year, I published a blog post on roughly this topic. Rereading it, I still agree with my own opinion I described back then (which is a good thing!), but having thought and read about and discussed this topic some more in the last months, there are some subtle nuances (and one maybe not so subtle one) I’d like to make. Which, in turn, makes for a good topic for another blog post.

First of all, looking back at the question of ‘Who should be responsible for creating automation?’, I think the correct answer is not ‘developers’ or ‘test automation engineers’. The correct answer to this question should be ‘the development team’. This includes developers, automation engineers, testers, designers, the works. I think that deep down, everybody agrees on this (save maybe a grumpy old fashioned developer that thinks writing automation code is way below his standard). A slightly (yet also very) different question here is ‘Who should be primarily tasked with creating automation?’. That’s where the two camps I mentioned above diverge.

One of the catalysts of the recent discussion on this topic is this blog post from Alan Page. His blog post was based on a series of Tweets Alan sent, which were in turn extensively annotated by Richard Bradshaw in another blog post. Whether or not you agree with their respective standpoints, I think both blog posts are recommended reading material for anybody working in the test automation space. Most of you will probably have read it already, but if you don’t, make sure you do.

My opinion? In an ideal world, where developers have the time, the knowledge as well as the drive that’s required to create useful automation, the dedicated automation engineer might be going the way of the dinosaur. Personally, I don’t see that happening in the foreseeable future, though. And this opinion is backed up by what I see with most of the organizations I’ve worked with over the past year and a half (10+ in numbers, in case you’re wondering). In most teams, developers either lack the time (mostly a bad excuse), drive (also a bad excuse) or knowledge (this is excusable but should be fixed anyway) to concern themselves with creating automation. The same goes for their testers.

As a result, they rely upon their own automation engineers to help them create, run and maintain their automation, or they hire someone from outside the organization. This is where I often come in, either to create the automation myself and learning employees how to extend, run and maintain it, or in a mentoring or coaching role, where I observe and help teams define their own automation strategy. And the number of projects that I see advertised (either directly to me or on freelance job boards and email lists) does not indicate a decline in the need for dedicated automation engineers either. Rather the contrary.

In the end, though, it does not (yet) matter to me WHO is tasked with creating and maintaining automation. Even stronger put, I don’t think it’s the most important problem (or discussion) that needs to be tackled with regards to automation, at the moment. Instead, I’d love to see more discussion, teaching and mentoring on what constitutes good automation, and how to implement automation in a way so that it is maintainable, reliable and valuable in the long run.

I don’t know if it’s just the time of the year, or the fact that I keep getting passed exactly the wrong (or right, depending on how you look at it) code bases, but in the last couple of weeks I’ve witnessed some truly horrifying pieces of automation cr*p. Selenium scripts where every third line was a Thread.sleep(). Cucumber step definition methods containing Selenium object locators. Horrible variable names (what in the name of Peter is ‘y’ supposed to tell me?). Writing a new Selenium test case for every possible combination of input and output parameters, even though the sequence of sendKeys() and click() actions stays exactly the same. And much more of such goodness.

Arguably the biggest gripe I have with this: these abominations were created by external consultants. Who had been working on them for months. And probably got paid a handsome hourly fee by their (and my) client for their efforts. That makes the problem at hand twofold:

  • Bad thing: there are too many self proclaimed ‘automation consultants’, ‘architects’, even the ‘senior’, ‘principal’, or Peter knows what else versions of them, that couldn’t write a decent test if their life depended on it.
  • Even worse thing: their clients don’t have the time or knowledge (probably the latter) to take a look and recognize what absolute garbage those expensive ‘consultants’ deliver.

Now that software development and delivery cycles are speeding up ever more, and teams are increasingly relying on automation to safeguard the quality of the releases they’re putting out into the world, it’s becoming due time to do something about this. Educate both the people responsible for creating automation and the teams that rely on their efforts about what constitutes good automation, and give them the tools to monitor and act upon automation quality. If we as test automation crafts(wo)men don’t start doing this sooner rather than later, I’m afraid that crappy automation becomes the new bottleneck in modern software development, just like all that testing was at the end of waterfall projects in times past.

Once we’ve tackled that problem, let’s move on to who’s the best fit to write what we agree upon is good automation.

9 thoughts on “My thoughts on who should create automation, and why there might be a more urgent problem at hand

  1. Bas,
    Again, great post. Totally agree with your points made. And I’ll just say these problems/issues have been around for a long long time. History has a nasty way of repeating itself.

    Two things I want to be a little more specific on. First, looking at Richard’s blog post I can only say that in respect to his term of “Test Automation” I’ve come to use the term “Automated Test Execution” instead. A little more specific in its meaning. And that is because there are other tools used in testing to automate (aid) the tasks we do as part of our work. That is why I’ve become more general in scope when saying test automation is the use of any tool in relation to our work. Thus we have tools that help to automate the processes of Test Management, Test Data Generation and so on in addition to the tools that “execute” our tests/checks (and the coding/programming we do to make it happen). This is just another form of the difference of opinions in terminology in our profession. Don’t get me started on Bug’s, er… I mean Defects, ah… I mean Issues. Oh screw it, whatever!

    Second, regarding the “Consultants” and other names people will call themselves (myself included) who couldn’t code themselves out of a wet paper bag (not including myself) they will always exist and will do damage. It happens in all industries and disciplines within them. The good ones (like you) will standout and the others who are Fakers/Poser’s will be weeded out eventually. It comes down to education of the “other” people (outside of Testing, and some within who don’t know better) so that they make educated and informed decisions in order to find the right and competent person for the job/work.

    I don’t like cleaning up other peoples messes, but sometimes that is what you do.

    Again, great job. Looking forward to what you, Richard and Alan write about next.

  2. Hi Bas,

    I’ve been following your blog for a few weeks now. They are always very well written and great insights, so thank you.

    As for this topic: I totally agree that test automation should always be everybody’s responsibility AND that it definitely IS software development.

    But I do think the best person to create automation is still a dedicated test automation engineer. Someone who has both good knowledge of testing but ALSO the necessary development skills. This could be a developer with the correct mindset and wiling to learn more about testing. Or someone like me, a tester who really likes to program and knows enough about OOP and design patterns to setup good test automation.

    Alternatively, it coud be a team effort where the developer works alongside the tester, the tester delivering the design of the automation software and the developer building it. I personally think that it is just more efficient to have both roles in the same person.

    Just some thoughts I have on the subject. I guess there really isn’t a true right and wrong answer. It’s more about what works best in a specific organisation and project.

    • Hey Michiel,

      long time no see!

      Thank you for sharing your insights. I totally agree, it’s really a matter of what works best for a team given their skill set, priorities, you name it.

  3. This still leaves QA Engineers in a tricky position. How do we stay employed in job markets where heavy automation and DevOps rule? That automation engineer rung is one I have been trying to hang on tight to. I hope I manage to climb up the next rung before it gets sawwed off!

    • Hey T.J., again: I don’t think automation engineers that are able to write good (as in maintainable, readable, transferable automation that doesn’t try to do stuff that automation is not meant for) will have a problem for years to come. Same goes for testers (I’m not sure what is meant by QA engineer) that are able to do things beyond writing and executing test cases based on requirements.

  4. At the moment our automation is coded by me (Dev Manager with a background in dev – many years ago – and test). Its something we need to address as we are rapidly changing and expanding our product.

    • Hey Chrissi,

      that’s one I haven’t heard before, a dev manager creating automation. Agreed, once your product grows automation quickly becomes a full time job, not something you can easily tack on your existing responsibilities 🙂

      • There are a couple of ways you can tackle this.

        • By reading the file into a String object and then doing given().body(yourString).when().post() etc.
        • By reading the file into a JSON object, then serializing it into a String and doing the same as above. Added value of doing this is that it might be easier to manipulate values before doing the POST.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.