On asking ‘why?’ in test automation

This blog post is about something that’s been bothering me for a while, and that keeps coming up for some reason.. Whenever I talk to clients, see a discussion on LinkedIn or StackOverflow or read a blog post on something related to test automation, all too often it’s about ‘how can I solve problem X with tool Y’ (with tool Y suspiciously often being equal to Selenium). The word that buggers me most in this question is ‘how?‘. My knee-jerk reaction to a lot of these ‘how?’ questions I see is ‘why?’. Or more specifically: ‘why the &%$^* would you want to do that in the first place?’.

About half a year ago, I wrote a blog post related to this frustration of mine on LinkedIn. So far, it hasn’t changed the world, since I still see a lot of ‘how?’ where I think ‘why?’ would be a far better question. But since, as it is so eloquently said in Latin (although to me, all Latin sounds pretty eloquent):

“Repetitio mater studiorum est” (“Repetition is the mother of all learning”),

I think it’s worth repeating here as well: With all questions related to test automation, first ask yourself ‘why?’ before even thinking about the ‘how?’.

‘Why?’ prevents you from automation for automation’s sake
Before asking ‘how can I implement test automation most effectively?’, ask ‘why do I want to implement test automation in the first place?’. Implementation of test automation should be a conscious decision, motivated by tangible and significant benefits to the overall software development process (and in the end, to the business objectives of the organization), not an activity that is adopted just because it sounds cool, or because *shudder* everybody else is doing it.

‘Why?’ steers your efforts in the right direction
Before asking ‘how can I automate this test?’, ask ‘why do I want to automate this test in the first place?’. Don’t become the world’s best automator of useless tests. Instead, become the world’s best selector of useful tests to automate. Selecting those tests that give you the most valuable information about the quality and the risk associated with the application you’re developing and delivering can only be done by asking ‘why?’ first. Only after you’ve decided on the best possible set of tests, start exploring how you can automate those tests in the most effective way.

‘Why?’ makes sure you use your tools in the best possible way
Before asking ‘how can I use tool X to automate this test?’, ask ‘why should I use tool X to automate this test?’. In a recent blog post, I talked (or ranted) about abusing Selenium for API tests. Even though the blog post was meant to be – at least partly – satirical, I see similar things happening on a regular basis. Another big example is the use of Cucumber or SpecFlow as an automation tool.

In the surprisingly recent past, I’ve been guilty of the above mistakes myself too, by the way. No need to be elitist and pretend I know it all. I just hope that more people will start to think and ask the right questions before they automate.

On a final note, in test automation training too, a lot of attention is being paid to the ‘how?’, without the proper amount of focus on the ‘why?’, and subsequently on the ‘what?’. Since I’m a firm believer in ‘practice what you preach’, I’ve started to develop training material that I think will contribute to asking the right questions in test automation. I believe that doing so will always lead to better test automation in the end. I hope to be able to present this training material to you, and to the general test automation and testing community, somewhere early next year (creating course material takes time!).

May next year be the year of the ‘why?’ in test automation, not just of the ‘how?’.

12 thoughts on “On asking ‘why?’ in test automation

  1. Thank you for an interesting blog post.
    I am observing the same situation with questions “How..” among a lot of junior QA / QA Automation engineers.
    I think the main cause of these questions is the lack of information (in the form of books, courses, videos) about test automation efforts (why, how and when it’s useful). Additionally there is not so much materials regarding test automation architecture.
    As a result we get a lot of newcomer testers with only one possible way to work: automation, more automation, more tests…

  2. Bas,
    Good question. Why should a project “automate” its testing execution and other activities. This is a fundamental part of the 5 questions (Who, What, When, Where and Why). But it should be the first question asked, and as you pointed out it is not (if asked at all). Once those questions have been answered then the How can be determined. These questions, and their answers, are part of the foundation work that needs to be done before a tool is selected and a line of code is written.

    And I’ll agree (and make the statement) that a lot of companies do not determine why they want to implement automation for testing. They just knee jerk react and jump in blindly. This just leads to an accident waiting to happen.

    I always try to ask the Why question up front, because that way I can come to understand the perceptions (and misperceptions/misconceptions) a client has about this type of work. This allows me to properly set expectations and get them on the right path for success. Not only do they need to know why they want to do it (automation), but also do they really understand what it is they are signing up for.

    But yes, asking “Why” first is a smart thing to do and will help to keep things on track from the beginning.

    • Thanks again for your insights and comments, Jim! I think it’s pretty obvious that I agree. Hopefully more people will start thinking this way, I won’t shut up about it until they do.

  3. Spot on! I guess this post should really inspire teams to start asking “Why?” first and maybe reconsider their automation practices so they really add value as you mention here. I myself am still at some entry point into the wake automaton, but still find some how-to attempts at “automation” bizarre and lacking some serious grounds – the main reason for their use/implementation probably being just to show someone can use a tool (most often Selenium) or is still learning it, or to complete some sort of practice/homework assignment. On a second thought, though, I must say this automation abuse might stem from the fact it is currently being presented / marketed from a developer’s perspective in most cases – focus on tools and technologies, rather than the actual rationale, quality and value issues more akin to testing that you bring up (methinks).

    • Hey Tom,

      thanks for sharing your thoughts! Glad to read there are people that think the same way.

      And don’t fret the grammar correction, it’s all good!

      — Bas

  4. Pingback: Java Web Weekly, Issue 154 | Baeldung

  5. Pingback: Testing Bits – 12/4/16 – 12/10/16 | Testing Curator Blog

  6. Pingback: Java Testing Weekly 50 / 2016

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.