Automated tests do not improve your testing process. Testers do.

‘Is manual testing going to go the way of the dinosaur?’

‘Are manual testers becoming obsolete?’

Just a few ways of phrasing concerns that have been raised a lot in the testing community in the past few months, possibly even years. With software delivery and test cycles becoming ever shorter, organizations are depending more and more on automated testing to deliver software that has at least an acceptable level of quality.

Testers not involved or even interested in automated testing (‘manual testers’, for lack of a better description) might start to worry whether their job is still secure. I would like to argue that it is.

The first and foremost reason is that automated tests do not improve the quality of a product, nor do they improve the quality of your testing process. Automated test tools are very useful for exactly one thing: quick and/or unattended execution of a number of predefined checks.

If those automated checks are of poor quality, you’ll get poor quality results. You’ll just get them faster. So while automated tests may speed up the testing process, they’ll never improve it.

However, testers do.

If your automated tests only check irrelevant things, your test tool won’t tell you.

Testers do.

Your automated test tool does not automatically generate tests for brand new software components (most of the time they don’t, at least).

Testers do.

Your automated tests do not discuss features with developers and business representatives to see whether the specifications are complete and unambiguous.

Testers do.

Your automated tests do not think ‘hey, what would happen if I do XYZ instead of ABC?’

Testers do.

I could go on for a while, but in short, there is still a lot of work to do for testers.

You won’t get away much longer with following the ‘old world’ process of waiting for specifications – writing test plans – writing test cases – executing test cases – reporting bugs – writing test reports. But you don’t have to fear that there is no place for you in the software development process.

And of course, you could always venture into test automation..

18 thoughts on “Automated tests do not improve your testing process. Testers do.

  1. Pingback: Reading Recommendations # 30 | Adventures in QA

  2. Pingback: Testing Bits – 8/16/15 – 8/22/15 | Testing Curator Blog

  3. Hi Bas,,,
    I seen your post ” Automation tests do not improve your testing process ” , i feel to ask one question , what is the process to follow in automation testing , and what are the benefits of using automated tools and for doing automation testing , a tester must be both manual tester and should have knowledge on tools ? can you share your view ..?

    thanks
    reddy

    • Hey Reddy,

      I don’t think there is just one process that can be followed when it comes to automation. It really depends on so many factors (including but not limited to the kind of product you’re working on, the required level of quality and trust needed before release, etc.).

      Of course, there are obvious benefits that can be gained from introducing test automation to your testing process, but as I stated in another comment, it’s not a magic bullet.

      I think that as a person who is asked to do test automation, you can’t really do without actual testing skills. It doesn’t mean you have to be a tester, it’s perfectly fine for developers to do test automation (in fact, that would be great), but you need a testing (or better put, a quality) mindset in order to make the introduction of test automation a success.

  4. Hi

    Your automated tests do not think ‘hey, what would happen if I do XYZ instead of ABC?’

    Testers can automate this

    Your automated tests do not discuss features with developers and business representatives to see whether the specifications are complete and unambiguous.

    Testers call this executable specifications

    If your automated tests only check irrelevant things, your test tool won’t tell you.

    Testers can help automate all the important things

    Your automated test tool does not automatically generate tests for brand new software components (most of the time they don’t, at least).

    ever heard of test driven development?

    • Hi Maarten, thanks for your comments!

      > Testers can automate this

      True, but what I tried to say is that there isn’t really a test automation substitution for exploratory testing. Maybe I should have phrased it better..

      > Testers call this executable specifications

      Also true, but tools won’t create them for you.

      > Testers can help automate all the important things

      I don’t think it’s can, it’s should. Testers (or as Kishen put it so nicely, quality minded people) are obliged to help out with determining what should be checked by automated tests, I think.

      > ever heard of test driven development?

      Sure 🙂 Still, you need people to write the test cases..

  5. I partially agree with your blogpost.
    One thing keeps bugging me when I read it. Putting Testers on a pedastal – improving quality…
    Quality comes from quality minded people / teams

    • Hi Kishen,

      it has never been my intention to put testers on a pedestal with this post, I just wanted to stress the fact that automated tests are not the magic bullet to improving quality and improving the testing process. Using the term ‘testers’ for the people that are required to improve product quality is somewhat of a generalization, I agree. It is all about quality minded people and teams, you’re absolutely right on that.

  6. You are right in and very precise. Your article is a good answer to heated debates ragging testing forums.

    One thing though, manual testers will eventually need to warm up to automation. Though it will still be testers who will talk with stakeholders regarding business scenarios and risks and consider ‘what-if’ scenarios, a part of their job will also be to write the what-if-s in automation codes. Another way of looking into this is that manual testers, who may be using there fingers to count numbers, will use a calculator instead. The calculator will not work by itself, however, testers insisting on using their fingers will fall behind as well.

    • Hey Abhideep,

      I like the analogy with using a calculator. I agree knowing more about automation is a very valuable skill for any tester, no doubt. It’s just that there will always be a need for testers / people with a testing mindset. That mindset cannot be replaced by tools.

  7. Hi Bas,
    Usually how do the testers know how the an application or software or website to be tested works?i mean in the starting itself do testers ask developers about the full work flow of the whole project and make an idea of how to automate in mind and as soon as developers finish the first phase(if it is divided into phases),they start implementing.(of course only if there is a need to automate the project 🙂
    what happens in my company is the moment they ask me to manually test a website(half finished),i may not have the idea of the work flow of the website
    and then only i ask the developer about the work flow of the project and thinks well is there a point in automating this project.That also is ok right?

    • That’s OK, yes. Automate what needs to be automated 🙂 and do the rest manually. There’s always a need for manual testing. Often enough, there’s no need for automation.

Leave a Reply

Your email address will not be published. Required fields are marked *