On automation implementation frustration

Recently (as in, in the last couple of months) I’ve wrapped up a few automation projects that have left me less than satisfied. Not because there were no technical challenges – there were plenty. Not because I felt bored – if I’m getting bored, I simply move on. Not because I didn’t learn anything new – I’ve become a much better engineer in the last couple of months and learned lots about creating useful automation efforts.

No, my dissatisfaction was caused by something different. Something I should have seen coming. Something I, in retrospect, should have addressed earlier.

But before I explain what caused this dissatisfaction, first, let’s take a quick look at what a typical project I’m working on looks like. Usually, it starts with a client with a test automation-related challenge. Sometimes they’re just getting started. Sometimes they’ve tried some things already, only to see it fail. In any case, at some point in time, they decide it’s a good idea to hire me (maybe that’s where things go wrong, no comment.).

I then get to work, usually starting out by asking a lot of questions. What does the application under test do? What does the software development process look like? What do the testers do? Where’s the (perceived) risk? What do the stakeholders think to gain from introducing test automation? What have they tried already and why did it fail? All questions that are important for me when deciding on the best possible next step(s).

Then, I usually start getting involved in the actual automation. Sometimes that means building a brand new solution. Sometimes it’s training others to do so, or to maintain and extend what I built. In other projects, it’s running awareness workshops to remind people why they’re implementing test automation in the first place, and to help them get realistic about the expectations around it. Often, it’s a mixture of all of these activities.

I’m not one to boast, but most of the time, things tend to go well during the project. I’ve seen and written enough horrible automation in the past to recognize and know what works and what doesn’t, and as a result, most of the time, I’m able to figure out an approach that brings value to the development process instead of being a time and money drain. So, that’s not the problem. There IS a problem though, and that’s when I start wrapping up.

Too often, by the time I am preparing for my exit from the project, I get the feeling that a lot of the work I’ve done has been in vain. There are no tangible clues that support this feeling, but still, I sometimes just know that once I walk out of the office for the last time, the automation I’ve created will become shelfware. The most important reasons for that? Teams that do not see test automation as software development, and teams that continuously give priority to feature delivery, often pushed by management setting deadlines. The latter is especially cruel when those same organizations claim they ‘do Agile’ and by that are able to ‘deliver software faster’. Sure, that might work in the short term, but it’s not a strategy that will result in a sustainable pace of delivery in the longer term. But I digress.

Now, I’ll be the first one to (at least partly) blame myself for the test automation starting to gather dust once I’m gone. In retrospect, in these past projects I did things right, like deciding on what to automate before starting out, and deciding what approach would be the most effective in terms of coverage, speed of execution and maintainability. However, in some cases, I seem to have forgotten something even more important: creating the right level of awareness and setting the right expectations for the automation. Looking back, I should have made a bigger effort showing the teams and organizations I work with that test automation isn’t something you do once. Or when you have the time. Instead, it’s a software development project within a software development project, and therefore it should be treated as such.

Writing about this here and now means I’ve learned a valuable lesson. But more importantly, I hope to remind you to not make the same mistakes I made too often in the past, by just getting started without keeping the end in mind. I know I will do better in the future. I hope you do too. Start asking questions like ‘who will be responsible for maintaining and extending the automation once I’m gone?’, ‘how are we going to make and keep automation an integral part of our development process?’, and so on. Don’t repeat my mistakes. Start with the end in mind.

As I said, I learned my lesson here. In the project I’m working on at the moment, I am currently working hard at creating the right amount of awareness, and helping the organization decide who is going to take ownership of the automation solutions I create once I’m gone. I set my last day of working at this project on April 26th, so there’s plenty of time. But as with a lot of things, time flies, and making your exit as smooth and as fulfilling as possible isn’t something you can start doing two weeks before you’ll bring the goodbye cake. And this project has an additional complicating factor in that it is the first time that they are executing a software development project on this scale. It’ll make for an interesting three months, I’m sure. But I’m also sure that once I do say goodbye to this team, I know that the automation I delivered will be in good hands. I’m looking forward to walking away much more satisfied this time.

12 thoughts on “On automation implementation frustration

  1. Great blog! Thank you for sharing Bas.

    Just some things that i’m thinking of… Shouldn’t it be a team effort? Especially if the organisation that hired you pretends to be working agile? In my opinion the development team should take ownership, and not only you.

    “Currently you are working hard on awareness”. Are you doing this by giving demos, training? Which methods do you use to change certain mindsets?

    • Oh yeah, it should definitely be a team effort. That’s the outcome I work towards (and should have worked towards in the less than successful projects I described). I often work with teams and organizations that do not have the required experience to treat it as a team effort, though, or that do not even know yet that it should be a team effort. But you’re absolutely right.

      I try to create awareness in many different ways. Presentations with tangible results to the board (budget holders). Including automation demos in the end-of-sprint demos (not every sprint, but doing this regularly helps). Showing people on a daily basis what I work on, and asking them to join in. Doing (more or less) ‘formal’ training. It’s often a combination of these activities that gets the result.

  2. I’ve been learning a lot from sales people lately. And one of the things I’ve heard from multiple unique people now is: charge a premium for your services.

    I’m not going to ask how much you charge, but if there’s any possibility that they don’t feel like using your stuff because it was a small investment, then charging a lot more may spur them to use it.

    If they have more money invested in it, they’ll make it work, and to do that they have to use it.

  3. If you frequently get hired because the top management decided they needed “more quality” and “automation” is “the way” to get there, than what you share is not that surprising. Quality is just another “project”. You do the project and then you are done.

    If this is the case, what is the dev team motivation to continue supporting the effort? Even if it is obviously the right thing to do. There will be resistance. It was not their decision, not their project. Management could think everything is “done”.

    This could have a long-term bad impact on all involved – team, management and you.

    Sounds a bit grim, but alas I am speaking from experience here 🙁 And again from experience – looks like what you are doing is exactly what you should do, so I believe everything will be fine in the end this time.
    Share what happens if you can, it will be helpful 🙂

  4. Bas,
    I know how you feel. Been through this myself over the years on many projects. I always try to emphasize the need to consider “Life after Jim leaves” for the automation. You do your best to start with this in mind and get people to buy-in to it, and as you exit out the door remind them of the needs for continuing maintenance and to treat it as its own software development project.

    Sometimes though it doesn’t happen that way. Why, multiple reasons. But you have to sometimes shrug your shoulders, keep your head up and move on. It’s like having your child leave the house. You hope you set them up to succeed and continue on. But you have to let go and let them fail & learn on their own.

    The only thing I can suggest is getting the client to agree to a “check up” 3 months down the road. That way you can keep an eye on your kid, and make sure that things are going along the way you hoped. And if not, then walk away and don’t worry as its not your problem anymore. You did your duty.

    Trust me, I’ve had to do that.

    • I know, but I still struggle to come to terms with that. I’ve made it a habit to check up with my clients after the fact (as you said, a couple of months down the line). Thanks for the suggestion!

      • Bas,
        I can understand you still struggling with this. It’s called pride in your work, and we all deal with that. You work hard, do good work and are proud of it. But again, don’t take it too personal if they client goes and screws it up long after you’re gone.

        Just take satisfaction in the idea that you did your best and set them up for success, and that if they blow it later on it isn’t your fault. As I said, you did your duty. Take pride in that. It’ll keep you sane.

  5. Pingback: Java Testing Weekly 5 / 2018

  6. I missed this article! Yes, definitely creating a certain level of awareness is vital in the integration of test automation in the software development project itself…

    Thanks for the insights, Bas. Currently, I am the only one in charge of the QA process but having this in mind would definitely be a great way to go on!

  7. Great!!!
    I totally agreed.
    Maintaining test automation must require a lot of effort than expected. If ownership is not clear, test automation soon will die.
    Test automation should not cover everything. Setting proper coverage, for maintenance, should be considered.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.