On where I think the test automation industry is going

One of the funny things about running a blog for a while is that people start to see you as an expert. Whether or not this is a reasonable observation in my case, I’ll leave to answer for the rest of you. Still, they’ll turn to you with questions regarding their career, advice on that, and if I know where the industry is going. I won’t disclose my answers to the former here, since there’s no such thing as general career advice and I don’t feel comfortable sharing questions and answers asked by individuals on a public blog.

In this post, I’ll share a couple of things that tend to make up my replies to the question of ‘where is the industry going?’. You’ll see that hidden in there is some career advice anyway..

The need for good automation will keep growing
This is the answer to a question I get sometimes, and one I’ve been asking myself quite often as well. Is test automation a good field to be in for the foreseeable future? I tend to think that it is. Sure, the field will change due to new technologies, market trends and research breakthroughs, but in general, test automation will remain an important (though I don’t want to overstate the importance of it either) activity in software development. That is, good automation. Hopefully the industry will start to learn that there is a lot of horrible automation being written and maintained at the moment, and will see that we’ll all be better off if the plug is pulled on those efforts. I’ve seen some things.. I don’t even want to know how much money is wasted in these projects!

People that want to be good automation engineers will see that they need software development skills
As a logical follow-up to the previous point, I foresee that the industry will start to recognize that in order to write good automation, one needs actual software development skills. I expect (or at least sincerely hope) that the end of creating automation that is hard to use, explain and maintain is nigh. I myself will do my very best to create this awareness, even if that means standing on some people’s toes. It’s about time test automation is taken seriously, by all parties involved. So, for those that want to move into the automation field, make sure you’ve at least got a grasp of the basic concepts of software design and development. You’ll need it to succeed in the long term.

Automation turns out not to be the silver bullet
Repeat after me: automation is not the silver bullet. Sure, it can make things more effective if applied well. But if you automate horse shit, you’ll get automated horse shit. Just automating something because you can (or because someone asks you to, for those with less backbone or less ability to think critically) does not mean that you should. Again, I see more and more people starting to become aware of this, mostly due to others that have seen the truth earlier and are willing (or feeling obliged) to share. Thanks guys, keep up the great work!

Machine learning and artificial intelligence will not take your job
Sure, ML and AI are emerging trends. And sure, they might have an effect on your job and the way you do it at some point in time. But I refuse to believe that at least in the foreseeable future, ML and AI are going to replace me. Maybe they’ll replace some of my tasks. Writing yet another Page Object and tests that use it (in a valuable way!), for example. And you know what? I’d be happy if they did. It gets boring, after a while. This means that some of my time will be freed up to dedicate to more interesting stuff, such as teaching others, doing research, devising and implementing automation strategies, or becoming the world expert on tiramis├╣ I always wanted to be. I’m looking forward to it.

Despite numerous efforts, the software industry will not be able to get rid of testers
Lastly, and I hope this is the last time I ever need to refer to this fallacy (although I suspect it won’t be), those pesky testers will live to see another day.

On finding my ideal ratio between consulting, teaching and writing

Those of you that have been reading my blog posts for a while know that I like to reflect on my own career from time to time. Since it’s been a while since I wrote such a blog post, I thought it would be a good idea to share with you what I’m up to at the moment.

Basically, my working time is still divided into three subjects: consulting, teaching and writing. I’m continuously trying to find out what is the ideal ratio between each of these activities, something that’s partly under my own control, yet also depends on the amount of work that’s coming my own. Or that I am able to steer my way, of course.

Here’s a quick recap of my activities in each of these three fields.

Consulting
On site consulting still makes up the major part of my working week. Not sure how the situation is in the rest of the world, but here in the Netherlands there’s plenty of work available for those who know a little about test automation and have some communication skills too, so getting projects isn’t too hard at the moment. I have to say ‘no’ more often than I can say ‘yes’!

That’s a very luxurious position to be in, I definitely realize that. And yet.. It’s safe, since I am guaranteed an income for at least 25-30 hours per week (it’s all billed by the hour), but I feel it does also make me complacent at times. This might sound strange (or spoiled), but I could actually do with a little less consulting work, but that would require having more work in the other two categories. Either that, or seeing a significant (but hopefully temporary) fall in my income, a prospect which I don’t particularly look forward to.

Teaching
It’s been too long, but I finally had (or created, depending on how you look at things) another opportunity to deliver on site training, this time with my former employer. I spent two evenings with around 10 students, teaching them about the concepts behind API testing and automation and introducing them to a range of tools (more on that training course and the approach I experimented with here). This reminded me how exciting and motivating it is to deliver training, and that it definitely is something I feel I should pursue harder. Ideally, I’d do at least two or three of these courses a month, but I’m nowhere near that frequency yet.

I have started working on a related project, though. It’ll be an online course around Selenium, which will hopefully see the light of day in the coming months.

Next to that, I’m actively working on finding more ways to deliver on site training, both at clients here in the Netherlands as well as at conferences. This is a slow process, but I’ve made some good connections in the past few months and I’m positive this effort will pay off soon enough.

And while we’re on the subject of conferences: I’ve got two talks coming up this month. First, I’ll be at the fall conference of the Dutch testers association TestNet (site in Dutch), and later this month it’s time for TestBash Manchester. Really looking forward to speaking at and being a part of both of these events!

Finally, I was a panelist at a webinar hosted by the people at Testim, where we talked about creating an automation strategy fit for CI/CD and the skills required to do so. For those of you that missed it, you can find a recording here.

Writing
This one I’ve actively put on the back burner for a while. In the past few months, I’ve written quite a few articles for TechBeacon, StickyMinds and some other one-off blog posts, next to my weekly blog post on this site, of course. That spread me a little thin so I decided to stick with my weekly OnTestAutomation blogs for a while.

I am currently working on something related to writing, though: I’m the technical editor for a book on test automation that’s to be released in the first half of next year. This is something I’ve never done before, but that’s only a good thing.

The gist of this is that I could do with a little more teaching and a little less consulting and that I’m actively working on making that happen. As ever, it’s an interesting journey.

Why there’s no such thing as codeless automation

In today’s blog post – which, again, is really nothing more than a thinly veiled rant – I’d like to cover something that’s been covered before, just not by me: codeless test automation and why I think there isn’t and should not be such a thing.

I’ve seen numerous ‘solution’ vendors advertise their products as ‘codeless’, implying that everybody in the team will be able to create, run and maintain automated tests, without having to, well, write code. I’ve got a number of problems with selling test automation in this way.

It’s not codeless. It’s hiding code.
The first gripe I have with ‘codeless’ automation is a semantic one. These solutions aren’t codeless at all. They simply hide the code that runs the test from plain sight. There are no monkeys in the solution that magically execute the instructions that make up a test. No, those instructions are translated into actual code by the solution, then executed. As a user of such a solution, you’re still coding (i.e., writing instructions in a manner that can be interpreted by a machine), just in a different syntax. That’s not codeless.

While it might be empowering, it’s also limiting.
Sure, using codeless tools might potentially lead to more people contributing to writing automated tests (although from my experience, that’s hardly how it’s going to be in the end). The downside is: it’s also limiting the power of the automated tests. As I said above, the ‘codeless’ solution is usually nothing more than an abstraction layer on top of the test automation code. And with abstraction comes loss of detail. In this case, this might be loss of access to features of the underlying code. For example, if you’re using a codeless abstraction on top of Selenium, you might lose access to specific waiting, synchronization or error handling mechanisms (which are among the exact things that makes Selenium so powerful).

It might also be loss of access to logging, debugging or other types of root cause analysis tools, which in turn leads to shallower feedback in case something goes wrong. While the solution might show you that something has gone wrong, it loses detail on where things went wrong and what caused the failure. Not something I like.

Finally, it might also limit access to hooks in the application, or limit you to a specific type of automated tests. If such a solution makes it potentially easier to write automated tests on the user interface level, for example, there’s significant risk that all tests will be written at that level, even though that might not be the most efficient approach in the first place. If all you’ve got is a hammer…

It’s doing nothing for the hard problems in creating maintainable automation.
Let’s face it: while writing code might seem hard to people that haven’t done it before, it actually isn’t that difficult once you’ve had a couple of basic programming classes, or followed a course or two on Codecademy. What is hard is writing good, readable, maintainable code. Applying SOLID and DRY principles. Structuring your tests. Testing the right thing at the right level. Creating a solid test data and test environment strategy. Those things are hard. And codeless test automation does nothing for those problems. As I tried to make clear in the previous paragraphs, it’ll often make it even harder to solve those problems effectively.

I’m all for creating solutions that make it easier to write, run and maintain automation. I hate people selling solutions as something they’re not. Codeless test automation is not going to solve your test automation problems. People that know

  • how to decide what good automation is
  • how to write that automation, and
  • how to pick the tools that will help them achieve the goals of the team and organization

will.