On finding my ideal ratio between consulting, teaching and writing

Those of you that have been reading my blog posts for a while know that I like to reflect on my own career from time to time. Since it’s been a while since I wrote such a blog post, I thought it would be a good idea to share with you what I’m up to at the moment.

Basically, my working time is still divided into three subjects: consulting, teaching and writing. I’m continuously trying to find out what is the ideal ratio between each of these activities, something that’s partly under my own control, yet also depends on the amount of work that’s coming my own. Or that I am able to steer my way, of course.

Here’s a quick recap of my activities in each of these three fields.

Consulting
On site consulting still makes up the major part of my working week. Not sure how the situation is in the rest of the world, but here in the Netherlands there’s plenty of work available for those who know a little about test automation and have some communication skills too, so getting projects isn’t too hard at the moment. I have to say ‘no’ more often than I can say ‘yes’!

That’s a very luxurious position to be in, I definitely realize that. And yet.. It’s safe, since I am guaranteed an income for at least 25-30 hours per week (it’s all billed by the hour), but I feel it does also make me complacent at times. This might sound strange (or spoiled), but I could actually do with a little less consulting work, but that would require having more work in the other two categories. Either that, or seeing a significant (but hopefully temporary) fall in my income, a prospect which I don’t particularly look forward to.

Teaching
It’s been too long, but I finally had (or created, depending on how you look at things) another opportunity to deliver on site training, this time with my former employer. I spent two evenings with around 10 students, teaching them about the concepts behind API testing and automation and introducing them to a range of tools (more on that training course and the approach I experimented with here). This reminded me how exciting and motivating it is to deliver training, and that it definitely is something I feel I should pursue harder. Ideally, I’d do at least two or three of these courses a month, but I’m nowhere near that frequency yet.

I have started working on a related project, though. It’ll be an online course around Selenium, which will hopefully see the light of day in the coming months.

Next to that, I’m actively working on finding more ways to deliver on site training, both at clients here in the Netherlands as well as at conferences. This is a slow process, but I’ve made some good connections in the past few months and I’m positive this effort will pay off soon enough.

And while we’re on the subject of conferences: I’ve got two talks coming up this month. First, I’ll be at the fall conference of the Dutch testers association TestNet (site in Dutch), and later this month it’s time for TestBash Manchester. Really looking forward to speaking at and being a part of both of these events!

Finally, I was a panelist at a webinar hosted by the people at Testim, where we talked about creating an automation strategy fit for CI/CD and the skills required to do so. For those of you that missed it, you can find a recording here.

Writing
This one I’ve actively put on the back burner for a while. In the past few months, I’ve written quite a few articles for TechBeacon, StickyMinds and some other one-off blog posts, next to my weekly blog post on this site, of course. That spread me a little thin so I decided to stick with my weekly OnTestAutomation blogs for a while.

I am currently working on something related to writing, though: I’m the technical editor for a book on test automation that’s to be released in the first half of next year. This is something I’ve never done before, but that’s only a good thing.

The gist of this is that I could do with a little more teaching and a little less consulting and that I’m actively working on making that happen. As ever, it’s an interesting journey.

Why there’s no such thing as codeless automation

In today’s blog post – which, again, is really nothing more than a thinly veiled rant – I’d like to cover something that’s been covered before, just not by me: codeless test automation and why I think there isn’t and should not be such a thing.

I’ve seen numerous ‘solution’ vendors advertise their products as ‘codeless’, implying that everybody in the team will be able to create, run and maintain automated tests, without having to, well, write code. I’ve got a number of problems with selling test automation in this way.

It’s not codeless. It’s hiding code.
The first gripe I have with ‘codeless’ automation is a semantic one. These solutions aren’t codeless at all. They simply hide the code that runs the test from plain sight. There are no monkeys in the solution that magically execute the instructions that make up a test. No, those instructions are translated into actual code by the solution, then executed. As a user of such a solution, you’re still coding (i.e., writing instructions in a manner that can be interpreted by a machine), just in a different syntax. That’s not codeless.

While it might be empowering, it’s also limiting.
Sure, using codeless tools might potentially lead to more people contributing to writing automated tests (although from my experience, that’s hardly how it’s going to be in the end). The downside is: it’s also limiting the power of the automated tests. As I said above, the ‘codeless’ solution is usually nothing more than an abstraction layer on top of the test automation code. And with abstraction comes loss of detail. In this case, this might be loss of access to features of the underlying code. For example, if you’re using a codeless abstraction on top of Selenium, you might lose access to specific waiting, synchronization or error handling mechanisms (which are among the exact things that makes Selenium so powerful).

It might also be loss of access to logging, debugging or other types of root cause analysis tools, which in turn leads to shallower feedback in case something goes wrong. While the solution might show you that something has gone wrong, it loses detail on where things went wrong and what caused the failure. Not something I like.

Finally, it might also limit access to hooks in the application, or limit you to a specific type of automated tests. If such a solution makes it potentially easier to write automated tests on the user interface level, for example, there’s significant risk that all tests will be written at that level, even though that might not be the most efficient approach in the first place. If all you’ve got is a hammer…

It’s doing nothing for the hard problems in creating maintainable automation.
Let’s face it: while writing code might seem hard to people that haven’t done it before, it actually isn’t that difficult once you’ve had a couple of basic programming classes, or followed a course or two on Codecademy. What is hard is writing good, readable, maintainable code. Applying SOLID and DRY principles. Structuring your tests. Testing the right thing at the right level. Creating a solid test data and test environment strategy. Those things are hard. And codeless test automation does nothing for those problems. As I tried to make clear in the previous paragraphs, it’ll often make it even harder to solve those problems effectively.

I’m all for creating solutions that make it easier to write, run and maintain automation. I hate people selling solutions as something they’re not. Codeless test automation is not going to solve your test automation problems. People that know

  • how to decide what good automation is
  • how to write that automation, and
  • how to pick the tools that will help them achieve the goals of the team and organization

will.

Why I think unit testing is the basis of any solid automation strategy

In a recent blog post I talked about why and how I still use the test automation pyramid as a model to talk about different levels of test automation and how to combine them into an automation strategy that fits your needs. In this blog post I’d like to talk about the basis of the pyramid a little more: unit tests and unit testing. There’s a reason -or better, there are a number of reasons- why unit testing forms the basis of any solid automation strategy, and why it’s depicted as the broadest layer in the pyramid.

Unit tests are fast
Even though end-to-end testing using tools like Selenium is the first thing a lot of people think about when they hear the term ‘test automation’, Selenium tests are actually the hardest and most time-intensive to write, run and maintain. Unit tests, on the other hand, can be written fast, both in absolute time it takes to write unit test code as well as relative to the progress of the software development process. A very good example of the latter is the practice of Test Driven Development (TDD), where tests are written before the actual production code is created.

Unit tests are also fast to run. Their run time is typically in the milliseconds range, where integration and end-to-end tests take seconds or even minutes, depending on your test and their scope. This means that a solid set of unit tests will give you feedback on specific aspects of your application quality much faster than those other types of tests. I stressed ‘specific aspects’, because while unit tests can cover ground in relatively little time, there’s only so much they can do. As goes for automation as a whole.

Unit tests require (and enforce) code testability
Any developer can tell you that the better structured code is, the easier it is to isolate specific classes and methods and write unit tests for them, mocking away all dependencies that method or class requires. This is referred to as highly testable code. I’ve worked in projects where people were stuck with badly testable code and have seen the consequences. I’ve facilitated two day test automation hackathon where the end goal was to write a single unit test and integrate it into the Continuous Integration pipeline. Writing the test took ten minutes. Untangling the existing code so that the unit test could be written? Two days MINUS ten minutes.

This is where practices like TDD can help. When you’ve got your tests in place before the production code that lets the tests pass is written, the risk of that production code becoming untestable spaghetti code is far lower. And having testable code is a massive help with the next reason why unit testing should be the basis of your automation efforts.

Unit tests prevent outside in test automation (hopefully)
If you’re code is testable, it means that it’s far easier to write unit tests for it. Which in turn means that the likelihood that unit tests are actually written increases as well. And where unit tests are written consistently and visibly, the risk that everything and its mother it tested through the user interface (a phenomenon I’ve seen referred to as ‘outside-in test automation’) is far less high. Just writing lots of unit tests is not enough, though, their scope, intent and coverage should be clear to the team as well (so, testers, get involved!).

Unit tests are a safety net for code refactoring
Let’s face it: your production code isn’t going to live unchanged forever (although I’ve heard about lines of COBOL that are busy defying this). Changes to the application, renewed libraries or insights, all of these will in time be reason to refactor your existing code to improve effectivity, readability, maintainability or just to keep things running. This is where a decent set of unit tests helps a lot, since they can be used as a safety net that can give you feedback about the consequences of your refactoring efforts on overall application functionality. And even more importantly, they do this quickly. Developers are humans, and will move on to different tasks if they need to wait hours for feedback. With unit tests, that feedback arrives in seconds, keeping them and you both focused and on the right track.

In the end, unit tests can, will and need not replace integration and end-to-end tests, of course. There’s a reason all of them are featured in the test automation pyramid. But when you’re trying to create or improve your test automation strategy, I’d advise you to start with the basis and get your unit testing in place.

By the way, for those of you reading this on the publication date, I’d like to mention that I’ll be co-hosting a webinar with the folks at Testim, where I’ll be talking about the importance of unit testing, as well as much more with regards to test automation strategy. I hope to see you there! If you’re reading this at a later date, I’ll add a link to the recording as soon as it’s available.