On crossing the bridge into unit testing land

Maybe it’s just the people and organizations I meet and work with, but no matter how active they’re trying to implement automation and involve testers therein, there’s one bridge that’s often too far for those that are tasked with test automation, and that’s the bridge to unit testing land. When asking them for the reasons that testers aren’t involved in unit testing, I typically get one (or more, or all) of the following answers:

  • ‘That’s the responsibility of our developers’
  • ‘I don’t know how to write unit tests’
  • ‘I’m already busy with other types of automation and I don’t have time for that’

While these answers might sound perfectly reasonable to some, I think there’s something inherently wrong with all of them. Let’s take a look:

  • With more and more teams becoming multidisciplinary, we can’t simply shift responsibility for any task to a specific subgroup. If ‘we’ (i.e., the testers) keep saying that unit testing is a developer’s responsibility, we’ll never get rid of the silos we’re trying to break down.
  • While you might not know how to actually write unit tests yourself, there’s a lot you CAN do to contribute to their value and effectiveness. Try reviewing them, for example: has the developer of the unit test missed some obvious cases?
  • Not having time to concern yourself with unit testing reminds me of the picture below. Really, if something can be covered with a decent set of unit tests, there really is no need to write integration or even (shudder) end-to-end tests for it.

Are you too busy to pay attention to unit testing?

I’m not a devotee of the test automation pyramid per se, but there IS a lot of truth to the concept that a decent set of unit tests should be the foundation of every solid test automation effort. Unit tests are relatively easy to write (even though it might not look that way to some), they run fast (no need for waiting until web pages are loaded and complete their client-side processing, for example..) and therefore they’re the best way to provide that fast feedback that development teams are looking for those in this age of Continuous Integration / Delivery / Deployment / Testing / Everything / … .

To put it in even bolder terms, as a tester, I think you have the responsibility of familiarizing yourself with the unit testing activities that your development team is undertaking. Offer to review them. Try to understand what they do, what they check and where coverage could be improved. Yes, this might require you to actually talk to your developers! But it’ll be worth it, not just to you, but to the whole team and, in the end, also to your product and your customers. Over time, you might even write some unit tests yourself, though, again, that’s not a necessity for you to provide value in the land of unit testing. Plus, you’ll likely learn some new tricks and skills by doing so, and that’s always a good thing, right?

For those of you looking for another take on this subject, John Ruberto wrote an article called ‘100 percent unit test coverage is not enough‘, which was published on StickyMinds. A highly recommended read.

P.S.: Remember Tesults, the SaaS solution for storing and displaying test results I wrote about a couple of months ago? The people behind Tesults recently let me know they now offer a free forever plan as well. So if you were interested in using their services but could not afford or justify the investment, it might be a good idea to check their new plan out here. And again, I am in no way, shape or form associated with, nor do I have a commercial interest in Tesults as an organization or a product. I still think it’s a great platform, though.

How to move up from being good to being great?

Summer holidays.. A perfect time for taking a step or two back from work and reflect on what I’ve been doing this year, work-wise. Having just returned from my vacation, I can confirm that this year has been no different. One theme (or rather, one question) kept coming back over and over again this time:

How do I go from being ‘good’ to being ‘great’?

I consider myself to be a decent, sometimes maybe even a good consultant / writer / teacher / whatever it is I’m doing that day. While this is enough to be pretty fully booked at the moment, and even have the luxury of being able to say ‘no’ to tasks or projects I don’t really want to take on, I feel I can do better in many ways. It doesn’t feel like I’m failing, but there’s a lot of things that I feel I can improve.

Here’s a(n incomplete) list of things I’d like to accomplish on my journey from good to great:

  • Improving my consulting skills
  • Improving my public speaking skills
  • Improving my teaching skills
  • Improving my writing skills

These improvements would, in turn, hopefully lead to the following results (and who knows what else):

  • I’d like to grow a (small) back log of clients that I keep in close touch with and that I can help with their automation on a periodic basis, moving away from the hourly billing model towards a daily fee or a retainer
  • I’d like to speak at more conferences, and not just at testing conferences (no need for siloing knowledge sharing when the whole world seems to consist of multidisciplinary teams nowadays)
  • I’d love to get more bookings for my training and workshop offerings
  • I’d like to write and publish a short (e)book containing my views on the test automation craft

Achieving these goals is only part of the story, though. What would happen if I accomplished the above tasks? Would I somehow automatically start to see myself as a great consultant, rather than ‘just’ a good one? Being the cynical and self-criticizing b*st*rd that I can be, will I ever consider myself being more than half decent? I’m not exactly sure, but I’d like to think that hitting the above targets moves me in the right direction, at least.

One thing that struck me when reviewing the list, though, is that they’re all totally unrelated to getting more knowledgeable with regards to specific test automation tools. I just don’t think that that’s going to get me much closer to what I want to be. Of course, tools are important (no test automation without tools), but I don’t think that’s key to the step I want to take.

According to you, readers of this blog, how do you think one moves up from ‘good’ to ‘great’? I’d love to see your input on this matter! In the meantime, I will start to reach out to people that I consider to be on the ‘great’ level, see how they got to where they are now and what I can learn from them. Call it looking for a mentor (or two). In time, hopefully I can be a mentor myself. I’d love to be able to do that and teach others what I know.

If there’s one thing that my time off work has made me realize this year, though, it’s that there’s so, so many things that I don’t know or don’t master! I’m already looking forward to what the future will bring.

Tackle the hard problems first

When you’re given the task of creating a new test automation solution (or expanding upon an existing one, for that matter), it might be tempting to start working on creating tests right away. The more tests that are automated, the more time is freed to do other things, right? For any test automation solution to be successful and, more importantly, scalable, you’ll need to take care of a number of things, though. These may not seem to directly contribute to the coverage you’re getting with your automated tests, but failing to address them from the beginning will likely cause you some pretty decent headaches later on. So why not deal with them while things are still manageable?

Run tests from within a CI pipeline
If you’re like me, you’ll want to able to run tests on demand, which could be either on a scheduled or on a per-commit basis. You likely do not accept the ‘works on my machine’ attitude from a developer, do you? Also, I’ve been ranting on about how test automation is software development and should be treated as such, so start doing so! Have your tests uploaded to a build server and run them from there. This will ensure that there’s no funny stuff left in your test setup, such as references to absolute (or even relative) file paths that only exist on your machine, references to dependencies that are not automatically resolved, and so on. Also, from my experience, especially user interface-driven tests always seem to behave a little differently with regards to timing and syncing when run from a build server, so running them from there is the ultimate proof that they’re stable and can be trusted.

Take care of error handling and stability
Strongly related to the previous one is taking care of stabilizing your test execution and handling errors, both foreseen and unforeseen. This applies mainly to user interface-driven tests, but other types of automated tests should not be exempt of this. My preferred way of implementing error handling is by means of wrapper methods around API calls (here’s an example for Selenium). Don’t be tempted to skip this part of implementing tests and ‘make do’ with less than optimal error handling. The risk of spending a lot of time getting it right later on, having to implementing error handling in more than one place and spending a lot of time finding out why in the name of Pete your test run failed exactly are just too high. Especially when you stop running tests on your machine only, which, again, you should do as soon as humanly possible.

Have a solid test data strategy
In my experience, one of the hardest problems to get right when it comes to creating automated tests is managing the test data. The more you move towards end-to-end tests, and the more (external) dependencies and systems involved, the harder this is to get right. But even a system that’s operating in a relatively independent manner (and these are few and far between) can cause some headaches, simply because their data model can be so complex. No matter what the situation is you’re dealing with, having the right test data available at the right time (read: all the time) can be very hard to accomplish. And therefore this problem should be tackled as soon as possible. The earlier you think of a solid strategy, the more you’ll benefit from it in the long run. And don’t get complacent when everything is a-ok right now. There’s always the possibility that you’re simply working on those tests that do not yet need a better thought out test data approach, but that doesn’t mean you’re safe forever!

Get your test environment in order
With all these new technologies like containers and virtual machines readily available, I’m still quite surprised to see how hard it is for some organizations to get a test environment set up. I’m still seeing this taking weeks, sometimes. And then the environment is unavailable for hours or even days for every new deployment. Needless to say that’s very counter-effective when you want to be able to run your automated tests on demand. So my advice would be to try and get this sorted out as soon as possible. Make sure that everybody knows that having a reliable and available test environment is paramount to the success of test automation. Because it is. And since all modern systems are increasingly interconnected and ever more depending on components both inside the organization and beyond those walls, don’t stop at getting a test environment for your primary application. Make sure that there’s a suitable test environment available and ready on demand for all components that your integration and end-to-end tests touch upon. Resort to service virtualization when there’s no ‘real’ alternative. Make sure that you can test whenever you want.

In the end, writing more automated tests is never the hard problem. Making sure that all things are in place that enable you to grow a successful test automation suite is. So, tackle that first, while your suite is still small, and increasing the coverage of your automated tests will become so much easier.