An approach to test your user interface more efficiently

As returning readers of this blog might have read before, I am pretty skeptical about writing automated tests that interact with the application under test at the user interface level. These UI tests tend to be:

  • slow in execution, since they are typically end-to-end tests from an application-layer perspective, and
  • demanding when it comes to maintenance, because the user interface is typically a part of an application subject to frequent changes.

However, the user interface often is an important component of an application and therefore it requires testing effort. Since this blog is all about test automation, I’d like to talk about a different approach to user interface test automation in this blog post.

But first, let me explain a subtle yet important difference. On the one hand, there’s testing the user interface. Here, the actual logic incorporated in the user interface is isolated and tested as a unit. Everything that’s happening ‘behind’ the user interface is out of scope of the test and therefore usually mocked. On the other hand, there’s testing application logic through the user interface. This is generally done using tools such as Selenium. This type of automated tests uses the user interface as its point of entry and validation, even though the actual logic that processes the input and generates the output to be checked is not implemented at the user interface layer at all.

Testing the user interface versus testing through the user interface

Now, only when you specifically want to perform end-to-end application tests, or when there really isn’t any other option than to use the user interface drive tests that validate underlying application logic should you resort to tools such as Selenium. In all other cases, it might be a better idea to see if there’s a better option available.

User interface architectures and testability
In the remainder of this post I want to zoom in on a commonly used user interface pattern, namely Model-View-ViewModel, or MVVM in short, and what I think is a suitable approach for writing automated tests for user interfaces adhering to this pattern.

MVVM explained (briefly)
The MVVM pattern lets you separate presentation logic (that defines what information is shown on the screen) from the actual presentation (that defines how the information is displayed). Schematically, the pattern looks like this:

A schematic representation of the Model-View-ViewModel pattern

The View is the user interface, which is made up of textboxes, labels, buttons, etc. It is responsible for defining the structure, layout and appearance of what you seen on the screen. The Model is an implementation of the domain model. It consists of a data model along with business rules and validation logic. The View Model is the intermediary between the View and the Model, handling user interface actions (such as the click of a button) and interacting with models usually by invoking methods in the model classes (for example to update a record in a database). It is also responsible for presenting data from the model (for example the results of a query) to the view in a representation that can be easily displayed on screen.

One of the key ideas behind MVVM and other user interface architectures, such as Model-View-Controller (MVC) and Model-View-Presenter (MVP), is that the separation of concerns ingrained in these architectures makes them testable using unit tests, instead of having to rely on expensive and slow end-to-end tests.

Testing user interfaces built on MVVM
Since all business logic is contained in the view model, it makes sense to make it the center of attention for our testing efforts. Since view models are implemented as classes in object orientedn languages these tests are usually defined as unit tests. This immediately shows the value of asking yourself ‘am I testing the user interface or merely testing my application through the user interface?’. In case of the former, writing complicated, slow and brittle end-to-end tests with tools such as Selenium is pure overkill.

Instead, we write a simple unit test that covers the business logic in the view model. We mock the model part, since business and implementation logic further down the application stack can be tested using (again) unit tests, or when the view model for example invokes web services for storing, processing and retrieving data, we can write API-level tests to cover that part of our application. The view part can – when MVVM is applied correctly – be left out of scope, since buttons, labels and text boxes are usually standardized objects that ‘only’ need to be positioned correctly on screen (in reality, designing good user interfaces is a respectable and skillful job of it’s own, no disrespect intended). If you really want to write checks to verify that the view part is implemented correctly, there’s always tools such as Galen Framework that allow you to check your user interface at the design level (i.e., visual checks).

Links to example implementations and tests
Rather than writing my own example here, I’d like to link to some well-written and understandable examples of the application of MVVM and the way you can write unit tests for your MVVM-based user interface here. I’m currently in the early stages of implementing a suitable testing approach for the user interface of the key application at my current project, so you can expect some real-life examples from my own hand in a future post. For now, I encourage you to take a look at the following blog posts and examples:

For those of you thinking about moving into the test automation field

Thinking about changing your career by moving into the test automation field? This post might be just for you…

So, you decided to become a test automation engineer / SDET / test automagician / … Excellent. Test automation is an important part of the software testing field. But if you’re thinking about taking this route just because you fear that your employer is planning to cancel all manual testing effort, maybe you just need to become a better tester. Show them what value you’re adding to your company and to its products and services as a tester. Either that or leave as soon as possible, because your employer clearly doesn’t know what testing is about and you’re better off at a place that values both automation and actual testing and testers.

What I’m trying to say is that you should first and foremost ask yourself ‘why?’. Why do you want to move into test automation? Is it because you’ve seen other people work on automating checks, test data generation or any other activity related to testing and thought ‘hey, that’s interesting, I want to know more!’ or ‘Hey, that’s nice and all, but I’m sure I could do this better / faster / fancier!’? Marvelous. The test automation field eagerly awaits your arrival. However, if all you’re doing is moving towards test automation because ‘it’s a trend’, ‘I’m afraid to lose my job’ or ‘the pay is better’ (actually, I can understand that last one to some extent..) then please do the field a favor and get better at what you’re currently doing. We need motivated individuals willing to learn, teach and share, not opportunists just looking for the next fad or pay raise.

Still interested? Here’s some more things to consider before making the switch:

Test automation is a craft (not in this way, though)
Using test automation well in your project and organization requires knowledge and skills. It’s just like a real job like that: you need to be willing and able to constantly learn and improve. How can you do that? Read and listen. A lot. Read blogs (the links in my blogroll should provide a nice start), listen to podcasts, go to conferences. Also, following the 10,000 hour rule, do a lot. Tinker with tools, repeat instructions presented in tutorials and then try and expand upon what you’ve learned and maybe most importantly: have your work reviewed. Nothing better than having someone more experienced look at your code and give tips on how to improve. I’ve been in the field for 10 years now and I still do this on a daily basis. Why? Because I know it will make me better at my job.

Test automation is software development
I’ve repeated this a lot of times now (yes, repeated, someone probably much smarter than myself said it long before): test automation is software development. Please treat it like that by:

  • Testing your tests. If you rely on the outcome of your automated checks when deciding to take something into production, you better make sure that your tests do what they promise, and that they fail when required.
  • Applying good design practices and patterns. They’ll make your test code better maintainable, readable and transferable to others.
  • Version controlling your stuff. Check in your test code in just the same way as you check in your production code. It makes it easier to share and review code and to revert any changes if necessary.

Don’t believe tool vendors and fanboys
Don’t get me wrong, I’ve got nothing against tool vendors. I even have a very good professional and personal relationship with some of them (you know who you are). I don’t take everything they say for granted, though. Even though the test automation field has long left behind the era of automation by record and playback and the miracles of codeless test automation solutions (it has, hasn’t it?), there are still tools and vendors that claim that everybody and their dog can use their solution and save millions of hours / dollars on testing. While some tools are indeed easy to use (a good UX design and / or a clear API go a long way) they’re forgetting that probably not everybody can use their solution AND do it successfully. Success for the project or organization that is, not for the tool vendors themselves.

Pretty much the same thing goes for tool fanboys. They’ll do anything to convince you to use their favourite tool, because all the alternatives suck / are too expensive / somehow insulted their family members. While they might have a point sometimes (in those cases where their favourite tool IS the right solution in your case) but make sure to investigate alternatives, or build your own tool if that’s a viable option. Try not to fall into the ‘if all you have is a hammer … ‘ trap.

Think beyond simulating user interaction
As Richard Bradshaw explained much more eloquently in his talk at this year’s Test Automation Day, test automation is much more than simulating the interaction between an end user and your application under test. A script that generates test data for testers is test automation. A routine that checks log files for exceptions during exploratory testing is test automation. In short, every tool and piece of code that is used to support or automate activities related to testing in some way, shape of form is a part of your test automation efforts. Broaden your scope.

Still thinking about moving into test automation?
Excellent. This post wasn’t meant to make you refrain from doing so. All I wanted to do (and what I’ll continue to do in the future) is create a view of the test automation field that’s a little more realistic than some of the pictures painted in other places on the Internet. Did I succeed? I don’t know, I’ll let you decide. As always, please do let me know. Leave a comment or send me an email, I’m always happy to talk to fellow test automation engineers and any of you thinking of becoming one.

Adding service virtualization to your Continuous Delivery pipeline

Lately I’ve been working on a couple of workshops and presentations on service virtualization and the trends in that field. One of these trends – integrating virtualized test environments, containerization and Continuous Delivery – is starting to see some real traction within the service virtualization realm. Therefore I think it might be interesting to take a closer look at it in this post.

What is service virtualization again?
Summarizing: service virtualization is a method to simulate the behaviour of dependencies that are required to perform tests on your application under test. You can use service virtualization to create intelligent and realistic simulations of dependencies that are not available, don’t contain the required test data or are otherwise access-restricted. I’ve written a couple of blog posts on service virtualization in the past, so if you want to read more, check those out here.

Service virtualization

Continuous Delivery and test environment challenges
The challenges presented by restricted access to dependencies during development and testing grow stronger when organizations are moving towards Continuous Delivery. A lack of suitable test environments for your dependencies can be a serious roadblock when you want to build, test and ultimately deploy software on demand and without human intervention. You can imagine that having virtual assets that emulate the behaviour of access-restricted components at your disposition – and more importantly: under your own control – is a significant improvement, especially when it comes to test execution and moving up the pipeline.

Using virtual test environments as artefacts in your pipeline
While having virtual test environments run at some server in your network is already a big step, ‘the next level’ when it comes to service virtualization as an enabler for Continuous Delivery is treating your virtual assets as artefacts that can be created, used and ultimately destroyed at will. I’ll talk about the specific benefits later, but first let’s see how we can achieve such a thing.

One way of treating your virtual test environments as CD artefacts is by containerizing them, for example using Docker. With Docker, you can for example create an image that contains your service virtualization engine, any virtual assets you would like to use and – if applicable – data sources that contain the necessary test data to emulate the required behaviour.

Another way of creating virtual test environments on demand is by hosting them using an on-demand cloud-based service such as Microsoft Azure. This allows you to spin up a simulated test environment when required, use and abuse it and destroy it after you are (or your deployment process) is done.

At the time of writing this blog post, I’m preparing a workshop where we demonstrate – and have the participants work with – both of these approaches, and while the technology is still pretty new, I think this is an interesting and exciting way of taking (or regaining) full control of your test environments and ultimately test faster, more and better.

Adding containerized service virtualization to your continuous delivery pipeline

Benefits of containerized test environments
Using containerized test environments in your Continuous Delivery process has a couple of interesting advantages for your development and testing activities:

  • As said, test environments can be spun up on demand, so no more waiting until that dependency you need is available and configured like you need it to be.
  • You don’t need to worry about resetting the test environment to its original state after testing has completed. Just throw the test environment away and spin up a new instance ready for your next test run.
  • Having the same (virtual) test environment at your disposal every time makes it easier to reproduce and analyze bugs. No more ‘works on my machine’!

Further reading
Want to read more about containerized service virtualization? See for an example specific to Parasoft Virtualize this white paper. And keep an eye out for other service virtualization tool vendors, such as HP and CA. They’re likely working on something similar as well, if they haven’t released it already.

Also, if you’re more of a hands-on person, you can apply containerization to open source service virtualization solutions such as HoverFly yourself.