An approach to test your user interface more efficiently

As returning readers of this blog might have read before, I am pretty skeptical about writing automated tests that interact with the application under test at the user interface level. These UI tests tend to be:

  • slow in execution, since they are typically end-to-end tests from an application-layer perspective, and
  • demanding when it comes to maintenance, because the user interface is typically a part of an application subject to frequent changes.

However, the user interface often is an important component of an application and therefore it requires testing effort. Since this blog is all about test automation, I’d like to talk about a different approach to user interface test automation in this blog post.

But first, let me explain a subtle yet important difference. On the one hand, there’s testing the user interface. Here, the actual logic incorporated in the user interface is isolated and tested as a unit. Everything that’s happening ‘behind’ the user interface is out of scope of the test and therefore usually mocked. On the other hand, there’s testing application logic through the user interface. This is generally done using tools such as Selenium. This type of automated tests uses the user interface as its point of entry and validation, even though the actual logic that processes the input and generates the output to be checked is not implemented at the user interface layer at all.

Testing the user interface versus testing through the user interface

Now, only when you specifically want to perform end-to-end application tests, or when there really isn’t any other option than to use the user interface drive tests that validate underlying application logic should you resort to tools such as Selenium. In all other cases, it might be a better idea to see if there’s a better option available.

User interface architectures and testability
In the remainder of this post I want to zoom in on a commonly used user interface pattern, namely Model-View-ViewModel, or MVVM in short, and what I think is a suitable approach for writing automated tests for user interfaces adhering to this pattern.

MVVM explained (briefly)
The MVVM pattern lets you separate presentation logic (that defines what information is shown on the screen) from the actual presentation (that defines how the information is displayed). Schematically, the pattern looks like this:

A schematic representation of the Model-View-ViewModel pattern

The View is the user interface, which is made up of textboxes, labels, buttons, etc. It is responsible for defining the structure, layout and appearance of what you seen on the screen. The Model is an implementation of the domain model. It consists of a data model along with business rules and validation logic. The View Model is the intermediary between the View and the Model, handling user interface actions (such as the click of a button) and interacting with models usually by invoking methods in the model classes (for example to update a record in a database). It is also responsible for presenting data from the model (for example the results of a query) to the view in a representation that can be easily displayed on screen.

One of the key ideas behind MVVM and other user interface architectures, such as Model-View-Controller (MVC) and Model-View-Presenter (MVP), is that the separation of concerns ingrained in these architectures makes them testable using unit tests, instead of having to rely on expensive and slow end-to-end tests.

Testing user interfaces built on MVVM
Since all business logic is contained in the view model, it makes sense to make it the center of attention for our testing efforts. Since view models are implemented as classes in object orientedn languages these tests are usually defined as unit tests. This immediately shows the value of asking yourself ‘am I testing the user interface or merely testing my application through the user interface?’. In case of the former, writing complicated, slow and brittle end-to-end tests with tools such as Selenium is pure overkill.

Instead, we write a simple unit test that covers the business logic in the view model. We mock the model part, since business and implementation logic further down the application stack can be tested using (again) unit tests, or when the view model for example invokes web services for storing, processing and retrieving data, we can write API-level tests to cover that part of our application. The view part can – when MVVM is applied correctly – be left out of scope, since buttons, labels and text boxes are usually standardized objects that ‘only’ need to be positioned correctly on screen (in reality, designing good user interfaces is a respectable and skillful job of it’s own, no disrespect intended). If you really want to write checks to verify that the view part is implemented correctly, there’s always tools such as Galen Framework that allow you to check your user interface at the design level (i.e., visual checks).

Links to example implementations and tests
Rather than writing my own example here, I’d like to link to some well-written and understandable examples of the application of MVVM and the way you can write unit tests for your MVVM-based user interface here. I’m currently in the early stages of implementing a suitable testing approach for the user interface of the key application at my current project, so you can expect some real-life examples from my own hand in a future post. For now, I encourage you to take a look at the following blog posts and examples:

10 thoughts on “An approach to test your user interface more efficiently

  1. Yet another useful post from you. Thanks Bas. As you said automating UI using GUI testing tools is expensive, time consuming, hard to maintain, though we cannot completely write them off. UI based tests are useful when we target end-to-end tests. Rather than automating most of the test suite using UI automation tools, we can target select high priority system tests.

    Liked your point of writing tests for the layer below presentation layer, and such tests are more beneficial. Such automation can be taken up when AUT is implemented using MVVM, MVC frameworks where Presentation layer is clearly separated from Model layer. Also automation can get more ROI when AUT is developed using SOA (REST or SOAP APIs) and service tests are automated.

  2. Pingback: Java Web Weekly, Issue 134 | Baeldung

  3. Pingback: Testing Bits – 7/17/16 – 7/23/16 | Testing Curator Blog

  4. Unfortunately, in MVVM, UI components are not just standard objects. They are that plus the bindings. And the bindings tend to break easily when you rename properties in VM. For example, WPF bindings are not typesafe and are not refactored when renaming VM properties. So it would be really nice to have a way to test those bindings without bringing up the whole GUI framework. But as far as I know, there isn’t any.

    • Hi Sergey,

      You make a very valid point. That’s one aspect of the View I somehow overlooked and indeed a part that could make your WPF application break. No matter how well you’ve tested the ViewModel. I haven’t seen a way to test these bindings without launching the application and essentially doing an end to end test either..

  5. Hi Bas, what do you think about codeless cloud platforms for UI testing (like Screenster for example) As i understood from you comparison between testing the user interface vs testing application logic through the user interface, tools like this could be very helpful

    • Hey Nikita,

      to be honest, I’m a bit on the fence about tools such as Screenster (and Galen, and Applitools, and …). They can definitely be useful, but only if human checking is not a feasible alternative. For example when you’ve got a CD pipeline all the way to production, yet you still want to do visual checking. The way these tools work, there’s SO much maintenance involved that you need to be very careful not to simply throw away everything and start recording anew with every major change.

      This because of the way these tools work: they check everything, unless you specifically ignore it, whereas tools like Selenium ignore everything unless specified otherwise. As such, they can work complementary (I’d never use Screenster or a similar tool as a replacement for Selenium), but it’s a fine line..

      Does that make sense?

      • Bas, thank you for the answer. Selenium has a strong reputation on the IT market, so it will be hard to replace it anyway. But new ui testing tools offer you new possibilities and makes your testing process easier and faster without a line of code. They cannot definitely replace a human check, but they help to make your testing more efficient. At least that’s my experience.

        • Hey Nikita,

          that’s very true and I agree with what you say. My point was twofold:

          • tools like Screenster and tools like Selenium should be used complementary, not either-or
          • Image-based comparison tools like Screenster should be used with care, otherwise you’ll just introduce another layer of maintenance hell (as with every tool)

          There is a place and time for tools like this, otherwise there wouldn’t be multiple solutions on offer.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.