API testing skills: why you need them

This is the first post in a three-part series on API testing. This post is an introduction on APIs, API testing and its relevance to the testing world. The second part of this series will feature some best practices for everybody involved in API testing. The third and final post will contain some useful code example for those of you looking to build your own automated API testing framework.

As information systems become more and more distributed and systems and devices become ever more interconnected, the use of APIs has seen exponential growth in the past couple of years. Where traditional (or old-fashioned) computer systems were monolithic in nature, nowadays they are often made up of reusable components that communicate and exchange information with one another through various APIs.

The figure below depicts the growth in number of publicly accessible APIs, as published by ProgrammableWeb:
Growth in publicly accessible APIs in recent years

Estimates for the number of publicly available APIs for the coming years range from 300.000 in 2016 to around a million by 2017.

With APIs becoming more and more common and important, proper testing of these APIs has become a hot issue with both API providers and consumers. As a provider, you wouldn’t want to be associated with an API of poor quality. As a consumer, you wouldn’t want your software system and/or your business to rely on a buggy API.

However, as APIs are designed for computer-to-computer interaction, rather than computer-to-user interaction, they do not have a user interface through which the tester can access the API. Moreover, to properly assess whether the output as given by the API is correct, a tester would need to know at least something about the internal workings of the API (i.e., perform white-box testing rather than traditional black-box testing). This might make API testing seem ‘hard’ or ‘difficult’ for some testers.

What makes API testing even more important is that in the current wave of layered information systems, business rules and business logic is often coded and enforced within the API layer (and not in the user interface or the database layer, for example). This is yet another reason for every development project that features the development or consumption of APIs to pay sufficient attention to API testing.

In the next part of this series, I will present some pointers for those of you who are involved in API testing, are looking to do so, or have been asked to do so. Many of my tips are also featured on the API Testing Dojo. There, you can also find some exercises to test and sharpen your API testing skills. A highly recommended resource.

Four people in your organization that will benefit from test automation

The introduction of test automation within your organization can, if done correctly, affect your testing process in a number of positive ways. In this post, we will look at the ways different stakeholders within your software development process might benefit from automated testing.

Test automation benefits the software testing team in a number of ways, of which some of the most important are:

  • Automation of regression tests takes away the repetitive effort of executing the same test scripts by hand over and over again, every time new functionality is introduced. Instead of having to spend time on this often tedious activity, software testers can focus on testing the changes that are introduced in the software, while the automated test tool takes care of the regression testing.
  • Not having to spend time on regression testing means that testers can perform more tests in the same amount of time.
  • Testers with an interest in programming and tools have the opportunity to learn valuable new skills by diving into the world of automated testing.

The development team might also benefit from the introduction of test automation:

  • As automated tests can generally be run faster and more often compared to when the same test scripts have to be executed manually, developers receive feedback on the quality of their code sooner.
  • Testers involved in automated testing often work closely together with developers to set up tests and analyse test results, resulting in a bigger appreciation for what used to be ‘the other side’. Of course, this appreciation works both ways (and this point could have been under ‘Testers’ just as easily).

Project managers
Automated testing has several benefits for project management too:

  • With automated testing, more tests can be performed within the duration of the project. On the one hand, today’s dynamic market requires fast-paced software delivery and therefore ever shorter development cycles. On the other hand, the same dynamic market expects high quality software as for a lot of products many alternatives are available, making it easy for an unsatisfied customer to change products or services. Automated testing is the only way for an organization to deliver high quality software quickly.
  • Automated testing enables tests to be executed sooner and more often, giving project managers continuous insight in the quality of the software, as well as in trends in this quality. For example, the availability of daily test results gives a project manager insight in whether the introduction of a new development method – or a new development team – has the desired effect on the quality of the product to be delivered.

End users
Finally, the end user of the software delivered also benefits from the introduction of automated testing, albeit in a more indirect way:

  • Automated testing, when implemented correctly, can greatly increase the speed with which software is delivered, meaning that the end user receives a release of the product he or she has asked for quicker than would be possible with traditional manual testing. As automated testing is able to shorten development cycles, any bugs reported or updates requested by the user can be dealt with quicker as well.
  • As the test team is able to achieve higher test coverage within the given amount of time, the quality of the end product will generally also be higher as more bugs can be resolved before delivery, meaning that the user will likely be happier with this product.

How did the introduction of automated testing benefit your organization or (for the contractors) your client?

Five considerations before choosing an automated testing solution

Selecting a tool to aid your project or organisation with the implementation of automated software testing can be a daunting task. Many tools are available, each of them with its own benefits and drawbacks. The following considerations will provide you with some much-needed guidance in the test tool selection process.

What kind of technology is to be automated?
The choice for a specific automated testing tool heavily depends on the type of application(s) your tests are written for. Many tools are specifically designed for a single type of application, such as Selenium, which is targeted towards websites and web applications running in a browser. Other tools are able to handle a lot of different types of applications, such as browser applications, Windows applications and SAP. Tools that fall into the latter category tend to be more expensive, but might be the better choice if your test scripts cover different types of applications.

Do I go for open source or for commercially licensed tooling?
At first sight, going the open source route when selecting an automated testing tool might seem the most cost-effective option, due to the lack of license fees and support contract fees that typically come with commercial test tools. However, open source tools come with necessary investments of their own, as it often takes more time and more technical (programming) knowledge to set up and maintain automated test scripts. One often overlooked advantage of open source test tooling is the fact that the more popular ones are often supported a large community on the Internet. This community can be tapped into for technical support and often offers a library of additional features and extensions for the tool in case.

Do I prefer in house implementation or do I want to outsource test automation activities?
Another consideration to make is whether to have your own team implement the automated test scripts or to leave this to a third party. Keeping automated test development in house has a number of benefits, such as:

  • The people working on test automation are likely to be very familiar with the application(s) under test.
  • Knowledge gained on test automation is retained within the organisation.
  • The opportunity to work on test automation might be a good motivator for people in the organisation willing to learn something new.

On the other hand, outsourcing test automation implementation also has certain benefits, the most important being that dedicated test (automation) service providers likely have a lot of experience with the selected tool, enabling them to implement it much more efficiently.

How much training is required for the implementation of test automation?
Some tools require more technical knowledge upfront before users can successfully automate tests with it, while others do all they can to abstract from the technical details and offer an interface that makes it possible to implement and run automated tests for everybody. The former category of tools typically requires more training for them to be used. Another factor to be considered is the programming or scripting language used by the automated test tool. If, for instance, a tool is Java-based and all you have in your company are PHP developers, more training might be required than when your software development department – and the testers included in it – works with Java on a daily basis.

Does the tool match our current software testing process?
Finally, another thing worth considering is the extent to which the tool matches your current software testing process. Questions that you might want to ask with respect to this are:

  • Do the reports produced by the tool match the information requirements from the software development team and from management?
  • Does the tool integrate nicely with my current or desired automated build or continuous integration software delivery setup?
  • Can scripts be run by everybody within the software development and testing team? Note that this differs from who is going to develop the scripts.

When there is a mismatch between the features of the tool under consideration and your current software testing process, there are two further options. One is to consider a different tool that better fits the process, the other is to adapt the process to fit around the tool. The former is generally the more preferable, but in some cases where the testing tool fits the software to be tested perfectly, it might be worthwhile to adapt your testing process in some points to get a good match between the two.