Getting started with JavaScript testing using Jasmine

New project, new environment, new tools to explore! I just started working as a freelance test engineer on a new project involving some of Holland’s most visited job sites. An environment where a lot of PHP and JavaScript is used, as opposed to the familiar Java and C#. This also means I get to learn a lot of new stuff (I’ve not worked much in web development projects before) and a lot of new and interesting test automation tools and concepts to explore and apply. Which I can then use to bore you.

The first tool I get to explore is Jasmine, a BDD-style tool for testing JavaScript code. As with most other BDD tools, it’s only truly a BDD tool if you use it as such and I’m not yet convinced that we’re doing that right in this project. However, Jasmine offers enough interesting features for a blog post, so here we go..

What is Jasmine and where can I get it?
Simply put, Jasmine is a stand-alone JavaScript testing tool that supports writing your tests in a BDD-like manner. It does not require a DOM and therefore a browser to run, which makes tests run incredibly fast. Its syntax is also pretty clear and simple, as we will see in this post.

You can download the latest Jasmine release from here, after which you can unzip it. The package contains the Jasmine source (also written in JavaScript) and some sample code and tests.

The code to be tested
My example code that will be tested is a simple Car object that supports a two methods, one to add a driver to a car and one to start driving the car:

function Car() {
}

Car.prototype.addDriver = function() {
  if (this.hasDriver) {
	  throw new Error("Someone's already driving this car");
  }
  if (this.isDriving) {
	  throw new Error("Can't add a driver when the car is driving");
  }
  this.hasDriver = true;
};

Car.prototype.startDriving = function() {
	this.isDriving = true;
}

A first Jasmine test
Let’s say we want to write a simple test to see whether we can actually add a driver to a car. In Jasmine, this is done as follows:

describe("A Car", function() {

  var car;
  var person;

  beforeEach(function() {
    car = new Car();
    person = new Person();
  });

  it("should be able to be assigned a driver", function() {
	car.addDriver(person);
	expect(car.hasDriver).toBeTruthy();
  });
});

The describe keyword starts a new test suite and takes two parameters: the first is a string providing a description for the test suite, the second is a method that implements the test suite. A test suite consists of a number of tests (or specs in Jasmine terminology), each defined using the it keyword. Just as with Java testing frameworks such as JUnit and TestNG, you have the option to perform setup and teardown actions, in this case using beforeEach.

Assertions are performed using the expect keyword and a matcher, in this case toBeTruthy(). This matcher checks whether the value of a variable evaluates to true (note that values such as 1 and ‘test’ also evaluate to true in JavaScript). Jasmine comes with a rich set of matchers, which are described here.

Running your tests
Running Jasmine tests can be done by creating an HTML file that includes both the source code and the specs files in the and opening it in a browser. The HTML runner file for my demo project looks like this:

<!DOCTYPE html>
<html>
<head>
  <meta charset="utf-8">
  <title>Jasmine Car Demo Spec Runner v2.4.1</title>

  <link rel="shortcut icon" type="image/png" href="lib/jasmine-2.4.1/jasmine_favicon.png">
  <link rel="stylesheet" href="lib/jasmine-2.4.1/jasmine.css">

  <!-- include Jasmine -->
  <script src="lib/jasmine-2.4.1/jasmine.js"></script>
  <script src="lib/jasmine-2.4.1/jasmine-html.js"></script>
  <script src="lib/jasmine-2.4.1/boot.js"></script>

  <!-- include source files -->
  <script src="src/Car.js"></script>
  <script src="src/Person.js"></script>

  <!-- include spec files -->
  <script src="spec/SpecHelper.js"></script>
  <script src="spec/CarSpec.js"></script>
</head>
<body/>
</html>

When we open this HTML file in our browser, the specs are run and we can see the following result:

Results of first Jasmine test run

Hooray!

By the way, if the assertion would fail, the Jasmine output would look like this:

Example of a failing Jasmine check

Checking for expected errors
Returning to the code for the Car object, we can see that some methods throw errors. If you want to check that these errors are indeed thrown, you can do that in Jasmine like this:

it("should throw an error when assigned a second driver", function() {
  car.addDriver(person);
  
  expect(function() {
	  car.addDriver(person);
  }).toThrowError("Someone's already driving this car");
});

Adding and using custom matchers
Even though Jasmine includes a variety of matchers to perform all kinds of checks, sometimes you might have to perform very specific checks not covered by one of the standard matchers. In this case, it’s also possible to write a custom matcher (which is called toBeDriving in this case):

beforeEach(function () {
  jasmine.addMatchers({
    toBeDriving: function () {
      return {
        compare: function (actual, expected) {
          var car = actual;

          return {
            pass: car.isDriving
          };
        }
      };
    }
  });
});

This custom matcher can then be used in Jasmine specs just like any other matcher:

it("should be able to start driving", function() {
  car.startDriving();
  expect(car).toBeDriving();		
});

Jasmine offers lots more features, such as the ability to write custom reporters. Check it out!

The example project I have used in this blog post can be downloaded here. It includes Jasmine 2.4.1 and a working HTML runner, so you can start experimenting with Jasmine right away.

Getting started with: JGiven

This is the eighth article in our series on new, popular or otherwise interesting tools used in test automation. You can read all posts within this series by clicking here.

What is JGiven?
From the JGiven.org website: JGiven is a developer-friendly and pragmatic BDD tool for Java. Developers write scenarios in plain Java using a fluent, domain-specific API, JGiven generates reports that are readable by domain experts.

In short, JGiven can be used as an alternative to other BDD frameworks, such as JBehave and Cucumber. Where the latter two separate the features from the test code, JGiven does not and might therefore be considered more suitable to unit and integration tests rather than automated system and regression tests.

Where can I get JGiven?
The JGiven sources can be downloaded from the JGiven GitHub site.

How do I install and configure JGiven?
The easiest way to get started with JGiven is by creating a new Maven project and including the following dependency:

<dependency>
	<groupId>com.tngtech.jgiven</groupId>
	<artifactId>jgiven-testng</artifactId>
	<version>0.9.5</version>
	<scope>test</scope>
</dependency>

This is for use in combination with TestNG, which I will do in the rest of this post. If you prefer to user JUnit, use jgiven-junit as your artifactId. Please note that this dependency does not include TestNG (or JUnit) itself, so be sure to include it separately in your pom.xml.

Creating and running a first JGiven test
Taking my trusted ParaBank application as an example, I want to perform a test determining whether I can login successfully using a JGiven scenario. I.e., I want to execute the following scenario:

Given I am at the login page
When I login as john with password demo
Then the login action will be successful

The implementation of this scenario as a TestNG-based test in JGiven is pretty straightforward:

public class LoginTest extends ScenarioTest<GivenIAmAtTheLoginPage, WhenILoginAsJohnWithPasswordDemo, ThenTheLoginActionWillBeSuccessful> {
	
	@Test
	public void aFirstLoginTest() {
		
		given().I_am_at_the_login_page();
		when().I_login_as_john_with_password_demo();
		then().the_login_action_will_be_successful();
	}
}

The ScenarioTest class requires three parameters, each representing a stage in the Given-When-Then scenario. To make this compile, we also need to implement each of the three classes that are used as a parameter. As an example, this is what the implementation of the GivenIAmAtTheLoginPage class looks like:

public class GivenIAmAtTheLoginPage extends Stage<GivenIAmAtTheLoginPage>{
	public GivenIAmAtTheLoginPage I_am_at_the_login_page() {
		
		return self();
	}
}

The other two classes (stages) are implemented in a similar way. Now that this is done, we can run our test, which lads to the following output in the console:

Test Class: com.ontestautomation.jgiven.tests.LoginTest

 Scenario: A first login test

   Given I am at the login page
    When I login as john with password demo
    Then the login action will be successful

PASSED: aFirstLoginTest

We can see that our test passes, so we have something to build upon. No actual test actions are performed yet, so we are going to add these next. Since we are performing a Selenium WebDriver test, it’s required that we pass along the browser instance as a parameter for each of the steps. This also means we can initialize it before the test is run (using @BeforeTest) and destroy it afterwards (using @AfterTest):

public class LoginTest extends ScenarioTest<GivenIAmAtTheLoginPage, WhenILoginAsJohnWithPasswordDemo, ThenTheLoginActionWillBeSuccessful> {
	
	WebDriver driver;
	
	@BeforeTest
	public void initBrowser() {

		driver = new FirefoxDriver();
		driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
	}
	
	@Test
	public void aFirstLoginTest() {
		
		given().I_am_at_the_login_page(driver);
		when().I_login_as_john_with_password_demo(driver);
		then().the_login_action_will_be_successful(driver);
	}
	
	@AfterTest
	public void tearDown() {
		
		driver.quit();
	}
}

The WebDriver actions associated with the Given, When and Then steps can now be added to the respective classes as well, for example for GivenIAmAtTheLoginPage this results in:

public class GivenIAmAtTheLoginPage extends Stage<GivenIAmAtTheLoginPage>{
	
	WebDriver _driver;
	
	public GivenIAmAtTheLoginPage I_am_at_the_login_page(WebDriver driver) {
		
		_driver = driver;
		
		_driver.get("http://parabank.parasoft.com");
		
		return self();
	}
}

If we implement the other two Stage classes in a similar vein – where the Then stage should include an actual TestNG assertion to make it a proper test case – and rerun our test, we can see that it passes again:

Test Class: com.ontestautomation.jgiven.tests.LoginTest

 Scenario: A first login test

   Given I am at the login page FirefoxDriver: firefox on WINDOWS (6bccf261-5ce0-4378-8118-b545b6c82eca)
    When I login as john with password demo FirefoxDriver: firefox on WINDOWS (6bccf261-5ce0-4378-8118-b545b6c82eca)
    Then the login action will be successful FirefoxDriver: firefox on WINDOWS (6bccf261-5ce0-4378-8118-b545b6c82eca)

PASSED: aFirstLoginTest

Note that JGiven automatically adds a bit of information on the browser instance used to the console output.

Useful features
Some useful additional features of JGiven are:

  • Several types of test execution and results reporting, including JSON and HTML
  • Support for parameterized steps (in the above example, you could for instance parameterize the username and password in the When step)
  • Support for tags to organize scenarios

Further reading
Apart from the features mentioned above, JGiven provides some more useful features for creating useful BDD-style Java-based tests. A complete reference guide can be found on the JGiven website.

An Eclipse Maven project including the tests and test classes I’ve used in this post can be downloaded here.

Writing BDD tests using Selenium and Cucumber

In this article, I want to introduce another Behaviour Driven Development (BDD) tool to you. It’s similar to JBehave, about which I’ve been talking in a previous post, but with a couple of advantages:

  • It is being updated regularly, in contrast to JBehave, which has not seen any updates for quite some time now
  • It works completely separately from your Selenium WebDriver implementation, which means you can simply add it as another dependency to your existing and new Selenium projects. Again, this is unlike JBehave, which uses its own Selenium browser driver implementation, and because these aren’t updated regularly, it’s likely you’ll run into compatibility issues when trying to run Selenium + JBehave tests on newer browser versions.

Installing and configuring Cucumber
The Cucumber Java implementation (Cucumber JVM) can be downloaded from the Cucumber homepage. Once you’ve downloaded the binaries, just add the required .jar files as dependencies to your Selenium project and you’re good to go.

Defining the user story to be tested
Like with JBehave, Cucumber BDD tests are written as user stories in the Given-When-Then format. I am going to use the following login scenario as an example user story in this post:

Feature: Login action
	As a user of ParaBank
	I want to be able to log in
	So I can use the features provided
	
	Scenario: An authorized user logs in successfully
	Given the ParaBank home page is displayed
	When user "john" logs in using password "demo"
	Then the login is successful

ParaBank is a demo application from the guys at Parasoft that I use (or abuse) a lot in my demos.

In order for Cucumber to automatically detect the stories (or features, as they’re known in Cucumber), you need to make sure that they carry the .feature file extension. For example, in this case, I’ve named my user story login.feature.

Translating the test steps to code
As with JBehave, the next step is to translate the steps from the user story into runnable code. First, we are going to define the actual class to be run, which looks like this:

import org.junit.runner.RunWith;
import cucumber.api.CucumberOptions;
import cucumber.api.junit.Cucumber;

@RunWith(Cucumber.class)
@CucumberOptions(format = "pretty")
public class SeleniumCucumber {
	
}

I’ve configured my tests to run as JUnit tests, because this will make it easy to incorporate them into a CI configuration, and it auto-generates reporting in Eclipse. The @RunWith annotation defines the tests to run as Cucumber tests, whereas the @CucumberOptions annotation is used to set options for your tests (more info here). Cucumber automatically looks for step definitions and user stories in the same package, so there’s no need for any additional configuration here. Note that in Cucumber, the class to be run cannot contain any actual implementation of the test steps. These are defined in a different class:

public class SeleniumCucumberSteps {

	WebDriver driver;
	
	@Before
	public void beforeTest() {
		driver = new FirefoxDriver();
	}

	@Given("^the ParaBank home page is displayed$")
	public void theParaBankHomepageIsDisplayed() {

		driver.get("http://parabank.parasoft.com");
	}

	@When("^user john logs in using password demo$")
	public void userJohnLogsInUsingPasswordDemo() {

		driver.findElement(By.name("username")).sendKeys("john");
		driver.findElement(By.name("password")).sendKeys("demo");
		driver.findElement(By.cssSelector("input[value='Log In']")).click();
	}

	@Then("^the login is successful$")
	public void theLoginIsSuccessful() {
		Assert.assertEquals("ParaBank | Accounts Overview",driver.getTitle());
	}
	
	@After
	public void afterTest() {
		driver.quit();
	}
}

Similar to what we’ve seen with JBehave, each step in our user story is translated to runnable code. As the user story is quite straightforward, so is its implementation, as you can see.

I’ve used the JUnit @Before and @After annotations for set-up and tear-down test steps. These are steps to set up your test environment before tests are run and to restore the test environment to its original setup after test execution is finished. In our case, the only set-up step we take is the creation of a new browser instance, and the only tear-down action is closing that same browser instance. All other steps are part of the actual user story to be executed.

I’ve used the JUnit Assert feature to execute the verification from the Then part of the user story, so it shows up nicely in the reports.

That’s all there is to creating a first Cucumber test.

Running the test
Now, if we run the tests we defined above by running the SeleniumCucumber class, Cucumber automatically detects the user story to be executed (as it is in the same directory) and the associated step implementations (as they’re in the same package too). The result is that the user story is executed successfully and a nicely readable JUnit-format report is generated and displayed:
Cucumber test resultsAs you can see, the test result even displays the name of the feature and the scenario that has been executed, which makes it all the more readable and usable.

Using parameters in test steps
In Cucumber, we can use parameters to make our step definitions reusable. This is done by replacing the values in the step definitions with regular expressions and adding parameters to the methods that implement the steps:

@When("^user \"(.*)\" logs in using password \"(.*)\"$")
public void userJohnLogsInUsingPasswordDemo(String username, String password) {

	driver.findElement(By.name("username")).sendKeys(username);
	driver.findElement(By.name("password")).sendKeys(password);
	driver.findElement(By.cssSelector("input[value='Log In']")).click();
}

When running the tests, Cucumber will replace the regular expressions with the values defined in the user story and pass these as parameters to the method that is executed. Easy, right? You can do the same for integer, double and other parameter types just as easily if you know your way around regular expressions.

Overall, I’ve found Cucumber to be really easy to set up and use so far, even for someone who is not a true developer (I’m definitely not!). In a future post, I’ll go into more detail on adding these tests to a Continuous Integration solution.

An Eclipse project containing the example code and the user story / feature I’ve used in this post can be downloaded here.