Using the ExtentReports TestNG listener in Selenium Page Object tests

In this (long overdue) post I would like to demonstrate how to use ExtentReports as a listener for TestNG. By doing so, you can use your regular TestNG assertions and still create nicely readable ExtentReports-based HTML reports. I’ll show you how to do this using a Selenium WebDriver test that uses the Page Object pattern.

Creating the Page Objects
First, we need to create some page objects that we are going to exercise during our tests. As usual, I’ll use the Parasoft Parabank demo application for this. My tests cover the login functionality of the application, so I have created Page Objects for the login page, the error page where you land when you perform an incorrect login and the homepage (the AccountsOverviewPage) where you end up when the login action is successful. For those of you that have read my past post on Selenium and TestNG, these Page Objects have been used in that example as well. The Page Object source files are included in the Eclipse project that you can download at the end of this post.

Creating the tests
To demonstrate the reporting functionality, I have created three tests:

  • A test that performs a successful login – this one passes
  • A test that performs an unsuccessful login, where the check on the error message returns passes – this one also passes
  • A test that performs an unsuccessful login, where the check on the error message returns fails – this one fails

The tests look like this:

public class LoginTest {
	
 WebDriver driver;
     
    @BeforeSuite
    public void setUp() {
	         
        driver = new FirefoxDriver();
        driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
    }
	     
    @Parameters({"incorrectusername","incorrectpassword"})
    @Test(description="Performs an unsuccessful login and checks the resulting error message (passes)")
    public void testFailingLogin(String username, String incorrectpassword) {
	         
        LoginPage lp = new LoginPage(driver);
        ErrorPage ep = lp.incorrectLogin(username, incorrectpassword);
        Assert.assertEquals(ep.getErrorText(), "The username and password could not be verified.");
    }
	    
    @Parameters({"incorrectusername","incorrectpassword"})
    @Test(description="Performs an unsuccessful login and checks the resulting error message (fails)")
    public void failingTest(String username, String incorrectpassword) {
	         
        LoginPage lp = new LoginPage(driver);
        ErrorPage ep = lp.incorrectLogin(username, incorrectpassword);
        Assert.assertEquals(ep.getErrorText(), "This is not the error message you're looking for.");
    }
	    
    @Parameters({"correctusername","correctpassword"})
    @Test(description="Performs a successful login and checks whether the Accounts Overview page is opened")
    public void testSuccessfulLogin(String username, String incorrectpassword) {
	         
        LoginPage lp = new LoginPage(driver);
        AccountsOverviewPage aop = lp.correctLogin(username, incorrectpassword);
        Assert.assertEquals(aop.isAt(), true);
    }
	     
    @AfterSuite
    public void tearDown() {
         
        driver.quit();
    }
}

Pretty straightforward, right? I think this is a clear example of why using Page Objects and having the right Page Object methods make writing and maintaining tests a breeze.

Creating the ExtentReports TestNG listener
Next, we need to define the TestNG listener that creates the ExtentReports reports during test execution:

public class ExtentReporterNG implements IReporter {
    private ExtentReports extent;
 
    @Override
    public void generateReport(List<XmlSuite> xmlSuites, List<ISuite> suites, String outputDirectory) {
        extent = new ExtentReports(outputDirectory + File.separator + "ExtentReportTestNG.html", true);
 
        for (ISuite suite : suites) {
            Map<String, ISuiteResult> result = suite.getResults();
 
            for (ISuiteResult r : result.values()) {
                ITestContext context = r.getTestContext();
 
                buildTestNodes(context.getPassedTests(), LogStatus.PASS);
                buildTestNodes(context.getFailedTests(), LogStatus.FAIL);
                buildTestNodes(context.getSkippedTests(), LogStatus.SKIP);
            }
        }
 
        extent.flush();
        extent.close();
    }
 
    private void buildTestNodes(IResultMap tests, LogStatus status) {
        ExtentTest test;
 
        if (tests.size() > 0) {
            for (ITestResult result : tests.getAllResults()) {
                test = extent.startTest(result.getMethod().getMethodName());
 
                test.getTest().startedTime = getTime(result.getStartMillis());
                test.getTest().endedTime = getTime(result.getEndMillis());
 
                for (String group : result.getMethod().getGroups())
                    test.assignCategory(group);
 
                String message = "Test " + status.toString().toLowerCase() + "ed";
 
                if (result.getThrowable() != null)
                    message = result.getThrowable().getMessage();
 
                test.log(status, message);
 
                extent.endTest(test);
            }
        }
    }
 
    private Date getTime(long millis) {
        Calendar calendar = Calendar.getInstance();
        calendar.setTimeInMillis(millis);
        return calendar.getTime();        
    }
}

This listener will create an ExtentReports report called ExtentReportTestNG.html in the default TestNG output folder test-output. This report lists all passed tests, then all failed tests and finally all tests that were skipped during execution. There’s no need to add specific ExtentReports log statements to your tests.

Running the test and reviewing the results
To run the tests, we need to define a testng.xml file that enables the listener we just created and runs all tests we want to run:

<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >

<suite name="Parabank login test suite" verbose="1">
	<listeners>
		<listener class-name="com.ontestautomation.extentreports.listener.ExtentReporterNG" />
	</listeners>
	<parameter name="correctusername" value="john" />
	<parameter name="correctpassword" value="demo" />
	<parameter name="incorrectusername" value="wrong" />
	<parameter name="incorrectpassword" value="credentials" />
	<test name="Login tests">
		<packages>
			<package name="com.ontestautomation.extentreports.tests" />
		</packages>
	</test>
</suite>

When we run our test suite using this testng.xml file, all tests in the com.ontestautomation.extentreports.tests package are run and an ExtentReports HTML report is created in the default test-output folder. The resulting report can be seen here.

More examples
More examples on how to use the ExtentReports listener for TestNG can be found on the ExtentReports website.

The Eclipse project I have created to demonstrate the above can be downloaded here.

FindBy strategies for Selenium explained

The @FindBy annotation is used in Page Objects in Selenium tests to specify the object location strategy for a WebElement or a list of WebElements. Using the PageFactory, these WebElements are usually initialized when a Page Object is created. In this post, I will demonstrate various ways in which you can use @FindBy annotations to efficiently locate (groups of) WebElements.

@FindBy
The @FindBy annotation is used to locate one or more WebElements using a single criterion. For example, to identify all elements that have the same class attribute, we could use the following identification:

@FindBy(how = How.CLASS_NAME, using = "classname")
private List<WebElement> singlecriterion;

If we are sure there is only a single element that is identified by our location strategy, for example when we use the element ID, we can also directly assign the result to a WebElement variable:

@FindBy(how = How.ID, using = "elementid")
private WebElement element;

To instantiate the elements, we call the initElements method of the PageFactory class:

PageFactory.initElements(driver, this);

@FindBys and @FindAll
In some cases we want (or need) to use more than a single criterion to identify one or more objects, for instance when page elements do not have a unique ID. In this case, there are two possible annotations that can be used:

  • The @FindBys annotation is used in case elements need to match all of the given criteria
  • The @FindAll annotation is used in case elements need to match at least one of the given criteria

Let’s take a look at an example that illustrates the difference between the two.

The Parabank homepage contains two textboxes, one for the username and one for the password. Both elements have a name attribute that we are going to use to identify them within a Page Object.

Using @FindBys:

@FindBys({
	@FindBy(how = How.NAME, using = "username"),
	@FindBy(how = How.NAME, using = "password")
})
private List<WebElement> bothcriteria;

The bothcriteria list should contain 0 elements, as there is no element that has both a name attribute with the value username and a name attribute with the value password.

Using @FindAll:

@FindAll({
	@FindBy(how = How.NAME, using = "username"),
	@FindBy(how = How.NAME, using = "password")
})
private List<WebElement> eithercriterion;

The eithercriterion list should contain 2 elements, as there is one element that has a name attribute with the value username and also one that has a name attribute with the value password.

For validation purposes, if we print the number of results found by all of the above strategies using

System.out.println("Using @FindBy, we found " + singlecriterion.size() + " element(s)");
System.out.println("Using @FindBys, we found " + bothcriteria.size() + " element(s)");
System.out.println("Using @FindAll, we found " + eithercriterion.size() + " element(s)");

we see this:
Results for different FindBy strategiesIt clearly works exactly as expected!

A more verbose FindBy
Finally, if you have a lot of elements within your Page Object, you can also use a more verbose way of specifying your @FindBy strategy. For example

@FindBy(className = "classname")

gives the exact same results as

@FindBy(how = How.CLASS_NAME, using = "classname")

Creating a video capture of your Selenium tests using Monte Screen Recorder

In previous posts, I have shown you how to include screen shots in your HTML test reports to help you in debugging tests and identifying the cause of test failures. However, in certain cases, a video showing exact screen states, screen transitions and input as generated by your tests might come in even more handy. This goes expecially when your tests run unattended. Such a video could also help presenting / demonstrating your work to your peers and other stakeholders.

Last week I stumbled upon Monte Screen Recorder, a Java library that can assist you in creating these video captures of your Selenium tests. In this relatively short post, I will show you how to create such a video capture.

Installation and configuration of the Monte Screen Recorder is easy. Simply download the .jar file from here, add it as a dependency to your project and you’re all set.

However, please note that using Monte Screen Recorder, as well as playing back the screen captures afterwards, requires the TechSmith Screen Capture Codec (TSCC) to be installed.

To create a screen capture, all we need to do create a new instance of the ScreenRecorder object with the settings we would like to use. For example:
Continue reading