Using the ExtentReports TestNG listener in Selenium Page Object tests

In this (long overdue) post I would like to demonstrate how to use ExtentReports as a listener for TestNG. By doing so, you can use your regular TestNG assertions and still create nicely readable ExtentReports-based HTML reports. I’ll show you how to do this using a Selenium WebDriver test that uses the Page Object pattern.

Creating the Page Objects
First, we need to create some page objects that we are going to exercise during our tests. As usual, I’ll use the Parasoft Parabank demo application for this. My tests cover the login functionality of the application, so I have created Page Objects for the login page, the error page where you land when you perform an incorrect login and the homepage (the AccountsOverviewPage) where you end up when the login action is successful. For those of you that have read my past post on Selenium and TestNG, these Page Objects have been used in that example as well. The Page Object source files are included in the Eclipse project that you can download at the end of this post.

Creating the tests
To demonstrate the reporting functionality, I have created three tests:

  • A test that performs a successful login – this one passes
  • A test that performs an unsuccessful login, where the check on the error message returns passes – this one also passes
  • A test that performs an unsuccessful login, where the check on the error message returns fails – this one fails

The tests look like this:

public class LoginTest {
	
 WebDriver driver;
     
    @BeforeSuite
    public void setUp() {
	         
        driver = new FirefoxDriver();
        driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
    }
	     
    @Parameters({"incorrectusername","incorrectpassword"})
    @Test(description="Performs an unsuccessful login and checks the resulting error message (passes)")
    public void testFailingLogin(String username, String incorrectpassword) {
	         
        LoginPage lp = new LoginPage(driver);
        ErrorPage ep = lp.incorrectLogin(username, incorrectpassword);
        Assert.assertEquals(ep.getErrorText(), "The username and password could not be verified.");
    }
	    
    @Parameters({"incorrectusername","incorrectpassword"})
    @Test(description="Performs an unsuccessful login and checks the resulting error message (fails)")
    public void failingTest(String username, String incorrectpassword) {
	         
        LoginPage lp = new LoginPage(driver);
        ErrorPage ep = lp.incorrectLogin(username, incorrectpassword);
        Assert.assertEquals(ep.getErrorText(), "This is not the error message you're looking for.");
    }
	    
    @Parameters({"correctusername","correctpassword"})
    @Test(description="Performs a successful login and checks whether the Accounts Overview page is opened")
    public void testSuccessfulLogin(String username, String incorrectpassword) {
	         
        LoginPage lp = new LoginPage(driver);
        AccountsOverviewPage aop = lp.correctLogin(username, incorrectpassword);
        Assert.assertEquals(aop.isAt(), true);
    }
	     
    @AfterSuite
    public void tearDown() {
         
        driver.quit();
    }
}

Pretty straightforward, right? I think this is a clear example of why using Page Objects and having the right Page Object methods make writing and maintaining tests a breeze.

Creating the ExtentReports TestNG listener
Next, we need to define the TestNG listener that creates the ExtentReports reports during test execution:

public class ExtentReporterNG implements IReporter {
    private ExtentReports extent;
 
    @Override
    public void generateReport(List<XmlSuite> xmlSuites, List<ISuite> suites, String outputDirectory) {
        extent = new ExtentReports(outputDirectory + File.separator + "ExtentReportTestNG.html", true);
 
        for (ISuite suite : suites) {
            Map<String, ISuiteResult> result = suite.getResults();
 
            for (ISuiteResult r : result.values()) {
                ITestContext context = r.getTestContext();
 
                buildTestNodes(context.getPassedTests(), LogStatus.PASS);
                buildTestNodes(context.getFailedTests(), LogStatus.FAIL);
                buildTestNodes(context.getSkippedTests(), LogStatus.SKIP);
            }
        }
 
        extent.flush();
        extent.close();
    }
 
    private void buildTestNodes(IResultMap tests, LogStatus status) {
        ExtentTest test;
 
        if (tests.size() > 0) {
            for (ITestResult result : tests.getAllResults()) {
                test = extent.startTest(result.getMethod().getMethodName());
 
                test.getTest().startedTime = getTime(result.getStartMillis());
                test.getTest().endedTime = getTime(result.getEndMillis());
 
                for (String group : result.getMethod().getGroups())
                    test.assignCategory(group);
 
                String message = "Test " + status.toString().toLowerCase() + "ed";
 
                if (result.getThrowable() != null)
                    message = result.getThrowable().getMessage();
 
                test.log(status, message);
 
                extent.endTest(test);
            }
        }
    }
 
    private Date getTime(long millis) {
        Calendar calendar = Calendar.getInstance();
        calendar.setTimeInMillis(millis);
        return calendar.getTime();        
    }
}

This listener will create an ExtentReports report called ExtentReportTestNG.html in the default TestNG output folder test-output. This report lists all passed tests, then all failed tests and finally all tests that were skipped during execution. There’s no need to add specific ExtentReports log statements to your tests.

Running the test and reviewing the results
To run the tests, we need to define a testng.xml file that enables the listener we just created and runs all tests we want to run:

<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >

<suite name="Parabank login test suite" verbose="1">
	<listeners>
		<listener class-name="com.ontestautomation.extentreports.listener.ExtentReporterNG" />
	</listeners>
	<parameter name="correctusername" value="john" />
	<parameter name="correctpassword" value="demo" />
	<parameter name="incorrectusername" value="wrong" />
	<parameter name="incorrectpassword" value="credentials" />
	<test name="Login tests">
		<packages>
			<package name="com.ontestautomation.extentreports.tests" />
		</packages>
	</test>
</suite>

When we run our test suite using this testng.xml file, all tests in the com.ontestautomation.extentreports.tests package are run and an ExtentReports HTML report is created in the default test-output folder. The resulting report can be seen here.

More examples
More examples on how to use the ExtentReports listener for TestNG can be found on the ExtentReports website.

The Eclipse project I have created to demonstrate the above can be downloaded here.

214 thoughts on “Using the ExtentReports TestNG listener in Selenium Page Object tests

  1. Thanks for posting this article Bas

    You have created driver in before suite. How should we handle scenario where we use multiple browsers?

    What is best strategy to launch them?

    I was using .net but going to use java with with extent reports as it is getting all updates before .net. Looking forward to use your approach!

    • Hi Karthik,

      one approach you might take is to create separate testng.xml files for every browser and parameterize the browser type. Then, in the @BeforeSuite, you can simply instantiate the desired browser based on the value of that variable using an if-then-else or case-statement. If you want to use just one testng.xml file, you could also parameterize the browser to be used on test level. Just remember not to instantiate the browser in the @BeforeSuite, but in the @Test methods themselves (possibly by using a helper method to prevent code duplication).

      • Hi Bas,
        In the ExtentReports version you used,the generating report has two options on the left side menus for Images view and Screencast view.what are these for?How we can use these?

        • If your reports contain screenshots or screencasts, you can use these buttons to generate an index of them at the top of your report. In that way, you don’t have to scroll through all of a very long report to find a specific screenshot or screencast.

          • hey bas,one more thing.since the working of each website is different we cannot make a common or universal framework that automates every website right?

          • No, and I would not recommend trying this to anyone. As you said, every application and every website is different, so each of them has its own requirements and therefore an approach that works well for one application might be rubbish for another. As Alan Richardson once said in a keynote I attended: don’t build frameworks, solve problems.

            Having said that, obviously there are some good practices (I don’t like the term ‘best practices’) that can be applied to a wide variety of applications and therefore to many test automation projects, such as using Page Objects, the DRY (Don’t Repeat Yourself) principle, etc.

  2. Hi Bas,
    This post is good to me for how to handle the test result in TestNG result collection listener and pass to extent report to generate html file.

    But I would like to know, if I want to add the test step log, for example, add each action log when test runing, click action, search acton:
    =====>Test Step Log start:
    Click: [Home–>Menu–>Login] button
    <=====Test Step Log end

    as I checked that generate extent report html file is the all tests completed (class "ExtentReporterNG"), so have any idea on my request?

    Hope can hear from you soon.

      • Hi Bas,
        I noted that in your @Test annotation includes description and test name fields, if add those two fields to:
        ExtentTest test = extent.startTest([Test Name] + ” – ” + [Method Name], [Test Description]);

        I think it is better, so for this have any idea to handle it?

        • Hi Anson, to get both the test method name and the test description in your report try this:

          test = extent.startTest(“[” + result.getMethod().getMethodName() + “], [” + result.getMethod().getDescription() + “]”);

          That worked for me. I’m not sure what you meant with [Test Name], I couldn’t find a TestNG test property like that.

          • Hi Bas,
            Sorry I missed something to talked about startTest.

            I meant that in TestNG, the @Test annotation example:

            @Test(testName = “Test Sampel”, description=”TestNG Sample”)
            public void f1(){

            }

            I meat that passed “testName” and “description” parameter’s value instead of getMethodName()

          • Hey Anson,

            the testName property of the @Test annotation is ignored when you place @Test on the method level like you do. It only works on class level, see also http://testng.org/javadoc/org/testng/annotations/Test.html#testName(). This means you can’t use this method to define test names on a method-level basis and have them show up in the reports. I’m not sure why you would want to do that anyway, if you already have a description field at your disposal.

            As I said in my previous comment, you can include the test description using result.getMethod().getDescription().

          • Sorry, I should talk how to add test step log in extent report.

            for example, in test running of selenium in TestNG, have a click action that click “Submit” button of home page, so in normally, the log should be:
            @Test
            public void f1(){

            test.log(LogStatus.INFO, “Click: [Home–>Submit] button”);

            }

            for this case, for your case handler, how to add this to extent report?
            as you know, the extent report instance created after all tests completed (class “ExtentReporterNG”).

          • I don’t think that’s possible, since the individual test methods are not aware of the ExtentReports object and therefore you can’t make a call to it from within the test methods.

  3. Hey Bas,
    I need a help.i need to capture all the http requests send by all the elements including link,images,css etc.
    on a webpage when that webpage loads.How can we do that in selenium webdriver?

    • Selenium doesn’t do that for you (it just automates browser interaction) but there might be some sort of Java library (assuming you’re using Java) that will. I’ll have a look as I can see this might be useful for some purposes..

      If you’re testing by hand you might want to try Fiddler. Too bad their FiddlerCore is available for .NET only, that would be a great solution..

        • Hmm.. I don’t think JAMon will do what you want but there’s good news too, it looks like BrowserMob is what you need. Here’s an example of how to use it combined with WebDriver.

          Again, I’ll try it out myself as well and see if I can make a small blog post out of it..

          • OK, BrowserMob is pretty easy to use with Selenium, I recommend you check it out. It results in a HAR (HTTP archive) which you can write to file (it’s JSON) and subsequently read and parse if you want using any of the tools mentioned here.

  4. Thank you bas for your effort :).yaa it will be really helpful for me if do a blog topic on BrowserMob.
    Actually in our company before uploading the files finished projects,they wanted to make sure there is no httpresponsecode errors like 402,403 etc.Everytime using the firebug net tab to check the response code especially if there are lot of pages,lot of links,lot of requests is difficult right?

  5. They suggested a logic.collect all links from home page and filter out unnecessary urls or links,That
    is only collect the required navigational links to each page.Then store it an array.
    After that by viewing he view source of each page, identify all the common tags(links,script,img) which contains http links and store this in another array.
    Then each time we pass the navigational urls stored in the first array inside a loop,design it in such a way that it calls another function.
    so inside this function we write the code for getting httpresponsecode(this is inside another function) when each time we pass the urls stored in second array.
    we can use this for testing different websites that only has images and stuffs that is no login form πŸ™‚

    • If all you need to look for is HTTP 4xx response codes there might be a pretty easy way to do that. As I said, I managed to record all HTTP traffic in a .har file using BrowserMob, I just haven’t found a way to parse it sensibly yet. All parser I tried generated error messages saying there is something wrong with the generated .har. But it’s plain text, so you might just as well do a crude search on it for unwanted response codes..

  6. Hi Bas,
    Somehow me and with the help of a friend implemented
    the logic using Lists.code to get httpresponse code using Lists to store and later looping through it by using for loop was on the stackoverflow website.so i am sending you such an error code captured extent report to bas@ontestautomation.com.The filename is bladerunner.html.Please download it and click on the link->Test case-Getting all httprequests and its statuscode when page is loaded,and take a look ok?

    • Hi Sherin,

      that looks pretty good! Could you please share the code you used to build this kind of report? I’d like to use that some day.. Looks like a decent alternative to using HTTP Archives..

  7. Hi Bas,
    I will share it on monday as soon as i get to the office.The code is in office.But remember it is not a very decent code.There are lot of if statements to filter out unwanted urls and null ones.
    Maybe you can work on it and improve πŸ™‚

    • And the most important thing is i only checked urls of three websites.So not sure it is gonna work for all websites without login forms πŸ™‚ .So need to test more websites πŸ™‚

      • Hi Bas,
        Since there are free websites available online that tests performance and all and return suggestions,for getting http response code apart from firebug also there could be such websites right?if not my code can be useful for others right?

  8. Since you liked my report,i have some queries.suppose we have two different websites to test. and both has login forms and suppose the name of textboxes and every attribute associated with it like id,classname, tagname,name,linked text,xpath are different for both login forms of websites.since in selenium webdriver we use any of the above attribute in findElementBy method to identify all webelements on a page,it is not possible to implement the httpresponsecode checking scenario here right?we can create a code for only getting response code of all websites without login forms right?

  9. Now i think website with login forms also could work in my code.Now i believe it is showing the httpresponse code when home page or login form page is loaded in case of website with login forms.i don’t know if this is possible or not but if we could write a code inside an if statement to check if the url we statically give contains a login form or not,if there is a login form then use findElement by method to login and continue the rest of the process in the code ,else directly go to the rest of the process in the code maybe it can work.Try that also bas ok?

    • I’m not sure I understand what you mean with your question about the login forms, but let me have a look at the code this week.. Curious to see how you have done this.

      • just like your paramount website.we have to login with username and password in the login form right?That is what i meant πŸ™‚ .The real reason i am so happy now is that i was very weak in coding in college days.Now i think if you have a passion,not let go attitude anyone can code πŸ™‚

  10. Hi Bas,
    Today i had to make another program from that code.And i made a mess of it.soon i will correct it and send to your mail.what you have in mind is the correct method.Browsermob method and HRA files.
    My code if has to work in a website with login feature,we have to provide the username and password in the code by using driver.findeby element method.There is one more serious issue.It can only find urls that are in the view page source code.some urls maybe hidden inside css,images,styles etc.But everything is shown in the net tab of firebug tab.i wonder how πŸ™‚

  11. Hi Bas,
    This is the first part-finding all ahref links.
    package http;

    import java.util.List;

    import org.openqa.selenium.By;
    import org.openqa.selenium.WebDriver;
    import org.openqa.selenium.WebElement;
    import org.openqa.selenium.firefox.FirefoxDriver;

    public class statuscode {
    private static String[] links=null;
    private static int linksCount = 0;
    public static WebDriver driver;

    public static void main(String[] args) {
    driver = new FirefoxDriver();

    driver.get(“http:bladerunner-local.com”);
    List linksize = driver.findElements(By.tagName(“a”));

    linksCount = linksize.size();
    System.out.println(“Total no of links Available: ” + linksCount);
    links = new String[linksCount];

    for (int i = 0; i < linksCount; i++) {
    String nullresult=linksize.get(i).getAttribute("href");

    if((nullresult==null)||(nullresult.contains("#"))||(nullresult.contains("javascript:void(0)")))
    {
    }
    else
    {
    links[i] = linksize.get(i).getAttribute("href");
    System.out.println(links[i]);

    }
    }

    }
    }

      • Thanks. I’ll try looking at the .HAR method again in a couple of days as that is the preferred method as you stated correctly. I need to figure out why the HAR I generated can’t be parsed.

  12. Hey Bas,
    Have you tried the program in this link-http://www.supermind.org/blog/968/howto-collect-webdriver-http-request-and-response-headers

  13. Hi Bas,

    I used the ExtentReporterNG expained above and it works for me. But when I see the fields “Total Time Taken”, “Start”, “End” the displayed result is completely misleading on the Dashboard. I suppose to get total time taken for the entire execution, in my case it will be around 2 min but whenever i run the script time shown is not more than five second. On the other hand if i see the total time taken by each test case, its properly working. Am I missing anything ?

    • Hi Anil,

      to be honest I have no idea how that time is calculated. I hate answering a question with another question but what are you going to use these timespans for? If you want to do performance testing Selenium isn’t exactly the right tool. If you just want a general measure of how long it takes to run a set of tests, you could just as well go by the creation time of the html reports..

  14. one more thing.The package names i gave was listener,pages and tests.so should i make changes somewhere else?Actually the ExtentReports HTML report is not getting created in the default test-output folder,what could be wrong?

  15. Sorry now only i noticed.Instead of @Beforemethod, @Aftermethod and all you used @Beforesuite etc.etc.why?Also why @override anotation?

    • @BeforeSuite is used to execute code exactly once at the start of a test suite. You can do general test setup there.

      The @override method is used to override a method definition from a superclass (inheritance). I’m sure Google will tell you all there is to know, both concepts are fairly often used and not specific to this example.

      • Actually i created testng.xml file manually,but when i open it in Text Editor,it does not get highlighted in blue colour(there is no color anywhere).Is that a problem?

          • valid?you mean when i rightclick on testng.xmal file and cllck Run as TestNG file it is running fine and correct ExtentReport is generated,then it is ok?
            I was refering to the file generated in eclipse when we double click or rightclicking and opening in Text Editor.

      • I just searched about IReporter and found It implements the definition for the method generateReport of the IReporter interface.
        so if @override is all about putting the same method in superclass again in the subclass or childclass,(if i am correct)i should see two same methods of public void generateReport(List xmlSuites, List suites, String outputDirectory),right?

        • Yes, but remember it’s not about putting the exact same code twice in different classes, it’s about changing the function of a method of a superclass. They do have the same constructor, though.

  16. Hi Bas,
    Actually you deliberately gave a wrong error message and failed one test right?Actually in both cases,the error message returning is the same right?
    well if i understood your code correctly,after a login (successful or unsuccessful),you are storing the title of that page in a string variable -TITLE.But this has to be called three times right,but i think you are calling it only once.

    • Yes, there is one deliberate error in there, I usually do that to show what the behaviour is in case a test fails. I think that is more interesting than to see only green checkmarks..

  17. Hi Bas,
    I have sent you a slightly changed results of the ExtentReport which contains the output of your above program.can you please check it and say if you find that more simpler and understandable to a person who reads the ExtentReport?

  18. Hi Bas,
    I believe this is the code-
    Assert.assertEquals(aop.isAt(), true);
    that calls the below get Title function
    public boolean isAt() {

    return this.driver.getTitle().equals(TITLE);
    }
    But shouldn’t this be called inside all three testcases right?

      • Ok,maybe i red your testcases wrong. There are three testcases and we pass incorrect credentials as parameters from testNG.xml file to two of them and correct credentials to the other one right?
        But from a logical point of view we have to check title for all the three testcases right?
        Maybe you are checking it but i couldn’t find how it is done in your code.That is what i was trying to say πŸ™‚

  19. if we have to implement Extent Report as a listener then testNG.xml file,passing parameters inside testNG.xml and code to implement IReporter is a must right?
    i mean suppose i want to take input from an Excel Sheet which also contains details of Testcases and all like in the Extent Report html file i have send you,without testNG.xml file it is not possible to implement right?

    • I can’t say without actually trying that out, but I can’t see why it wouldn’t work off the top of my head..

      Oh and there’s never a ‘have to’ as in ‘we have to implement this and that in such and such a way’. There’s options. Some are better than others.

  20. if we have to use both testNG and Extent Report in an automation script in java,it only makes sense when we implement Extent Reporter as a listener of testNG right?
    i mean without the IReporter and testNG.xml file we can also just write code for Extent Report under@Test annotation and achieve the required Extent Report output right?,which i guess won’t make any sense right?

    • Why wouldn’t that make any sense? In that case you’d still get the benefits of both libraries. I use the same approach in my current project (except instead of TestNG I use NUnit since I am working with C#).

      • ok.so the current project you are working on, what is it about?(if you can share)
        Both libraries?-you meant libraries of TestNG and Extent Report?
        What i thought was there is a standard way for automating a full website.That is using page object model,using any of the three framework model(data driven,keyword driven and hybrid framework),using TestNG or Junit framework,using Excel Sheet for reading input and an Extent Report for geetting the result reports.But that necessarily may not be the case right?i mean the project you currently working may or may not follow the three framework model i said above right?

        • I’m setting up a automated testing solution for a web application for a client. It’s based on Selenium, SpecFlow (the C# version of Cucumber) and NUnit and uses ExtentReports for creating HTML reports.

          There is no standard way of automating a web site. Every site has its own quirks and timing issues. It’s true that a lot of generic concepts are applicable (I use Page Objects, LoadableComponents etc.) but it’s an illusion to say I can use the exact same code for another application. If only because another client will have his own requirements with regards to ease of use, reporting and maintainability.

  21. There is something called @Dataprovider which is very helpful right?when i searched for implementing datadriven framework using TestNG,most of them have used @Dataprovider.so i guess if we have to implement any of the framework model(Datadriven or keyword or hybrid)using TestNG,then we should use @Dataprovider which stores the input from Excel sheet in an array and passes it and calls the corresponding function the number of times the excel sheet contains data filled rows right?

  22. ok.if we implement Extent Report using Listener,then only we can write to Extent Report after checking the status of TestNG Assert.assertTrue,like below-
    if (Assert.assertTrue(“original string”,”expected string”))
    {
    Logger.log(Logstatus.PASS,”String message”)
    }
    else
    {
    Logger.log(Logstatus.FAIL,”String message”)
    }
    correct?
    I am not sure bas you are checking like this in your TestNG Listener Program,but without listener the only option i think is suppose i am checking the title of the facebook page for a successful login or not then is the below code right?
    Note-Here i am not using Assert.assertTrue of TestNG.

    String title=driver.getTitle();
    if(title.equals(“Facebook”))
    {
    Logger.log(Logstatus.PASS,”Login Successful”)
    }
    else
    {
    Logger.log(Logstatus.FAIL,”Login Failed”)
    }

    • If you use the ExtentReports TestNG listener, you don’t need to call Logger.log(), that’s done automatically for you once you call the TestNG Assert methods. That’s the whole point of doing it like this.. Try the code and you’ll see that an ExtentReports report is created, even though I don’t use calls to ExtentReports in my tests.

      • ok.Regarding what you said on clinging to a standard program for every automation,i think now i see your point.
        There are so many ways to solve a problem.if i have three different websites doing the same thing,i should not write three different programs,one is enough and each time just change the url inside driver.get.As simple as that.same in the case of functions.if needed instead of writing it once again,call it and also give function names and variable names something related to the thing you automate.so these are things i should follow even if i automated something by writing simple piece of code inside public static void main(),right?

  23. Hi Bas,

    I am trying to implement extent report on my project. Project is already implemented and failed test cases screenshot we are capturing in the listener. My problem is I am not able to fetch those screenshots in the test method level in the report. Could you please help me on this how would I able to get the screenshot at runtime from listeners into the report. I am using the same ExtentReporterNG file available in the presentation. I am able to get the exception text but I want to get the screenshot also. Please help!!

      • I don’t think you can do that. When you use ExtentReports as a TestNG listener, the report is built using only data that is provided by TestNG. As TestNG does not support creating and passing screenshots, ExtentReports cannot access these.

        Also, since the ExtentReports report is generated after test execution has completed (instead of built up during test execution when using it in the regular, non-TestNG-listener way) your screenshots probably wouldn’t reflect the desired browser state anyway..

  24. Hi Bas,

    I can see the failed screenshot names are displayed in my extentReport. but they are empty.

    there is no solution for getting the screenshot from testng classes to extentReport?

    • As I said in another reply:

      I don’t think you can do that. When you use ExtentReports as a TestNG listener, the report is built using only data that is provided by TestNG. As TestNG does not support creating and passing screenshots, ExtentReports cannot access these.

      Also, since the ExtentReports report is generated after test execution has completed (instead of built up during test execution when using it in the regular, non-TestNG-listener way) your screenshots probably wouldn’t reflect the desired browser state anyway..

      So unless you extend TestNG itself to support screenshots, I don’t think there’s a way to have screenshots included when you use ExtentReports as a TestNG listener.. I’d love to be proven wrong though!

  25. Hi Bas,

    I have a doubt,

    Is there any logic to get the different method names which we are invoking inside one @Test annotation

    Ex:
    @Test
    public void loginTest()
    {
    LoginPage login = new LoginPage();
    login.checkLogin();
    }

    can it possible to display “checkLogin()” in Extent Report?

    • I can’t think of a way to do that when you’re using ExtentReports as a TestNG listener. If you want to be really flexible in selecting what to add to the report I suggest you use ExtentReports in the regular way, i.e., setting up a report in @BeforeSuite, adding a test in @BeforeTest or @BeforeMethod (depending on what you want) and passing the ExtentReports and ExtentTests to each test using the ITestContext.

      This is the approach I use in my current project, although I don’t use TestNG but SpecFlow and NUnit. SpecFlow offers a similar concept with its ScenarioContext and FeatureContext, which I use to pass the ExtentTest and ExtentReports objects, respectively.

      I could create a separate blog post on how to do this but that’s probably going to take a couple of weeks as my time is a little limited and there’s a lot of other stuff waiting to be written about…

  26. HI Bas,

    I am implementing extent reports for Mobile Automation .
    Could you please let me know how to take screen shot for failed scenarios.
    regards
    sree

    • If you use ExtentReports as a TestNG listener (and you do judging from where you place your comment): you can’t. The TestNG listener does not support screenshots.

      If you use ExtentReports in the default manner: a quick Google search should reveal how to create screenshots for your mobile automation tool of choice. I have never done that so I’m afraid I can’t help you out any further.

  27. Hello Bas,

    I have implemented Extent Report at the listeners level on my project.

    But now i want to integrate it with Extent Report logging also in the script level so that all loggings along with failures will come in the report.

    I have tried to implement it but single method is coming twice in the report 1 from the logging and another from the listeners. Could you please help me on this.

    Test Code below for the listeners:

    I am using single instance for the instance report so added below method in the Extenet report.

    public synchronized static ExtentReports getReporter(String filePath) {
    if (extent == null) {
    extent = new ExtentReports(filePath + File.separator + “Extent.html”, true, NetworkMode.ONLINE);

    extent
    .addSystemInfo(“Host Name”, “Rahul”)
    .addSystemInfo(“Environment”, “SIT”);
    }

    return extent;
    }

    Also, in the script level i have added below code for logging purpose:

    public class GoogleTest extends ExtentReporterNG{

    public void googleAssertPass() {

    ExtentReporterNG.getReporter(outputDirectory);
    test = extent.startTest(“test1”, “test1 – description”);
    System.setProperty(“webdriver.chrome.driver”, “C:\\projects\\test\\chromedriver.exe”);
    driver = new ChromeDriver();
    test.log(LogStatus.INFO, “Calling google”);
    driver.get(“http://google.com”);
    test.log(LogStatus.INFO, “Verifying google Search button”);
    Boolean isFound = IsElementPresent(driver,
    By.cssSelector(“input[value*=’Google Search’]”));

    Assert.assertTrue(isFound);
    driver.quit();
    extent.endTest(test);
    extent.flush();

    }

    }

  28. Hi Bas, i have utilized your work in my current framework which works fine but the issue i have noticed when there is a Retry with Test method that time it doesn’t group in same test method, PFB screenshot with comment i have mentioned, can Skip test (due to Retry) group in same test method instead of appending alone in class.

    Screen Shot 2016-03-17 at 11.21.32 AM

    Thanks
    Vinit

  29. Hello Bas,

    Thanks for posting this blog.
    I need help on Extent reports

    Currently were it is displayed all test cases on same page, but my requ after click on Test Scenario then it has to go test cases finally should have test case data (poistive,negative&skip)

    Like, Register scenario , Resgister_Id_1 –> Test Cases id1,id2..,–>if i click Test casesId1 –> respective data

    Please replay me when you have time.
    Thanks for help.

    • You can’t do that as far as I know because you’re tied to the way TestNG reports are created when you use ExtentReports as a listener

          • Thanks for help.

            But how it would work when dynamic data. today if negative cases are increased. Could you please explain me brief.

          • I’m not sure if I understand your question correctly. If you want to add new test cases to your test suite, you can just do so right? What does that have to do with your previous question about reporting?

  30. Hi Bas,

    Thanks for this wonderful article.
    Can you tell me how I can add screenshots to the Extent Reports using the ExtentReporterNG class?

    All I want to do is to have screenshots within the Extent Report while using the Extent Report as a TestNG listener.

    • Hi Anuj, as far as I know that’s not possible.. This solution entirely relies on the API provided by TestNG and that doesn’t support screenshots. There might be a way around it but I don’t know how..

  31. Hi Bas,

    Just commented on another post of yours.

    sorry for being annoying but I am quite new to automation/extent report.

    I have used your code with minor tweeks ( path to directory + timestamping my output file name ) . How come when I run mine, my report looks different to yours? Is there a way I can change the look of the report around?

    Any help would be much appreicated, yours looks much nicer than mine and more geared towards what I need extent report for,

    Thanks,
    John

    • Hi John,

      no problem, if I wouldn’t want you to comment I would have disabled the comments!

      The layout of the reports generated by ExtentReports has changed a couple of times. Mine was generated with a version older than the latest (I believe I used 2.10). I believe that’s the reason why. I can send you the earlier library version if you want it and cannot find it online. The API has seen some (pretty minor) changes as well, so you code might not convert 1-to-1, but nothing you won’t be able to figure out I believe.

      • Hi Bas,

        Thanks very much.

        Thats no problem you don’t need to send me anything πŸ™‚ .

        I was just curious as how it was different but that explains it !

        Thanks Bas, much appreciated.

        Regards.
        John

  32. Hi Bas,
    Your article is really very useful.

    I need help to get each test case name in the report.
    For example if I have stored manual test case name in a variable for every test case.
    Now I want to show which method is for which test case in the extent Report

    How can I achieve it?

    Thanks in advance.

    • Hi Vrushali,

      your best option would be to annotate your @Test methods using the description attribute, for example:

      @Test(description = “myDescription”)
      public void myTest() {
      }

      Then you can modify the ExtentReports listener to also echo the description into your report by replacing this:

      test = extent.startTest(result.getMethod().getMethodName());

      with this:

      test = extent.startTest(β€œ[” + result.getMethod().getMethodName() + β€œ], [” + result.getMethod().getDescription() + β€œ]”);

      The [] brackets are optional, of course. You can print the test description and the method name any way you want.

      • Thanks a lot. its working.

        May be its very silly thing but I am unable to do it.
        How to use more than one condition after test annotation ?

        for exa. If I want to use description as well as priority with @Test , then how to do that

        • I think you can simply separate them with commas. This is an example from the TestNG documentation that uses three attributes:

          @Test(threadPoolSize = 3, invocationCount = 10, timeOut = 10000)
          public void testServer() {
          }

  33. How to display the method names as per their execution sequence in Extent Report ?
    In my case, sequence of methods get changed?

    • Do you mean that (for example) the tests are executed in the order A-B-C but end up in the order C-A-B in the report? I haven’t noticed that before.. Are you sure the tests are really executed in the order you think they are? With TestNG, they don’t have to be executed in the order they appear in your test class.

  34. Yes I have given the priorities.And the sequence in which they get execute is different than which shows in the report

    • OK I see. I’ll try and experiment a bit myself this week to see what the reason behind this is.. There should be a logical explanation.

  35. Hey Bas,

    Hope your keeping well. I commented previously but I am back with another question.

    Is it possible to use ExtentReport to provide a live feed of your test results?

    For example, the second the tests start running the report is published and everytime an update is made such as a test passing or failing ExtentReport is automatically refreshed so you can see your tests progressing from ExtentReport itself?

    Thanks again!

    • Hey John, no I don’t think so, the report is not actually written until the flush() method is called. There might be a way around this but I’d have to test that.. It depends on whether the ExtentReports object is ’emptied’ when the flush() is called. The report won’t refresh automatically, that I can assure, but you can use a simple browser plugin to do exactly that for you..

      • Okay thanks Bas, if you find away around it let me know.

        I was under the assumption you could publish the report after say the first test step and as each test step is finished the report would keep updating till finished?

        Thanks Bas.

          • That would be great if you could Bas.

            Would be handy to know if you could achieve it be a great feature!

            Thanks again Bas.

          • Hey John,

            done. It is actually quite simple. Just call .flush() after every test instead of once at the end of your test run. BUT you need to make sure the ExtentReports object itself is only initialized once, instead of being reinitialized at the start of every test. For example, I used TestNG. The ExtentReports is called once using @BeforeSuite, but the .flush() is called after every test using @AfterMethod. I hope this makes sense.

            The only thing that can’t be solved via code is the HTML refresh as this is outside the control of the ExtentReports library (it doesn’t know where you’ve opened the actual HTML file). But this can be taken care of by using a simple browser plugin as I said. At least for Chrome there are a lot of them, just do a Google search for ‘chrome auto refresh’.

          • Thats brilliant Bas, and that is for the Listener function is it?

            Any chance you could post some sample code if possible. I’m still quite new to Java so generally I’d having to see a working demo to have a full understand. If thats okay?

            Thanks a million Bas!

          • Actually, no. I tested it for the standalone version only. Let me have a look at the TestNG listener version.

            I’ll send you the code via email afterwards.

          • Apologies John I should have specified that. My project is HUGE the tests are nearly going into the thousands so the Listener is the most manageable solution for me.

            My email is colinoregan101@gmail.com .

            Thanks Bas.

          • Hmm.. I’m positive that this is possible but it’s a lot more work. The problem is that the solution provided in my blog post (also the solution as presented on the ExtentReports web site) only features the IReporter. This one is invoked only once, after all test methods have been run (this is just how TestNG works).

            This means that the actual ExtentReports object is only created after test run execution has been finished (in the generateReport method in the Listener definition), unlike the other solution where you have full control over when and where the ExtentReports object is created and updated. Working around this means rewriting or at the least bypassing the default way TestNG handles report creation (it stores all test results in intermediary objects and creates a report from it after test execution). This feels unnatural to me. I fully understand your requirements, but I’m not sure that it’s a good idea to implement it like this. Either that, or I have overlooked an obvious alternative..

          • Thats no problem Bas, I understand.

            I knew with reporter it may not have been do able due to having limited control over what it is happening.

            I had a go myself, but from reading your reply I probably don’t understand the code fully.

            I did the following

            buildTestNodes(context.getPassedTests(), LogStatus.PASS);
            extent.flush();
            buildTestNodes(context.getFailedTests(), LogStatus.FAIL);
            extent.flush();
            buildTestNodes(context.getSkippedTests(), LogStatus.SKIP);
            extent.flush();

            I believed that the above code would do a flush after every, pass, skip or fail which would then result in the report being updated.

            Thanks,
            John

          • No, that just results in the report being updated three times after test execution has been finished, so unfortunately not what you’re looking for..

          • Leave me know if anything springs to mind Bas.

            In the mean time I’ll work away to see if I could make a solution, Ill leave you know any progress I make if your interested.

          • Hey Bas, is it possible to use the TestNG ITestListener in conjuction with ExtentReports?

            I have found out that you can use TestNG ITestListener to achieve what I want but is there anyway of using this with ExtentReport?

            Or is there any other reporting tools you would recommend to work with this?

            Thanks Bas!

            Regards,
            John.

          • Do you mean that in addition to the way ExtentReports now implements the IReporter you’d also like it to use the ITestListener? Sure that’s possible, all you need to do is to write your own implementation of the ITestListener that does exactly what you want it do to, just as ExtentReports does with the IReporter at the moment. I’ve never done it before though, although I have a pretty clear idea of how it would work.

            Unfortunately, I don’t really have all that much time on my hands to actually do the work..

          • Thanks for the reply Bas.

            I’m just wondering can I use ITestListener instead of IReporter to listen in and create a report that can be used to monitor my tests?

            Or would I still have to use IReporter with the ITestListener?

            I’ve only noticed this morning that IReporter is part of TestNG and not only ExtentReport so that got me thinking that maybe I could use ITestListener with ExtentReport.

            I’m new to all this so I’m slowly learning.

            I just wanted to make sure it was possible or achievable before I put some time and effort into it.

            That is no problem Bas, I shall work away myself πŸ™‚

            If you do find yourself free at any stage an example would be beneficial but its no problem if you can’t!

            Thanks Bas.

            ( If I get something working I’ll keep you updated as I think it would be a useful feature to get to the bottom of ! )

          • Hey John,

            the IReporter is there to make sure the report is actually written. You could technically transfer its implementation to ITestListener but that defies the meaning of the interface a bit. But if it works for you, then by all means do. I’ll make a mental note to have a go myself if I do find some free time. Please keep me updated!

    • The masonry option has been removed from recent ExtentReports versions I believe.. I doubt there has ever been a version that included both support for Masonry and the ability to use it as a TestNG listener. You’d have to ask Anshoo directly for that.

    • Hi Saagar, how to run what? ExtentReports reports are simply generated while you run your tests so it’s as easy as running your tests through the command line.

  36. Hi Bas,
    Thanks for providing approach to integrate extent reports with TestNG project. I created it and its working seamlessly.
    Just need small information: Is there any way we can exclude test showing test steps from the report. As all my test cases are single step, so it is displaying same info in both charts.
    Appreciate your help.

    Regards,
    Srini

    • If I understand your question correctly then all you would need to do is comment out the test.log(status,message) line in the implementation of the listener. Please let me know if that is what you are looking for.

      • Hi Bas,
        Actually I am using only TestNG implementation. I am not using any extent logs. But report is displaying with steps section as well (I guess it’s default). So, I would like disable the steps pie chart and steps status info. Don’t know whether any configuration available to display only test cases status and testcard pie chart.

        Thank you,
        Srini

  37. Hi BAS,

    Testng test results(onTestPass,onTestFailed,onTes tSkipped) to stored in database on daily,weekly and monthly basis. Please let me know the code.

    • You simply want to store some numbers in a database? There should be plenty of good examples on the Internet on how to do that..

  38. Hi Bas,

    First of all it is a excellent example. I am creating a Hybrid test famework. Where every test calls action from a Keyword class where I have kept all functions mapped to each keyword. Each test steps has been written in an xls file where I have kept both action keyword, UI variables and UI object types.

    In such scenario how I can fit ExtentReports? Can you give me some idea on that. Or what could be the best practice?

  39. Thanks.. This is good example to start with.
    I have one query.
    Can we have the report with steps description, expected and actual result with screenshot under different test?

  40. Hi Bas,

    How can i get the Test name as in testng like in the extent reports.
    I want to get Login tests in the extent reports.
    I am using as a listener.

    • Hi Shaqib, I think something similar has been asked before, could you please check the comments to see if your answer is there? If not, please let me know.

  41. Hi Bas,

    The ExtentReports TestNG listener code that you have given above will list all the methods with @Test annotations.
    My requirement is that I want to display the class file names containing the methods as below:
    Class A {
    test1()
    test2()
    }

    Class B{
    test3()
    test4()
    }

    Class C{
    test5()
    test6()
    }

    I want the report to display something like below:
    Class A
    test1
    test2
    Class B
    test3
    test4
    Class C
    test5
    test6

    What changes will be required in the listener class file?

    Please advise.

    • That’s a tough one.. What you’re asking for is adding another level of reporting (the standard listener has just one: the test level). Standard ExtentReports supports grouping by category but I don’t think that it works in this way.

      I’ll try and think of a solution but I haven’t got one at the moment.

        • No, I just can’t see how this would work. Could you check the TestNG documentation to see if it’s possible to add test groups in the report? If TestNG doesn’t support that, you can’t add it via ExtentReports.

    • Hey Azgar,

      what do you mean by multiple test cases? Multiple test methods in a single test class? Or multiple test classes, each containing a number of test methods?

  42. multiple test classes each containing number of test methods and report should generate only one ..i dont know how to do that? can u please give some example..it wil help me thanks for quick response.

    • Are you using ExtentReports as a TestNG listener? If so I think you can do this by creating a TestNG.xml that runs all of your tests in a single run, this should lead to a single report containing all of your tests. If you’re not using the listener, but the standalone version of ExtentReports, you can move report creation to a method annotated with @BeforeSuite, so that it is run exactly once. Report flushing is put in an @AfterSuite method.

  43. Is assertAll() in soft assertion considers assertion in one specific test method where it is written or it considers all the assertions in the class?

  44. Hi Bas

    I have above listened and ran my testng.xml file. After execution I dont see any ExtentReportsTestNG.html being created in the test-output folder. I dont understand where am I doing it wrong
    please help

  45. Hi Bas,

    No Error Message. (As you said previous comments: The report is written to the location specified in the listener definition class, not to the test-output folder.)

    How can I specified outputfolder location or
    How to specify location in the listener definition class

  46. Hi Bas,

    I wrote ExtentReportNG.java file like below:

    public class ExtentReporterNG implements IReporter {
    private ExtentReports extent;
    @Override
    public void generateReport(List xmlSuites, List suites, String outputfolder) {

    extent = new ExtentReports(“E:\\ExtReport\\” + “ExtentReportsTestNG.html”, true);

    //test = extent.startTest(β€œ[” + result.getMethod().getMethodName() + β€œ], [” + result.getMethod().getDescription() + β€œ]”);

    for (ISuite suite : suites) {
    Map result = suite.getResults();

    for (ISuiteResult r : result.values()) {
    ITestContext context = r.getTestContext();

    buildTestNodes(context.getPassedTests(), LogStatus.PASS);
    buildTestNodes(context.getFailedTests(), LogStatus.FAIL);
    buildTestNodes(context.getSkippedTests(), LogStatus.SKIP);
    }
    }

    extent.flush();
    extent.close();
    }

    private void buildTestNodes(IResultMap tests, LogStatus status) {
    ExtentTest test;

    if (tests.size() > 0) {
    for (ITestResult result : tests.getAllResults()) {
    test = extent.startTest(result.getMethod().getMethodName());

    /*test.getTest().startedTime = getTime(result.getStartMillis());
    test.getTest().endedTime = getTime(result.getEndMillis());*/

    test.getTest().setStartedTime(getTime(result.getStartMillis()));
    test.getTest().setEndedTime(getTime(result.getEndMillis()));

    for (String group : result.getMethod().getGroups())
    test.assignCategory(group);

    String message = “Test ” + status.toString().toLowerCase() + “ed”;

    if (result.getThrowable() != null)
    message = result.getThrowable().getMessage();

    test.log(status, message);

    extent.endTest(test);
    }
    }
    }

    private Date getTime(long millis) {
    Calendar calendar = Calendar.getInstance();
    calendar.setTimeInMillis(millis);
    return calendar.getTime();
    }
    }

    But Error showing is:
    [TestNG] Running:
    E:\Download\ExtentReportsTestNG\testng.xml

    log4j:WARN No appenders could be found for logger (org.apache.http.client.protocol.RequestAddCookies).
    log4j:WARN Please initialize the log4j system properly.

    ===============================================
    Parabank login test suite
    Total tests run: 3, Failures: 1, Skips: 0
    ===============================================

    java.lang.NoClassDefFoundError: freemarker/template/TemplateModelException
    at com.relevantcodes.extentreports.ExtentReports.(ExtentReports.java:85)
    at com.relevantcodes.extentreports.ExtentReports.(ExtentReports.java:374)
    at com.ontestautomation.extentreports.listener.ExtentReporterNG.generateReport(ExtentReporterNG.java:28)
    at org.testng.TestNG.generateReports(TestNG.java:1106)
    at org.testng.TestNG.run(TestNG.java:1065)
    at org.testng.remote.AbstractRemoteTestNG.run(AbstractRemoteTestNG.java:126)
    at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:137)
    at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:58)
    Caused by: java.lang.ClassNotFoundException: freemarker.template.TemplateModelException
    at java.net.URLClassLoader.findClass(Unknown Source)
    at java.lang.ClassLoader.loadClass(Unknown Source)
    at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
    at java.lang.ClassLoader.loadClass(Unknown Source)
    … 8 more

  47. Hi Bas,

    Is this correct way to give like output folder or not?

    extent = new ExtentReports(β€œE:\\ExtReport\\” + β€œExtentReportsTestNG.html”, true);

  48. Hi Bas,

    I created with new project with extent-config.xml and dependant jars. Now its working, the problem with dependices as you said.

    Thank you so much Bas

    And could you please help me how to place company logo in extent reports 2.41

  49. Started Time is: 2016-09-10 10:33:08
    Ended Time is: 2016-09-10 10:33:08

    So Expected difference time should display (example)Total Time taken: 0h 0m 35s.

    But actual showing time displaying like:
    Total Time taken: 30m 0s (It is showing in the below Quick Test Summary)

  50. Hi Bas,

    For Ex: if I not use Testng Listeners, then it is possible to put logo in extent Listener. Like below code only i have used:

    public class ExtentManager {

    public static ExtentReports Instance() {
    ExtentReports extent;
    String Path = “./mohan1409.html”;
    System.out.println(Path);
    extent = new ExtentReports(Path, false);
    extent.config().documentTitle(“Automation Report”).reportName(“Regression”);

    return extent;
    }

    public static String CaptureScreen(WebDriver driver, String ImagesPath) {
    TakesScreenshot oScn = (TakesScreenshot) driver;
    File oScnShot = oScn.getScreenshotAs(OutputType.FILE);
    File oDest = new File(ImagesPath + “.jpg”);
    try {
    FileUtils.copyFile(oScnShot, oDest);
    } catch (IOException e) {
    System.out.println(e.getMessage());
    }
    return ImagesPath + “.jpg”;
    }
    }

    Note: Here i am not using any Testng Listener’s.

    How to write custom CSS to put logo in ExtentReports?

  51. Hi Bas,

    I am able to place company logo in extentreports 2.41 version.

    But I need to remove ExtentReport hyperlink is there at top left corner. I dont want below html code in ExtentReports 2.41


    ExtentReports

    ExtentReports

    Thanks in Advance Bas.

    • I don’t know if that’s possible. I think it isn’t.

      Personally, I think it’s only fair to leave some sort of credits to the creator of ExtentReports in the report. I’m sure you don’t NEED to have that link removed. You might WANT to, but that’s something entirely different.

  52. Hi Bas,
    I am trying to implement the logs
    logger.log(LogStatus.INFO, “Pass the URL “);
    into the test scripts, so that these logs can be observed in the HTML report. But i want to implement as method and call it in test scripts. It was very helpful with your method to get the HTML report in the same I am trying to implement the method for log. Could you please help me this.

    Regards
    Pramod SR

    • Hi Pramod,

      I’m not sure that’s possible. Unlike the regular usage of ExtentReports, when using it as a TestNG listener first all tests are run, then the report is created in one go afterwards. If you had some way of capturing all the extra info you would want to include in the report, there’s no way to determine where it needs to go exactly.

  53. After creating the class got error on followingon lines.

    1. public void generateReport(List xmlSuites, List suites, String outputDirectory) {

    Quick fix error Onhover
    Remove @Override anotation.

    2.Map result = suite.getResults();

    Quick fix error Onhover
    Remove type arguments.

  54. Hi Bas,
    On extent dashboard view, I can see the test step chart. What exactly the step signifies . Or war exactly a step is?. I want all my assert statement to be counted as steps. Is this possible ? if yes how?

    • Hey,

      every call to the test report object is a step. All your asserts are automatically counted as steps, since they’re reported by the TestNG report listener to the ExtentReports report. If you only want to have pass and fail steps in your report, simply modify your listener in such a way that those are the only statuses that are reported.

      • Hey, Thank you so much for you quick reply.

        I want only asserts should be counted as steps, not every call to test report object. so that the number of steps should be the number of asserts. May you please let me know How can I achieve this.

Leave a Reply

Your email address will not be published. Required fields are marked *