Up and running with: TestNG

This is the fifth article in our series on new, popular or otherwise interesting tools used in test automation. You can read all posts within this series by clicking here.

What is TestNG?
From the TestNG.org website: TestNG is a testing framework inspired from JUnit and NUnit but introducing some new functionalities that make it more powerful and easier to use, such as annotations and support for data driven testing.

Where can I get TestNG?
TestNG can be downloaded from this site. For Eclipse users, it is highly recommended to install the TestNG plugin for Eclipse for maximum ease of use. IDEA IntelliJ supports TestNG natively as of version 7.

How do I install and configure TestNG?
Since TestNG is supported natively from IntelliJ 7 onwards, there’s no need for additional configuration for IntelliJ users. When you install the TestNG for Eclipse plugin as described, you’re set to create your first TestNG test as well, as can be read here. In other situations, you can download the .jar files from here as well.

Creating a first TestNG test
As I have done in the past, I’ll (ab)use the ParaBank demo application at the Parasoft website for our first TestNG test. I’ll use Selenium WebDriver to perform the test steps, and will use TestNG to perform the checks that we want to do and the reporting. Let’s say we want to verify that we can login successfully to the ParaBank application, given the right credentials. A Selenium + TestNG test that performs this test looks like this (import statements removed for brevity):

public class ParabankTestNG {
	
	WebDriver driver;
	
	@BeforeSuite
	public void setUp() {
		
		driver = new FirefoxDriver();
		driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
	}
	
	@Test(description="Tests a successful login")
	public void testLoginOK() {
		
		driver.get("http://parabank.parasoft.com");
		driver.findElement(By.name("username")).sendKeys("john");
		driver.findElement(By.name("password")).sendKeys("demo");
		driver.findElement(By.xpath("//input[@value='Log In']")).click();
		Assert.assertEquals("ParaBank | Accounts Overview",driver.getTitle());
		driver.findElement(By.partialLinkText("Log Out")).click();
	}
	
	@AfterSuite
	public void tearDown() {
		
		driver.quit();
	}
}

Note that this test looks almost identical to a Selenium + JUnit test. The only difference is the use of the @BeforeSuite and @AfterSuite annotations for test setup and teardown, where we would use @Before and @After in JUnit. TestNG uses a large variety of annotations that can be used to enhance your tests and test suites.

Running the test
Again, as I’m an Eclipse user, I’ll show you how to execute your tests in Eclipse only. Please refer to the TestNG website for instructions for other IDEs.

For those that installed the TestNG plugin for Eclipse, there are two simple ways to start a TestNG test. First, we can simply right-click our test .java file in the Package Explorer and select ‘Run As > TestNG Test’. This is perfectly suitable when you have a single class containing all of your tests.

However, for larger projects, this will typically not be the case. For those situations, we can create an XML file called testng.xml that contains instructions on which tests to run and how they should be run. You can find instructions on the use of testng.xml files here. As an example, we can run all TestNG tests in a specific package using the following instructions:

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
 
<suite name="My first TestNG test suite" verbose="1" >
  <test name="Login tests"   >
    <packages>
      <package name="com.ontestautomation.selenium.testng" />
   </packages>
 </test>
</suite>

The verbose attribute specifies the verbosity of information logged to the console, where 1 is low and 10 is high.

Running a test using the testng.xml file can be done just as easily by right-clicking on it in the Package Explorer and selecting ‘Run As > TestNG Suite’. Test results can be reviewed in the ‘Results of running suite’ tab in Eclipse. Note that using meaningful names for tests and test suites in the testng.xml file make these results much easier to read and interpret:

TestNG test results in Eclipse

Using the testng.xml file also makes it easy to specify exactly which tests are run in a specific test suite, and also in which order they are executed. By default, the order in which the tests appear in the testng.xml file defines the order in which the tests are run.

Useful features
one very useful feature of TestNG is the ability to easily parameterize tests from testng.xml. For example, if we want to parameterize the input values from the login test above, we first redefine the method in the test class to take parameters:

@Parameters({"username","password"})
@Test(description="Tests a successful login")
public void testLoginOK(String username, String password) {
		
	driver.get("http://parabank.parasoft.com");
	driver.findElement(By.name("username")).sendKeys(username);
	driver.findElement(By.name("password")).sendKeys(password);
	driver.findElement(By.xpath("//input[@value='Log In']")).click();
	Assert.assertEquals("ParaBank | Accounts Overview",driver.getTitle());
	driver.findElement(By.partialLinkText("Log Out")).click();
}

Note the use of the @Parameter annotation to link the arguments of our test method to the parameters you define in the testng.xml file:

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
 
<suite name="My first TestNG test suite" verbose="1" >
  <parameter name="username" value="john"/>
  <parameter name="password" value="demo"/>
  <test name="Login tests"   >
    <packages>
      <package name="com.ontestautomation.selenium.testng" />
   </packages>
 </test>
</suite>

Another useful option of TestNG is the fact that it automatically generates readable HTML reports containing the test results. By default, these are created in a test-output directory relative to your project location. The HTML report generated from the test above, for example, looks as follows:
TestNG HTML test report
You can further personalize and adjust the reports as described here.

Further reading
An Eclipse project including the TestNG test I’ve demonstrated above and the reports that have been generated can be downloaded here.

Happy TestNG testing! Since I’ve got to know TestNG in the past couple of weeks, I’ve discovered quite a few interesting possibilities that I want to share with you in future posts, so stay tuned.

6 thoughts on “Up and running with: TestNG

  1. Hi bas,
    Two things:
    1)Is there any connection between the annotations we give and contents inside testng.xml file?only @Parameters has the connection ?but inside xml file it is parameter whereas inside the ParabankTestNG.java file it is @Parameters.
    2)why the annotations @BeforeSuite and @AferSuite,i mean is it the number of times it gerts executed that make our choice?i gave @BeforeMethod and @AfterMethod.But i believe i got the same output results.

    • 1) All methods annotated by @Test are seen as test methods by TestNG and will be treated as such. So if you have three methods annotated with @Test in your test class and you run that class using TestNG, three tests will show up in the reports.

      2) @BeforeSuite and @AfterSuite are used to perform setup and teardown actions for the test run. They are run exactly once before cq. after all test methods (annotated with @Test) are run. This can (in case of Selenium tests) for example be used to create or close a browser instance. I think the TestNG documentation described the difference between both annotations clearly.

  2. Hi Bas,
    Two things:
    1)Creating an XML File can be done like this right?
    From File Menu,Select New,Click Other,then select XML File and click Next right?
    2)The HTML Report is inside the generated index.html
    under test-output folder when we right click our project and click refresh right?

  3. Hi Bas,
    The key to both junit and testng frameworks actually is all about annotations and how wisely we put the code inside a method under these anotations right?The rest is just how we want to execute scripts,the order(priority),
    multiple class execution(if there are 3 classes to be executed one by one inside a package then inside the testng xml file we give
    ,) right?

    • Sort of, I guess… I’m not sure I understand exactly what you mean by ‘the key’ to either of those but a lot of functionality in them depends on the proper use of annotations and, in case of TestNG, on the use of TestNG.xml, indeed.

Leave a Reply

Your email address will not be published. Required fields are marked *