(This post was written by Bilal Issa from the  LeanFT R&D Team)

Intro

With every passing day, users are getting more demanding.  They expect to access modern Web apps using different browsers, and developers are asked to provide familiar, if not similar, behavior and user experience across devices and operating systems. As a result of these demands it is getting even harder to track and test changes with the ever growing matrix of test configurations.

Adding to this complexity is the fact that modern software development is shifting towards Continuous Delivery. This is a blessing for obvious reasons, but it introduces new challenges for several shareholders:

• The developer who wants to release content more often
• The automation engineer who needs to make sure that every change meets the safety and quality levels required
• The tester who wants to make sure the test process is optimized in terms of resources and time to completion.

Here’s where LeanFT comes to the rescue! It introduces support for running tests in parallel on the same host machine and leveraging the parallel execution capabilities baked into TestNG and NUnit 3.x. Not only does this reduce the execution time of the automated test, but it also provides better insights in a unified LeanFT run report which includes all test results.

Note: Both TestNG and NUnit 3.x are the selected frameworks which are supported by LeanFT 12.54 and higher as the leading frameworks with parallel capabilities, for Java and C# respectively. Moreover, LeanFT supports parallel runs using any framework with parallel capabilities, provided that the SDK and report are initialized correctly.

Parallel execution is supported by LeanFT for web application testing and native-mobile application testing.

In this blog, I provide step-by-step instructions on how to run LeanFT tests in parallel. I will demonstrate this using the TestNG project template. (Note: NUnit 3.x framework will not be covered in this blog.)

My setup includes IntelliJ IDEA 2016.1.1 with Lean Functional Testing 12.54.

Creating a LeanFT TestNG test

Let’s start by simply creating a new LeanFT project using the TestNG template:

1. Go to File > New… > Project…
2. Select LeanFT Test > LeanFT TestNG and hit Next.
3. In the Testing Framework section, select TestNG.
4. In the SDK section, select LeanFT and hit Next
5. Give your project a name and click Finish.

You now have a LeanFT project based on the TestNG framework, where you can start writing your tests.

In the example below I present a simple test that runs in parallel on three different browsers.
To write the test, I used the Object Identification Center (OIC) which is very handy when it comes to writing descriptions to accurately identify test objects (To learn more on how to use the OIC, see this blog post: Creating robust tests with LeanFT’s Object Identification Center).

@Test
public void test_Chrome() throws GeneralLeanFtException, ReportException {
    testBrowser(BrowserType.CHROME, "TABLETS");
}

@Test
public void test_Firefox() throws GeneralLeanFtException, ReportException {
    testBrowser(BrowserType.FIREFOX, "MICE");
}

@Test
public void test_IE() throws GeneralLeanFtException, ReportException {
    testBrowser(BrowserType.INTERNET_EXPLORER, "LAPTOPS");
}



public void testBrowser(BrowserType browserType, String category) 
           throws ReportException, GeneralLeanFtException {
    
    String url = "http://www.advantageonlineshopping.com/";
    Browser browser = BrowserFactory.launch(browserType);
    browser.navigate(url);

    WebElement categoryElement = browser.describe(WebElement.class, 
         new WebElementDescription.Builder()
             	.className("shop_now roboto-bold ng-binding")
                .tagName("SPAN")
                .innerText(category).build());

    categoryElement.click();
    
    browser.sync();

    WebElement categoryLink = browser.describe(WebElement.class, 
         new WebElementDescription.Builder()
               .className("select  ng-binding")
               .tagName("A")
               .innerText(category   " ").build());

    Verify.areEqual(categoryLink.getInnerText().trim(), category,     
                    String.format("Make sure you've reached the %s category page", category ));

    browser.close();
}

The flow above is rather trivial: it opens Chrome, Firefox, and Internet Explorer, navigates to AdvantageOnlineShopping.com on each browser, selects a product category, clicks it, and verifies that you’ve landed on the correct page.

Running the test in parallel

The quickest way to start is to modify the ‘testng.xml’ file, located in the resources folder of the project, to let it run the class methods in parallel. Here’s the slightly modified xml: 

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >

<!-- TestNG Documentation: http://testng.org/doc/documentation-main.html -->
<!-- Change "false" to "tests", "methods", "classes" or "instances" for running in parallel. See more details in the TestNG documentation. -->
<suite name="LeanFTSuite" parallel="methods" thread-count="3" verbose="2">
  <test name="LeanFtTest">
    <classes>
      <class name="com.leanft.blogs.parallelTesting.LeanFtTest" />
    </classes>
  </test>
</suite>

Note the highlighted parts, which indicate that the parallel level is “methods” with a thread count of 3. Also note that the methods are of the specified class (which is the one who includes the code above).

To learn more on how to write parallel tests in TestNG you can go through the official TestNG documentation at the following link: http://testng.org/doc/documentation-main.html#parallel-running

Run the test as you would normally run any other TestNG test based on a suite xml. Do the following:

1. In the menu, go to Run > Edit Configurations…
2. Click the ‘ ’ icon (or Add new configuration).
3. Select TestNG from the list.
4. (Optional) Name your run configuration.
5. Choose Suite, select your testng.xml in the resources folder (see highlighted below), and hit OK.

       6. Select your configuration and hit the run button.

That’s about it! You have successfully run your first LeanFT TestNG test in parallel.

Tip: In IntelliJ IDEA you can also run the test by right-clicking the ‘testng.xml’ file and then Run.

Running parametrized tests in parallel using suite xml

To step up the game, let’s discuss the use of parameters so that we end up with a cleaner, more maintainable testing code.

1. First, let’s refactor the test code above to have only one method. We’ll use the TestNG @Parameters annotation to do this:    

@Test
@Parameters({"browser-type", "product-category"})
public void testBrowser(BrowserType browserType, String category)  throws ReportException, GeneralLeanFtException {
  . . .  

}

 2. Now we’ll create a new ‘testng-with-params.xml’ file which has a suite with the parallel=”tests” attribute. Here’s an example: 

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >

<suite name="LeanFTSuite" parallel="tests" thread-count="3">
 <test name="BuyTabletsOnChrome">
  <parameter name="product-category" value="TABLETS" />
  <parameter name="browser-type" value="CHROME" />
  <classes>
   <class name="com.hpe.leanft.blogs.parallelTesting.LeanFtTest" />
  </classes>
 </test>
 <test name="BuyMiceOnFirefox">
  <parameter name="product-category" value="MICE" />
  <parameter name="browser-type" value="FIREFOX" />
  <classes>
    <class name="com.leanft.blogs.parallelTesting.LeanFtTest" />
    </classes>
 </test>
 <test name="BuyLaptopsOnIE">
  <parameter name="product-category" value="LAPTOPS" />
  <parameter name="browser-type" value="INTERNET_EXPLORER" />
  <classes>
   <class name="com.leanft.blogs.parallelTesting.LeanFtTest" />
  </classes>
 </test>
</suite>

Each TestNG test level has a set of two parameters with the same name as specified in the @Parameters annotation above the test method.

3. Let’s run the test: Right-click the ‘testng-with-params.xml’ and click Run.

4. We’ll open the report from LeanFT > View Last Run Results.
    Note that the unified run results include all the TestNG tests which were run in parallel.

Test parametrization using TestNG Data Provider

To run methods in parallel without using external suite xmls, you can use a “data provider” which allows you to inject the parameter values via code only.

Here’s how: 

@DataProvider(name = "browser-provider", parallel = true)
public Object[][] provide() throws Exception {
    return new Object[][] {
            {BrowserType.CHROME, "TABLETS"},
            {BrowserType.FIREFOX, "MICE"},
            {BrowserType.INTERNET_EXPLORER, "LAPTOPS"}};
}

@Test(dataProvider = "browser-provider", threadPoolSize = 3)
public void testBrowser(BrowserType browserType, String category) throws ReportException, GeneralLeanFtException 
{
   ...
}

Note the use of ‘parallel = true’ in the @DataProvider annotation. This is essential for the tests to run in parallel. The ‘threadPoolSize’ attribute in the @test annotation is optional, but it is recommended that you specify how many threads should be used to run your tests.

While using suite xmls only enables running parametrized TestNG tests in parallel, inline code parametrization provides even better run results, as the TestNG data provider dynamically creates a single test method container with multiple methods, and provides each method with the respective set of parameter values.

Here’s how the results are organized now:

Note that the ‘testBrowser’ method now aggregates several method-case elements. Also note that under the main suite there’s now a single TestNG test which aggregates all the runs.

Where applicable, it is recommended to use the @dataProvider approach, as it leverages the best of TestNG and provides better LeanFT run results.

About LeanFT

 Lean Functional Testing (LeanFT) is a new functional test automation solution, built specifically for continuous testing and continuous integration.

LeanFT enables advanced automation engineers and dev-testers in Agile teams to leverage the standard development IDEs for test automation.

LeanFT integrates with Visual Studio, Eclipse, and IntelliJ IDEA, and enables creating automated functional tests using modern programming languages (C#, Java, and JavaScript). In addition to its .NET, Java, and JavaScript SDKs, LeanFT provides plugins for these IDEs, which suggest powerful tools like the Object Identification Center and Application models.

You can find more information about LeanFT, in the LeanFT Help Center

Try Lean Functional Testing for yourself!

Feel free to leave a comment in the box below.