IMPORTANT NOTICE:  As of 21 February, anything posted in this community WILL NOT BE MIGRATED to our new community site.   We have pulled all user information and data from this site and are now in an approximately week long process of importing users, roles, and data to our new site.  We are leaving this site open and active so you can post and hopefully get a response until the migration is complete.   Once complete, the URL  that currently takes you to this site will take you to our new site so your bookmarks will work as always.  Please read the information on a New Login Process

Selenium Test Management

Are you doing test automation with Selenium and facing a rapidly growing amount of tests? Are you looking for some kind of test management tool to keep track of all your assets and execute them most efficiently to get the best return on your investment? 

Silk Central is the right tool for this purpose, as it enables you to not only manage and execute your tests followed by powerful result analysis, but it also introduces you to managing the software development life cycle (SDLC) where requirements, tests, results, and issues are interrelated to each other for fast feedback cycles, resulting in high quality software. 

This blog article describes how you can schedule the execution of your existing Selenium test assets (written in Java or .NET) with Silk Central.

Related blogs: Keyword-driven testing with Selenium

Prerequisites

To be able to execute your Selenium tests with Silk Central, the following artefacts need to be accessible from the Silk Central execution server: 

  • for Java the compiled .jar files
  • for .NET the .dll files
  • and all additional files needed for execution (e.g. test data) 

Silk Central, the open test management solution, supports numerous ways to access such files - for example  via direct HTTP access from a build server, FTP, SMB, a UNC share, and more.

Configure Silk Central to execute all your Selenium tests

 The following abstract explains the initial configuration steps that are required to execute your Selenium assets with Silk Central. Most of these steps are one time actions - even the test configuration itself, once configured, will automatically execute and add any newly created tests without the need of additional user interaction. Your maintenance efforts are minimal. 

Silk Central will execute your test suite based on a schedule or triggered by a build server - even in parallel against different configurations (different browsers, operating systems, languages, databases, etc.). 

Source Control

The first step is to create a Source Control profile to specify the location from which the execution servers retrieve the needed test artefacts. Select Source Control Integrations from the Tests menu and create a new source control profile. The following screenshot shows an example for artefacts that are shared via UNC:

Once you have created a Source Control profile, click Tests in the menu and create a new test container. Specify a name for the test container and select the newly created Source Control profile on the create dialog. 

The next step is to add a JUnit test for your Java-based Selenium tests, or an NUnit/MSTest test for your .NET based tests. 

JUnit

To configure your JUnit tests in Silk Central you have to: 

  1. Select the newly created test container.
  2. Click "New Child Test", specify a name, select the type "JUnit test" and click next.
  3. Leave the Test class empty to execute all JUnit tests that are found in the classpath.
  4. Specify the classpath relative to the path you have configured in the source control profile. For example, if your tests are in a jar called tests.jar that resides in the root folder and all further jars that are needed for executing the test are in the lib folder, you would configure: ./tests.jar;./lib/* 

NUnit/MSTest

To configure your .NET tests in Silk Central you have to: 

  1. Select the newly created test container.
  2. Click "New Child Test", specify a name, select the type "NUnit test" or "MSTest test" and click next.
  3. Click Browse and select the assembly containing your Selenium tests.
  4. For Nunit, specify the folder containing the file nunit-console.exe, if this is not configured in the PATH environment variable. 

The structure in the Tests area could look like the following:

Your Selenium test suite is now configured in Silk Central and is ready to be executed - how to do this is explained in the next abstract. 

Running tests

Click on "Execution Planning" in the menu and create a new execution plan, specifying your newly created test container. Assign your test on the Assigned Tests tab. 

On the Deployment tab you can specify where the tests are to be executed. Silk Central supports Windows and Linux execution servers. Below the assigned keywords you can see the list of all matching execution servers that can be used for execution.

Check Silk Central's online help if you need assistance in setting up your execution environment. 

You can now either schedule an execution or directly start it using the Run button in the toolbar. A web service API is also available, if you wish to trigger executions from another tool. 

Simultaneous cross-browser testing

If you want to execute your test suite simultaneously on different browsers, right-click your newly created execution plan and select "Convert to Configuration Suite". Then create a configuration for each browser you want to test: 

  • Select the Configurations tab.
  • Click "Add/Remove Parameters".
  • Assign a new parameter called "browser". This value will be passed to the test as a system property or environment variable and can be used to start the proper web driver. The Java code for this might look like this: 

    if ("IE".equals(System.getProperty("browser"))) {

      driver = new InternetExplorerDriver();

    } else if ("Chrome".equals(System.getProperty("browser"))) {

      driver = new ChromeDriver();

    } else { //default

      driver = new FirefoxDriver();

    }

.NET code might look like this:
 

    if ("IE".Equals(Environment.GetEnvironmentVariable("browser"))) {

      driver = new  InternetExplorerDriver();

    } else if ("Chrome".Equals(Environment.GetEnvironmentVariable("browser"))){

      driver = new ChromeDriver();

    } else {

      driver = new FirefoxDriver();

    } 

A configuration suite for testing Internet Explorer, Firefox and Chrome would look like the following:

In this example, the configurations will be executed on an execution server that has the keyword "SeleniumES" assigned. 

In order to execute your test suite on the configurations simultaneously, simply connect additional execution servers to Silk Central which each have this keyword. The configuration suite, respectively the configurations, will be distributed to all available execution servers.

Result analysis

 A picture says more than a thousand log entries. In Silk Central you can therefore record a video of your Selenium test execution or create a screenshot of the complete desktop when a test fails. This is a tremendous help when analyzing failed tests. 

To enable video recording or screen capturing,  go to the Deployment tab of your configurations. There you can select from different video recording and screen capturing options:

The Test Run Results dialog has everything you need to analyze your results. Screenshots are shown in line to get a quick overview in which state the application was when the test failed. For a more detailed analysis, you can download a video that shows step-by-step what the test did until it failed.

 

With Silk Central you can also compare test executions on different browsers. This gives you a good overview to find out if the failure is related to the used browser or if it is an application under test (AUT) issue, which fails on all browsers. 

To compare the test executions on the different browsers, go to the Runs tab of your configuration suite, multi-select the runs of interest, right-click and select "Compare Automated Test Results". In this view you can click on the status icons to get further information on the test run, for example screenshots, videos and log files.

Further benefits using Silk Central for your Selenium tests

Besides the features mentioned above, Silk Central gives you much more that helps you to deliver high quality software. To just name a few:

    • Requirements coverage: Link your tests with your requirements and get an overview on which requirements are covered by which tests and see which areas are affected when a test fails.
    • Bug tracking: Link your tests with your bugs. This trace helps the developer to reproduce a bug and verify the fix.
    • Reporting: Create customized reports for your specific information needs, from trend reports to overview or comparison reports, and more. All the required data is available in Silk Central.
    • Code coverage: See what code is covered by your Selenium tests. Silk Central delivers the following code coverage tools out-of-the-box:
    • Silk Central Java Code Analysis: To perform code analysis on Java applications
    • DevPartner .NET: To perform code analysis on .NET applications 

 

For more information on these topics, please see the Silk Central online help

Trial

A free 45 day product trial can be downloaded at:

http://www.borland.com/Products/Software-Testing/Test-Management/Silk-Central/Product-Trial

Watch the video

You can also watch the following video, to learn how to manage and execute Selenium tests with Silk Central: