6 min read time

Integrating External Test Automation Tools with ALM/QC

by   in DevOps Cloud (ADM)

Introduction

ALM/Quality Center stands at the forefront of quality and test management, empowering users to efficiently manage and track their testing activities. Its robust integration with UFT and other OpenText tools enhances visibility and ensures traceability for both manual and automated tests. But what about incorporating third-party testing tools into the mix?

In the era of Agile and DevOps, companies are eager to automate a significant portion of their testing efforts, utilizing a mix of open-source and proprietary tools. However, the true value of automated testing lies in its ability to link test outcomes with specific requirements and defects, ensuring that every test run contributes to a comprehensive understanding of the project's quality. The challenge then becomes: How can these diverse test results be aggregated and made visible within a central management tool?

Enter the OpenText Application Automation Tools Plugin for Jenkins. This powerful plugin was developed to facilitate the creation of DevOps toolchains within Jenkins, accommodating a vast array of automation tools from both third-party vendors and OpenText. This integration allows for a seamless workflow, from setting up service virtualization environments for testing to selecting and using top-tier testing tools. Moreover, it enables the direct submission of automated test results from Jenkins jobs to ALM/QC integrating these outcomes into your project's quality metrics.

We're thrilled to unveil the enhancement to this plugin: Field Mapping for Test Results. This new feature elevates automation by allowing users to define flexible mapping rules within Jenkins job configurations. Consequently, test results from any tool can be automatically associated with the appropriate test or test set in ALM/QC. This automation extends to the creation of test cases or test sets, which can now be generated on-the-fly during test execution. No longer is there a need for manual setup before running your tests.

Stay tuned for a detailed example of this feature in action, showcasing just how effortlessly your testing workflow can be optimized with ALM/QC and the Application Automation Tools plugin for Jenkins.

An example step-by-step

Whether you're working with NUnit, Selenium, Appium, or Cucumber, ALM/QC can be your hub for managing automated test results, thanks to its ability to process XML-formatted test outcomes. Here's how to get your external tests integrated using Jenkins and the Application Automation Tools plugin.

Step 1: Install the plugin

Head over to the Jenkins "Manage Plugins" section, look for the "Application Automation Tools" plugin and hit install. (Manage Jenkins->Manage Plugins)


Figure 1

Step 2: Configure the plugin

After installation, it's time to let Jenkins know where to send the test results:

  1. Navigate to "Manage Jenkins"->"Configure System".
  2. Add your ALM server details here. This is where your test results will find their new home.
  3. Don't forget to add your credentials by clicking the "Add credentials" button.


Figure 2

Step 3: Setting Up Your NUnit Test Job

Create a new Jenkins job and configure it to run a NUnit test. This is where you'll specify your NUnit console command, just like in our handy screenshot.


Figure 3

Step 4: Post-build actions

  1. First, ensure your test results are published as an XML report, this step is crucial for the next part.
  2. Head to the "Post build Actions" tab and add the "Upload test result to ALM using field mapping" action. This brand-new feature lets you map XML report fields directly to ALM/QC fields.


Figure 4

Step 5: Dive Into Field Mapping


Figure 5

Step 6: Detailed Field Mapping Configuration

  • Specify Test Folder & Test Set Folder: Define where new test cases and sets will be created in ALM if matching ones are not found.
  • Define Testing Result File Pattern: Use wildcards to identify which XML files should be processed, like "**/Unit.xml" for files containing "Unit" and ending in ".xml".
  • Map Fields Accurately: You need to map fields for the “test”, “testset”, and “run” entities in ALM, using XPath to point to the correct elements or attributes in your XML results.


Figure 6

Let’s define the mapping for "testset".

  • The “root” attribute sets the path to the target test sets in the XML test result file. Using the example in Figure 10, if I set “root” to "x:/test-results/test-suite/results/test-suite/results/test-suite", it means the test sets can be found at the XPath specified after “x:”.
  • The “name” attribute is used to specify test set name which will eventually be used in ALM. For instance, in the screenshot above I set the name attribute as "v:${BUILD TIMESTAMP} Build Verification Test". This means every time the Jenkins job is run, a test set with name like "[time stamp]Build Verification Test" is generated in the Test set Folder. You may also specify name by using the value of the XML element ‘name’ attribute, i.e. configure “name” attribute to “x:@name”. In this case, multiple test sets in ALM may be generated by the plugin.
  • The “subtype-id” attribute is another field that you need to define for 'testset'. In the example, I give it the value 'v:hp.QC.test-set.default'. To understand what value should be used, please refer to the plugin's support doc for details.

Now you have understood how to define field mappings, you can follow the same procedure to define mappings for tests and test runs as well.

Step 7: Run status mapping

Translate the test execution statuses from your framework (e.g., "Success" for passed tests in NUnit) to their equivalents in ALM, ensuring accurate status reporting in ALM.

Step 8: Execute and Review

After configuring, run your job. Post-execution, the console output will detail every action taken, from login to entity creation and any issues encountered. This output is crucial for troubleshooting any problems.

Step 9: Understanding Integration Details from Console Output

After executing the job, the OpenText Application Automation Tools plugin provides a wealth of information in the console output. This includes login details, authentication processes, entity creation, request specifics, and any errors encountered. This detailed feedback is invaluable, especially when troubleshooting failed jobs. For instance, as illustrated in our example output (Figure 7), the plugin lists any tests or runs that it couldn't create or import into ALM, offering clear insights into what went wrong and where.


Figure 7


Figure 8


Figure 9

Step 10: Review Results in ALM/QC

Upon successful completion of the Jenkins job, your test results are seamlessly integrated into ALM, with new test cases generated in the designated folder. This direct synchronization ensures that all your test outcomes are accurately reflected within ALM, enhancing your project's quality management (Figures 10 and 11 illustrate a test case generated from the XML data and its successful marking on a test instance).


Figure 10


Figure 11

Summary

The OpenText Application Automation Tools plugin is a game-changer for integrating ALM/QC with automated testing software run by Jenkins. It simplifies the setup process for Jenkins jobs to execute tests and automatically exports the results in XML format. By configuring field mapping, you can effortlessly translate these test results into corresponding entities within ALM/QC. This capability allows ALM/QC to serve as a central hub for managing a wide array of testing activities, including those utilizing automated tests from external, third-party tools.

A cornerstone of effective software quality and test management is the ability to aggregate and analyze all relevant data comprehensively and automatically. This approach not only streamlines the testing process but also significantly boosts your competitiveness in delivering high-quality software products.

Take the Next Step

Don't wait to enhance your testing workflow. Explore the plugin now, delve into the comprehensive plugin documentation for further insights, and don't hesitate to reach out to customer support for any assistance you may need in getting started.

Labels:

Application Lifecycle Management