Introduction

ALM/Quality Center is the leading tool for quality and test management enabling its users to execute and track their testing activities. ALM/Quality Center has a strong built-in integration with UFT and other Micro Focus tools to provide users with the visibility and traceability of both manual and automated tests. How about its integration with 3^rd-party testing tools?

 As Agile and DevOps became popular in recent years, companies want to automate large portion of their testing activities and they use a wide variety of tools – open-source or proprietary, to achieve this goal. However, running automated tests without keeping track of the results and connecting the dots between test coverage, requirements and defects is meaningless. They want these test results visible and traceable in a management tool, but how?

The answer is Micro Focus Application Automation Tools plugin. It is a Jenkins plugin introduced a few years ago, to enable building DevOps toolchains on Jenkins, using a wide range of automation tools from both 3^rd-party vendors and Micro Focus. With that, your toolchain can include setting up the service virtualization environment for testing, using best-of-breed testing tools of your choice, and many more. The results of automated tests run through Jenkins jobs can be sent to ALM/Quality Center or ALM Octane by the plugin, to become part of the quality metrics data.

We are excited to announce a new feature of the plugin: Field Mapping for the test results uploaded to ALM/Quality Center. This further raises the level of automation. By defining flexible mapping rules in Jenkins job configuration, the test results from any tool get linked to a test or testset automatically in ALM/Quality Center. You don’t even need to manually create test cases or test sets before executing these tests – they will be created automatically. Below you will see a working example.

An example step-by-step

We will use a NUnit test as an example to illustrate how you can integrate external tests with ALM/Quality Center.

Logically, the same approach works for any automated test run by Jenkins that exports test results in XML format. With this, many popular tools can be integrated with ALM/Quality Center, for example Selenium, Appium and Cucumber.

Install the plugin

Access the Jenkins “Manage Plugin” page (Jenkins -> Manage Jenkins -> Manage Plugins or URL http://<Jenkins_Server>:<Jenkins_Port>/pluginManager). Find and install the plugin “Micro Focus Application Automation Tools”.

Figure 1

After the plugin is installed, go to the Jenkins server configuration and add ALM server information (Menu Jenkins -> Manage Jenkins -> Configure System or URL http://<Jenkins_Server>:<Jenkins_Port>/configure).

Figure 2

Click the “Add credentials” button to set your credential.

Add Jenkins job to run a NUnit test

Add a job whose build step executes a NUnit console command like the screenshot below.

Figure 3

Post-build actions

First, the test result needs to be published in a report in XML format. This is the pre-requisite for the field mapping we’ll describe later. 

Go to the “Post build Actions” tab and add an action of “Upload test result to ALM using field mapping” – this is a new type we just released. The configuration of this action will specify how fields in the XML report get mapped to ALM/Quality Center ones.

Figure 4

Configuration for “Upload test results to ALM using field mapping”

Figure 5

Above is the screen for setting the post-build action “Upload test results to ALM using field mapping”. There are several parts you need to fill in.

Test Folder and Test set Folder

The plugin first tries to find existing ALM test cases matching the test results. And if none can be found, it generates new test cases for these results and put them into the Test Folder specified. Similarly, the plugin searches among the current test sets according to the defined field mapping.  If none matching the test results, it will generate new test sets for this part of the results, and put them into the Test set Folder specified.

Testing result file

It specifies the filename pattern of the test result files to be processed. This area accepts wildcards, so "**/*Unit*.xml" means all the files whose name contains “Unit” and has an “.xml” suffix.

Field mapping

You need to define the mappings for three types of ALM entity: "test", "testset" and "run". If you are not familiar with these concepts, click on the hyperlinks on them to learn.  

The plugin uses XPath to locate XML element or attribute in the test result files. To help understand the mapping, here’s an example test result file in XML:

Figure 6

Let’s define the mapping for "testset".

• The “root” attribute sets the path to the target test sets in the XML test result file. Using the example in Figure 10, if I set “root” to "x:/test-results/test-suite/results/test-suite/results/test-suite", it means the test sets can be found at the XPath specified after “x:”.
• The “name” attribute is used to specify test set name which will eventually be used in ALM. For instance, in the screenshot above I set the name attribute as "v:${BUILD TIMESTAMP} Build Verification Test". This means every time the Jenkins job is run, a test set with name like "[time stamp]Build Verification Test" is generated in the Test set Folder. You may also specify name by using the value of the XML element ‘name’ attribute, i.e. configure “name” attribute to “x:@name”. In this case, multiple test sets in ALM may be generated by the plugin.
• The “subtype-id” attribute is another field that you need to define for 'testset'. In the example, I give it the value 'v:hp.QC.test-set.default'. To understand what value should be used, please refer to the plugin's support doc for details.

Now you have understood how to define field mappings, you can follow the same procedure to define mappings for tests and test runs as well.

Run status mapping

After defining "Field mapping", the next step is to explain how to interpret the run status (i.e. test execution status) value. The plugin will use the "result" attribute value in XML file as the run status value. Different test frameworks/tools use different values to express test status, for example, NUnit uses the word "Success" to represent "Passed" and "Failure" to represent "Failed.". For this reason, we need to tell the plugin how to translate each status to its equivalent in ALM.

With the post-build actions configured, you can now save the job, and run it!

Understand integration details from console output

After the job is run, the plugin generates a lot of information in the console output, such as details about login, authentication, entity creation, request details, errors, etc., and it helps investigation when the job failed. For example in Figure 14, at the end of the output, the plugin shows a list of tests or runs which it failed to create or import to ALM.

Figure 7

Figure 8

Figure 9

Review results in ALM/Quality Center

After the Jenkins job finishes successfully, test results will be correctly synchronized to ALM, and new test cases be generated in the specified folder. The screenshot reveals the test case has been generated from the XML data in figure 10.

Figure 10

Test result is successfully marked on test instance.

Figure 11

Summary

Micro Focus Application Automation Tools plugin enables ALM/Quality Center to integrate with any automated testing software run by Jenkins. You just need to set up Jenkins jobs to execute the tests and export test results in XML format, and configure field mapping to translate the test results into entities in ALM/Quality Center. With this, you can use ALM/Quality Center to manage all forms of testing activities, including automated tests with 3^rd-party tools conducted externally.

A key to software quality and test management is consolidating all relevant data to see the big picture from them. The more completely and automatically it is done, the more competitive you are in creating high-quality applications.

Try out the plugin now! Learn more from the plugin documentation. Contact customer support for assistance.