What is the purpose of the Code Analysis section in SilkCentral TestManager 2006 R2 and how do I enable it?

0 Likes

Problem:

What is the purpose of the Code Analysis section in SilkCentral TestManager 2006 R2 and how do I enable it?

Resolution:

QUESTION
----------------------------------
What is the purpose of the Code Analysis section in SilkCentral TestManager 2006 R2 and how do I enable it?

ANSWER
----------------------------------


The Code Analysis section offers code-coverage data for Java AUT"s (Applications Under Test), packages, classes, methods, and statements, enabling you to perform the following:

  • Test-impact analysis: Determines which tests should be run in response to specific code changes.

  • Effort analysis: Determines how many hours of automated testing and manual testing are required to adequately cover specific code changes.

    Each of these code-analysis tasks can be addressed by running pre-configured reports.

    Conventional code coverage

    The goal of conventional code coverage is to deliver information about what code is covered by tests. This approach is typically used to gain code-coverage information for unit tests. Test Manager"s code-analysis functionality goes well beyond this, delivering data for unit, functional, and load tests (both automated and manual) in managed environments. Code coverage measurements are utilized to track test progress and guide test planning. Test Manager"s innovative approach to code coverage draws on the relationship between specific tests and the code they test. This approach enables you to perform impact/dependency analysis of code changes from the testing perspective. By helping you to identify the test runs that are most relevant to a specific code change, it also enables you to better optimize your testing.

    Test Manager code analysis

    Test Manager"s code-coverage functionality is provided in the Code Analysis unit (which is accessible from the Code Analysis button on the workflow bar). The Code Analysis unit features a navigation tree that lists all products that have been created for the selected project. You can drill down into products to select specific versions, and at the deepest level, specific builds.

    Enabling Code Analysis

    Setting up your AUT to collect code coverage information

    This section offers an overview of how Test Manager can be used to find out which classes and methods of your application under test (AUT) have been invoked within the jar-files.

    Prerequisites

    The file sctmcc.dll must be located in the directory from which the AUT will be executed. This file can be downloaded directly from your Test Manager GUI by selecting Help > Tools > Code Analysis Instrumentation Library.

    Test Manager"s code analysis works with Java versions 1.4, 1.5, and 1.6. However, there is a difference in how the DLL can be loaded when starting up the virtual machine (VM).

    When starting up the application under test from the commandline, the following arguments must be passed to the VM in order to load sctmcc.dll:

  • Java 1.4

    -Xbootclasspath/a:. -Xrunsctmcc:port=19129, (see below)

  • Java 1.5 and 1.6

    -agentlib:sctmcc= (see below)


    The following options can be used for all supported Java versions:

  • port=19129
    Port of code coverage service.

  • coveragetype="line"
    Possible values are "line" or "method". "line" must be used for getting code coverage information with SilkCentral Test Manager.

  • coveragepath={"library1.jar";library2.jar"}
    Jar-libraries to monitor for code coverage information.

  • name="ServerName
    Name of the monitored application.

    Examples:

    Java 1.5:

    "C:\Program Files\Java\jre1.5.0_07\bin\java"
    -agentlib:sctmcc=port=19129,coveragetype="line", coveragepath={"C:\dev\deploy\lib\library1.jar";"C:\dev\deploy\lib\library2.jar"}, name="myAUTserver" com.caucho.server.http.HttpServer

    Java 1.4:

    "%JavaDir%\java.exe" -Xbootclasspath/a:.
    -Xrunsctmcc:port=19129, coveragetype="line", coveragepath={"C:\Program Files\Borland\SilkTest\JavaEx\JFC\swingall.jar";
    "C:\Program Files\Borland\SilkTest\JavaEx\JFC\Swing11TestApp.jar"}, name="Test Application"
    -Dsilktest.tafont=arialuni.ttf -cp .;%FontDir% ta

    You must configure Test Manager to gather code coverage data from an application under test. This can be done for any number of execution definitions listed on Test Manager"s Deployment tab, in the Execution unit.

    Enabling code analysis for an execution definition

    Procedure To enable code analysis for an execution definition:

    1. Click the Executions button on the Workflow bar.

    2. Select an execution definition from the navigation tree.

    3. Select the Deployment tab.

    4. Click the Edit button in the Code Analysis Settings section of the tab.

    5. On the Edit Code Analysis Settings dialog, select the Enable code analysis checkbox.

    6. In the Hostnames text field, enter a comma-separated list of hostnames (with port, if default port 19129 is not used) where code analysis information is to be gathered (for example, labmachine1, labmachine2:8000, 198.68.0.1).

    For each execution definition, you need to define the host names of the machine resources where the AUT is running. For example, with a client/server system, you must not only gather code coverage information on the client (which probably runs directly on an execution server) you must also gather data from the server (which likely runs on a different machine). This applies to all multi-tiered applications.

    Note For JUnit code analysis runs, you do not need to specify a hostname.

    7. Click OK to save your settings.

    Once code analysis has been defined for an execution definition, each future run of that execution definition will gather code coverage information from the defined hostnames. While monitoring an execution from Test Manager"s Activities page, you will see that after gathering the sources for test definitions, Test Manager gathers full code coverage information before beginning test runs. The Code Coverage Controller, which is integrated into each Test Manager execution server, controls all defined hosts during execution runs. For each test definition of an execution definition, the controller starts and stops all associated instances, collects XML-based code coverage files for the test definition, and merges the results into a single file. The test definition then saves the merged code coverage file to its execution results.

    Results compilation

    Once an execution definition"s test executions are complete, you can view its results. You will notice that there is a new result file for the execution definition called FullCoverageInfo.xml and an additional CodeCoverageInfo.xml file for each test definition result. Test Manager uses these result files to aggregate and calculate all code analysis data.

    Note Aggregated data is not immediately available and calculations may take time to compile.

    For additional information check the SCTM online help "Code Analysis" section.

Old KB# 24574
Comment List
Anonymous
Related Discussions
Recommended