Introduction

Test Framework gives a set of predefined test scenarios to help you certify a successful upgrade and allows you to adapt and enhance the same based on the feature set used in your Access Manager deployment.

 Using this solution, you can perform the following:

• Taking a record of how Access Manager is working before an upgrade through a set of pre-defined test cases available in Test Framework.
• Validating that the Access Manager setup is working as expected after an upgrade and no functionalities are broken after the upgrade. Test Framework provides a set of pre-defined test cases for this purpose.

In addition to the default features and test cases, you can customize it, if required, as follows:

• Integrate an automated upgrade procedure written using Ansible with Test Framework to perform the upgrade and also validate the scenario from a single place.
• Add more functional test cases based on your Access Manager deployment.

A high-level flow of actions is described in the below figure:

This document provides you guidelines to set up and use the Test Framework.

This solution has been tested and validated for upgrading Access Manager from version 4.5 to 4.5 Service Pack 2 on a Linux platform.

Prerequisites for Setting up Test Framework

Ensure that you have the following ready:

1. 1. Python 2.7 with the latest pip and latest setup tools are available on the controller node.
   2. Test Framework needs to be set up on a controller node from which test cases can be executed on all the Access Manager nodes in parallel.
   3. To achieve parallel test execution on all the Access Manager Nodes, it’s recommended to enable a password-less connection among the controller node and all Access Manager nodes. Follow the below steps to enable a password-less connection between the controller node and Access Manager Nodes:
      1. SSH port (22) should be open on Access Manager nodes.
      2. Generate an SSH authentication key pair on the controller node using the command: ssh-keygen –t rsa
      3. Copy the public key to Access Manager Nodes by using the command: ssh-copy-id <user>@<Access_Manager_Node_IP>
      4. Enter the password of Access Manager Node when it prompts for a password. After entering the password, an SSH key is copied to the Access Manager Node, and a success message is displayed.
      5. Copy the SSH key to all Access Manager nodes on which parallel test cases need to be executed. 
      6. The following is a sample screenshot of copying the SSH key to the remote node using the ssh-copy-id command:
      7. Once the public key is copied to remote Access Manager Nodes, verify if SSH can be done to remote Access Manager Nodes without providing the password. To verify this, run the given command on the controller node for each of the Access Manager Nodes: ssh <user>@<Access_Manager_Node_IP>
      8. Sample screenshot of password-less connectivity:

NOTES:

1. • Every time a snapshot is reverted to the controller node, a new SSH key needs to be generated and should be copied to all other Access Manager Nodes.
   • Every time a snapshot is reverted on Access Manager Node, the SSH key should be copied from the controller node to the Access manager Node.

Installing Plugins Needed for Test Framework

The following plugins are needed for Test Framework:

Instead of manually installing these plugins, you can use the setup.py script to easily install these plugins. The script will install the latest version of plugins.

Perform the following steps to run setup.py:

1. Run the 'python setup.py' command on the controller node.
2. Verify if the required plugins - pytest, pytest-html, and pytest-sugar are installed successfully by running the 'pytest –version' command.
3. Sample screenshot of output –

Updating Access Manager Setup Details in the Property File

The input.py file works as a property file for Test Framework, which can be used by admins/users to update Access Manager setup details. It is important to update the input.py file with Administration Console, Identity Server, and Access Gateway IPs as given below. Other parameters mentioned in the input.py file are specific to the test cases.

• AC_host_list = [‘<AdminConsole_IP#1>’,’< AdminConsole_IP#2>’]
• IDP_host_list = [‘<IdentityServer_IP#1>’,’<IdentityServer_IP#2>’,’…….’,’<IdentityServer_IP#N>’]
• AG_host_list = [‘<AccessGateway_IP#1>’,’<AccessGateway_IP #2>’,’……’,’<AccessGateway_IP#N’]
• Sample screenshot of the updated parameters in input.py file:

Running Test Cases in Test Framework

1. Run main.py using the 'python main.py'  command.
2. Select the appropriate options to run test cases. Each option with description is given below:

Feature Test Case Execution

You can run each of the feature test cases individually by selecting respective feature option OR, you can run test cases on multiple features.

1. Enter feature keywords as highlighted below, to select multiple features at a time.

2. Separate execution reports are generated for each of the features in folder ‘reports’.
3. Example - OAuth_test_report.html for OAuth feature and nalytics_Server_test_report.html for Analytics server feature.

NOTE:

• Feature functional test cases will get executed if the feature is enabled in Access Manager deployment.
• To execute these test cases, there are few pre-requisite parameters that should be updated in the input.py file as per your Access Manager deployment.
• Example: For the OAuth feature, section – ‘OAuth pre-requisite configuration’ should be updated in the input.py file. For the Analytics server, section – ‘Analytics server configurations’ should be updated in the input.py file.

Execution Summary and Report

• During the test case execution, the name of each test case getting executed, their execution status (Pass/Fail), and their execution progress bar are shown. Sample screenshot:
• After the execution is completed, an HTML report is generated and its location is displayed at the bottom of the execution summary. Also, execution time and a total count of test cases passed and failed are displayed. Sample screenshot:
• All the HTML reports are generated in folder ‘reports’. A sample of HTML report is given here:
• In the above HTML report, you can click ‘show details’ available in the ‘Result’ column to view each test case execution details. Here is a sample screenshot:

Adding Test Cases to Test Framework

Here are different ways in which you can add test cases to the framework.

Adding automated scripts as a Test Case in Test Framework

You can plug the recurrently used automated scripts/test cases into the Test Framework and run as a test case from a single platform with Test Framework.

Follow the steps to add these customized automated scripts:

Example:  If the automated script file is named automated_test_case.sh

1. Select the sample file test_custom_automated_test_cases.py from the folder named test.
2. Under function test_script(), update below parameters:

• Update host_list against which this script should be run.
• To run automated test case script on all Access Manager nodes, update ‘host_list’ using host_list = get_all_host_list()
• To run automated test case script only on Administration Console, update ‘host_list’ using host_list = AC_host_list
• To run automated test case script only on Identity Server, update ‘host_list’ using host_list = IDP_host_list
• To run automated test case script only on Access Gateway, update ‘host_list’ using host_list = AG_host_list

3. Update the cmd parameter with the automated script file name. This code snippet captures details of both successful output of script and errors (if any) thrown during script execution
4. Example of updated parameters:

Adding New Test Cases to Existing Features

You can add new test cases to existing features by adding them to the respective test module file. Follow the pytest coding guidelines while adding new test cases.

Example: A new test case for the OAuth feature can be added in the test/test_OAuth.py file. Similarly, for the Analytics Server feature, the new test case can be added in the test_analytics_server.py file.

 Adding a New Functional Feature Test Module

To add a new feature test module to the Test Framework, follow below steps:

1. Create a new feature-test module file in the folder named, test. Example: Name it test_SAML.py, if you want to add SAML related test cases.
2. Write functional test cases in the test_SAML.py file. You must name each test case starting with keyword ‘test_’ so that test case can be auto-discovered during execution.
3. Run this feature module file using option-4 (Feature functional test cases) of the test framework.

Integrating Automated Installation/Upgrade solutions with Test Framework

If you are an admin/user who has automated Access Manager installation or upgrades procedure, you can even integrate that solution with Test Framework, and use Test Framework as a single tool to install/upgrade Access Manager and verify successful installation/upgrade using the test cases available in Test Framework.

An Ansible script written to automate the Access Manager upgrade procedure is provided as an example to explain how the automated upgrade solution can be integrated with Test Framework.

NOTE:

• This Ansible script is completely independent and outside the Test Framework code.
• Test Framework is not dependent on Ansible.
• Refer to /cyberres/accmgmt/accessmanager/w/access_manager_tips/15178/how-to-automate-the-upgrade-process-of-netiq-access-manager-using-ansible to learn how to automate the upgrade process of Access Manager.
• Integrating automated upgrade procedure to Test Framework will help admin/user to fasten the whole process of validating an Access Manager setup before the upgrade, then upgrading it and later validating the setup for basic functionalities of Access Manager. And all these can be achieved from one single platform i.e. Test Framework.

Example: Integrating Automated Access Manager Upgrade Procedure with Test Framework

1. Consider that Ansible scripts written to automate upgrade procedure are present in folder /etc/ansible on the controller node. Ensure that the controller node for the Ansible script and test framework is the
2. Consider if Ansible scripts are run independently outside Test Framework, then they are run using the 'ansible-playbook upgradeToNAM45.yml' command, where 'upgradeToNAM45.yml' upgrades Access Manager to version 4.5.0.
3. To integrate 'upgradeToNAM45.yml' with Test Framework, update the parameter ‘ansibleScriptFilePath’ in the input.py file as given below:

• • ansibleScriptFilePath = ‘/etc/ansible/upgradeToNAM45.yml’  ,where ‘ansibleScriptFilePath’ is the absolute file path.

Running Access Manager Upgrade Scripts Within Test Framework

1. Run main.py and select option 2 – Access Manager installation/Upgrade. This displays further options as given below:
   1. 1. Fresh installation
      2. Upgrade
      3. Exit
   

2. Choose the appropriate option. For this example, choose option 2 – Upgrade, to run the Ansible script (upgradeToNAM45.yml) from within Test Framework and perform Access Manager upgrade.

3. Sample output of execution is as given below:

4. After execution is complete, the following sample output is displayed:

Where PLAY RECAP provides a summary of the Ansible task run status against each of the node’s IPs. These nodes IPs can be IPs of Administration Console, Identity Server, or Access Gateway for which the upgrade script was run.