State-Driven Testing (SDT): Framework Definition Design Patterns

State-Driven Testing (SDT): Framework Definition Design Patterns

Simple Dialogs

Test Objects/Actions Design

The screenshot below shows a simple dialog that contains two text fields, one combo box and two buttons. Good candidates for the corresponding test object name include Login, LoginDlg, and Login_Dialog. Note that no spaces are allowed. Use ‘_’ or uppercase letters to separate words. It is good practice to use an uppercase letter at the beginning of each object name.

The available actions on this dialog are setting the user name and password, selecting a user type, and clicking the two buttons. For complete coverage, the ‘x’ command that closes the dialog should also be expressed in a test method.

Here are the resulting test methods: setUserName, setPassword, selectUserType, selectOk, selectCancel, and closeDialog. An alternative is to combine the two text-field test methods into a single method (for example, setUserNameAndPassword). It is good practice to use a lowercase letter at the beginning of each test method name.

For logging and verification purposes it is recommended that you define test methods that return the values of the two text fields and/or the combo box (for example, getUserName, getPassword or getUserType).

The framework definition should resemble the following:

State Transition Pattern: Modal Dialog

A common action in applications is clicking a UI control such as a button or menu item which results in the opening of a modal dialog. This brings the application into a state where the user can only execute actions on the modal dialog. No other actions can be performed until the user closes the dialog.  

In the SDT Framework, this is realized in two steps:

  1. Open Dialog: Add a state transition method NewAppState to the action, which triggers the opening of the modal dialog. As input for NewAppState use the test object that represents the modal dialog
  2. Close Dialog: For each action on the modal dialog that closes the dialog, add a state transition method RestoreAppState to the SDT test object that represents the modal dialog

Note: Executing RestoreAppState will result in the previous application state (list of test objects before the modal dialog opened) being available again

In the example below, the Edit A dialog (test object AEditDialog) is opened when the Edit button on the action bar (test object AAction) of the main window is clicked.

The framework definition should resemble the following:

Tabbed Dialogs and Web Pages

Test Objects/Actions Design

This section describes how a dialog or Web page containing multiple tabs can be split into multiple test objects. Best practice is to create one test object that acts as the frame object for switching between the different tabs and containing button actions that are common across all tabs.

In the first example dialog below, such a breakdown results into seven test objects. The first test object is AgentOptionsDlg. It contains six actions for selecting the six tabs and two actions for the OK and Cancel buttons, which are valid across all tabs. Good candidates for naming the switch-tab actions include switchToTabTiming and selectTabTiming.

The framework definition should resemble the following:

The breakdown for a tab control that is embedded in a Web page works the same as a tabbed dialog. Applying the same approach to the tabbed Web page below results in nine test objects. The only difference from the above example is that there are no common buttons across all tabs. So the frame test object only contains the actions that relate to navigating between the tabs.

State Transition Pattern: Tab Selection

Modal Tabbed Dialogs/Pages

The opening of a modal dialog that contains multiple tabs is executed by an action on the AUT such as the clicking of a button or selection of a menu item. With modal dialogs, the application state stack receives an additional layer with either one test object in it and, in cases where the selected tab is always the same by default, also the test object of the selected tab. This means that only the test methods of the frame test object, and eventually the test methods of the default tab test object, are available. To switch to a different tab, the frame test object provides one test method for each tab. The state transition test method required to adjust the application state is SetAppState. For an input parameter, SetAppState takes the frame test object and the tab test object, which is the new selected test object.

For the Agent Options dialog example above, the framework definition should resemble the following:

Modeless Tabbed Dialogs/Pages

The difference with modeless tabbed dialogs/pages is that the previous state transition does not create a new layer (application state) on the application state stack; it reuses the existing layer and adds the new modeless tabbed dialog/page to the application state. Any time a different tab is selected, the state transition must change the current application state so that the following requirements are met:

  • Existing test objects (before the dialog/page was opened) and the frame test object remain in the same application state
  • Newly selected tabs are added
  • Previously selected tabs are removed

The first two requirements can be fulfilled using the state transition method AddAppState with the newly selected tab as input. This guarantees that the existing application state will be extended. For the third requirement, the state transition method RemoveAppState is required. This takes all other tab test objects as input parameters. This approach is necessary because it is not possible to know which tab was selected previously.

Note: When a new tab is added to this dialog/page, state transitions need to be adapted accordingly.

For the tabbed Web page example above, the framework definition should resemble the following:

Menu

Test Objects/Actions Design

The guideline of limiting test objects to a maximum of 20 test methods should be applied when structuring the test objects that represent the menus of the AUT. When a menu does not contain more than 20 menu actions then one test object is enough. In this document we will describe the breakdown of a complex menu structure as this is typical when testing real-world applications.

The framework design starts with the test object that represents the menu bar. This test object contains the actions that are used to access the sub menus and their corresponding commands. Each sub menu is represented by one test object. If the number of actions for a test object exceeds 20, then consider breaking the sub menu down into smaller test objects. Such careful structuring during framework definition saves time during test case assembly and makes test cases more readable.

If we apply these practices to the menu structure below, then we get one test object for the menu bar and nine test objects for the menus on the menu bar. As some of the menus contain more than 20 actions, they have been broken into smaller test objects.  The example below shows how this approach applied to the sub menu Test Logic of menu Insert.

The framework definition should resemble the following:

State Transition Pattern: Menu Action

The state transitions needed to use menus and menu items in the test case assembly consist of the following:

  • Accessing the menu bar by either adding it immediately after starting the AUT or by adding it with the state transition test method AddAppState when executing a test method/action of the AUT main window.
  • Traversing one or multiple sub menus until the menu item is reached using the state transition method AddAppState to access a sub menu and the state transition method RemoveAppState to remove a sub menu from the current application state.

For the menu above, the framework definition should resemble the following:

Toolbar

Test Objects/Actions Design

When an AUT contains toolbars, then each toolbar gets its own test object and a test method is created for each item on the toolbar. Usually a tooltip appears when the cursor passes over a toolbar item. Best practice is to use the tooltip as the test method name (for example, clickRecord for the example tooltip below).

In the following example there are five toolbars. Each toolbar contains multiple toolbar items.

The framework definition should resemble the following:

State Transition Pattern: Toolbar

Tree

Test Objects/Actions Design

Many applications include tree-menu controls that consist of nested nodes for the display of hierarchical data. Selecting specific nodes in the tree menu results in varying display within the right-hand pane.

In an SDT framework this type of control is depicted as a single test object containing test methods/actions for the selection of varying node types (based on information such as unique ID or name). The crucial thing is to create one test method for each tree node type.

The example below shows the tree-menu control of a test management application. The control provides hierarchical information about all test cases for a project. The root is the project node. This is followed by the test container level. The leaf nodes specify the different test types that are supported by the application (for example, manual tests and JUnit tests). A test object and a separate test method/action will be set up for the project node, the test container node, and each test type.

The framework definition should resemble the following:

State Transition Pattern: Trees with different node types

To express the behavior of the tree-menu control described above, the state transition must change the application state in the following ways:

  • The newly selected tree node type must be added
  • The previously selected tree node type must be removed

The state transition method AddAppState with the newly selected tree node type as input ensures that the new test object is added. This guarantees that the existing application state is extended. For the removal, the state transition method RemoveAppState is required. This method takes all other tree node type test objects as input parameters. This approach is required because you can’t know which tree node type was selected previously.

Note: Whenever a new tree node type is added to the tree-menu control, the state transitions must be adapted accordingly.

For the tree-menu control example above, the framework definition should resemble the following:

Negative Testing

To achieve satisfactory test coverage you must have tests that validate varying error conditions. In SDT this is solved by defining multiple test methods for the same action that lead to varying AUT states. The additional effort required for this framework definition is justified by the improved readability of the test cases.

Example: Login functionality – Testing with both valid and invalid login credentials

For this example the SDT framework should resemble the following:

Verifications

In principle, there are three different types of SDT verification:

Implicit

Verification is done by the test method/action itself, which checks if the action was successful.

Examples:

  • If a test method/action clicks a link, the method/action can additionally check if the correct Web page/dialog is displayed following the action.
  • If a test method/action enters a value into a text field, the method/action can additionally check if the UI control contains the proper value following the action.

Explicit & Typed

Verification is achieved using a test method/action that is separate from the test object.

Example:

The test method below takes nodeId as input, verifies if a node with this ID exists in the Test Planning tree, and throws an exception if the node does not exist.

Explicit & Generic

The current SDT prototype provides the option of specifying one or more test objects that are always available in the application state. This means that the test methods/actions can be executed at any time in the test case. Best practice is to specify a Utils test object that contains generic verification functions.

Examples:

The examples below compare two integer, Boolean, and string values and throw an exception if a difference is detected between the values. The actual value is returned by an action from a previous step and the expected value is a hard-coded value or a value read from a data-driven file.

 

DISCLAIMER:

Some content on Community Tips & Information pages is not officially supported by Micro Focus. Please refer to our Terms of Use for more detail.
Comments

Thank you for this information.  Can you please clarify which product this pertains to?  I am currently using SilkTest Classic (4Test), but I might be willing to switch to Silk4Net.  

I think I get the overall structure, but I am not clear on how to represent and verify state transitions in code.  Is there a working example I could download?

Version history
Revision #:
1 of 1
Last update:
‎2012-06-21 17:53
Updated by:
 
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.