Considerations when executing a SilkTest test case as part of a GUI-Level Loadtest
Running a functional test in SilkTest is a lot different than running a performance test in SilkPerformer. As such extra consideration should be given when creating a functional test in SilkTest which will later be used in SilkPerformer to run as a performance test (GUI-Level test). For example consider that during functional testing, SilkTest will drive a single instance of the AUT (Application under Test) and this test will be generating the stress of a single user on the server. While during a SilkPerformer GUI-Level test, each virtual user will be running an instance of SilkTest which in turn will be driving an instance of the client application in order to simulate multiple client server interactions (i.e. a performance test). This may cause the client and/or server response to be slower and thus objects (e.g. windows, dialog boxes, text fields etc) may not be available to interact with as quickly as are during a functional test.
With this in mind it is recommended that you implement certain modifications to make your SilkTest test case more robust and better prepared for a Performance Test. As such below is a list of suggestions which you can implement to help in such instances.
1. In the SilkTest.t file introduce or increase the use of Sleep functions between API calls. This will replicate the real life behaviour of think times i.e. where a customer may pause before deciding to press a particular button. It will slow down the test execution to reflect real world usage which is a goal of performance testing.
2. In the SilkTest Agent Options (Options | Agent | Timing) increase the Window timeout to take into consideration slower server responses under load conditions, so that the windows of the AUT have a better chance of being in the correct state within the specified timeout.
3. Increase the mouse delay and keyboard delay to slow down the mouse and keyboard movements so that objects are more likely to be ready for interaction with the mouse or keyboard events under load conditions.
4. Increase the Application Ready Timeout, as during a performance test the AUT may be tested dozens of times by each individual virtual user. Therefore it makes sense to ensure adequate time is specified to ensure that the AUT is ready at the start of each transaction executed by a virtual user during a SilkPerformer GUI-Level test.
5. Ensure that the test case in SilkTest is a robust test case; this means adding custom code and logic such as the Exists() method in the Classic Agent or WaitForObject() method in the Open Agent to verify that objects are ready for interaction before subsequent calls are executed during the performance test.
6. In SilkPerformer go to “Settings | Active Profile | Replay | GUI-Level Testing | Execution Settings” and that the check box “Log SilkTest Errors” is enabled and that “Generate Truelog file for” is set to “Custom”. As this will result in the generation of a TrueLog file in SilkPerformer which will help you identify at exactly which point in the SilkTest test case the execution failed.
7. Likewise use Print statements in your SilkTest test case for debugging in order to identify where a script is exactly failing, as the .res file can be opened from SilkPerformer after a load test has executed.
Old KB# 31703