Contents

Introduction

The 'parallel' Statement
The 'multitestcase' Statement
The SetUpMachine() Statement
The 'spawn' Statement and the 'setMachine' Statement
The 'rendezvous' Statement
Alternative Launch Syntax
Thread Control
The 'share' and 'access' statements
The 'critical' Statement
The 'semaphore' Datatype

Introduction

SilkTest has long had support for multi-threaded testcases but the caveat is that it only works for remote parallel testing. In other words, you cannot run multiple threads on a local testcase, nor can you run multiple threads on a single remote machine. This only works for two or more machines being run remotely.

There are two ways to set up a multi-threaded test:

1. Using the 'parallel' statement
2. Using the 'multitestcase' statement

You should also note that multithreaded remote testing does not imply that all machines should be running the same application under test. Each machine can be running an entirely different test and the results will be returned to the launch (host) PC. There are some differences between the two calling methods which you must be aware of. The following five control methods are unique to the multitestcase method:

SetUpMachine()
SetMultiAppstates()
SetMachine()
spawn
rendezvous

In addition, there are a set of control methods that are common to both multitestcase and parallel calls:

access
acquire
critical
release
semaphore
share

We will discuss each of these in turn, but first we will look at the two methods of starting a multithreaded testcase.

The 'parallel' Statement

If you choose to use the parallel statement you must be aware of two additional caveats:

The test MUST be launched from a main() function and it does not invoke the Recovery System.

The second is not quite so much of a problem as it first appears because if you are calling testcases inside your main() function you can create your own Recovery System using the TestcaseEnter() or TestcaseExit() methods to ensure that the application under test is in a state fit to start testing. See the Help files topic 'main function, using in a script' for how to call a testcase from a main() function.

The advantage of the 'parallel' statement over using 'multitestcase' is that it implicitly waits for all threads to complete at the end of the multithreading block before continuing on with the test. Multitestcase requires that you explicitly tell SilkTest where the end of the multithreading block is (see below). The following illustrates the outline of a parallel testcase using a remote server to supply information to the remote clients. The requirement to only multithread on remote machines does not stop you from calling the local machine as part of the test. This will work correctly as long as the calls to the local (host) PC are outside the 'parallel' code blocks.
 
[-] main ()
  [ ] HMACHINE client1_handle
  [ ] HMACHINE client2_handle
  [ ] HMACHINE server_handle
  [ ]
  [ ] server_handle = Connect ("server")
  [ ] client1_handle = Connect ("client1")
  [ ] client2_handle = Connect ("client2")
  [ ]
  [ ] SetupServerState ("server")
  [ ]
  [ ] // Start the multithreaded calls to the remote PCs
  [-] parallel
    [ ] RunMyTest ("client1")
    [ ] RunMyOtherTest ("client2")
    [ ] // The testcase will halt here until both client1 and client2 are done
  [ ]
  [ ] // The testcase is done so tidy-up
  [ ] TidyUpServer ("server")
  [ ]
  [ ] //Clean up behind yourself
  [ ] Disconnect (server_handle)
  [ ] Disconnect (client1_handle)
  [ ] Disconnect (client2_handle)
  [ ]
  [ ] //Testing ends here
  [ ]
  [ ] //----------------------------
  [ ] //Actual test code follows
  [ ]
  [ ] //Setup the server
  [-] SetupServerState (STRING sMachineName)
    [ ] SetMachine(sMachineName)
    [ ]
    [ ]
    [ ]
  [ ]
  [ ] //Call the first client
  [-] RunMyTest (STRING sMachineName)
    [ ] SetMachine(sMachineName)
    [ ]
    [ ]
  [ ] //Call the second client
  [-] RunMyOtherTest (STRING sMachineName)
    [ ] SetMachine(sMachineName)
    [ ]
    [ ]
  [ ] // Return the results to the server and clean up
  [-] TidyUpServer (STRING sMachineName)
    [ ] SetMachine(sMachineName)
    [ ]
    [ ]


The other big disadvantage of using the 'parallel' statement is that you cannot call an appstate; again, this can be alleviated by creating your own appstate in the TestcaseEnter() method as long as you are calling testcases inside the main() method. You cannot do this in the example above as it runs directly from the main() method without any internal testcases.

The 'multitestcase' Statement

The multitestcase method has the advantage that you have access to the Recovery System and that you can call an appstate during the setting up procedure. The testcase outline is not the same as the parallel statement above although it does have similarities. Rather than provide the entire outline of a mulitestcase, we will build it up section by section. Each of the following sections will outline the methods that are unique to multitestcase.
 
The SetUpMachine() Statement

This is the only unique statement that has more than one parameter:

[ ] SetUpMachine(sMachine,wMainWin(optional),sAppState(optional))

Before you can run any code on the remote PCs you must enable the remote PCs using the SetUpMachine() method. The method connects to the remote machine and prepares it through the specified appsatate before executing any test code. SetUpMachine() implicitly issues the Connect() statement which you need to explicitly state if using the parallel method. Although wMainWin and sAppState are optional, if you do not declare a wMainWin then SilkTest cannot invoke the Recovery System as the Recovery System needs to know the application's main window (see the Recovery System in any frame file). By careful choice of variables you can effectively black-box the calls to SetUpMachine() so that a simple change to a set of global variables will call a whole new set of remote machines and/or different appstates.
 
[ ] //List the PCs that will be running the tests
[ ] string sRemotePC1="Pippin"
[ ] string sRemotePC2="Frodo"
[ ] string sRemotePC3="Bilbo"
[ ]
[ ] //List the appstates we want to start with
[ ] string sAppState1="TheParty"
[ ] string sAppState2="GoToMordor"
[ ] string sAppState3="DestroyTheRing"
[ ]
[ ] // List the main windows for the recovery system
[ ] string sMainWin1="GoldenPerch"
[ ] string sMainWin2="PrancingPony"
[ ] string sMainWin3="WellingHall"
[ ]
[-] multitestcase Adventure (string sRemotePC1,string sRemotePC2,string sRemotePC3)
  [ ] SetUpMachine(sRemotePC1,sMainWin2,sAppState3)
  [ ] SetUpMachine(sRemotePC2,sMainWin1,sAppState2)
  [ ] SetUpMachine(sRemotePC3,sMainWin3,sAppState1)
   [ ] SetMultiAppStates()


You must terminate the SetUpMachine() calls with a call to SetMultiAppStates() to ensure that the application(s) are driven to the correct state before starting the test. Once the SetUpMachine functions are completed, you can go on to start the threads.

The 'spawn' Statement and the 'setMachine' Statement

The spawn statement notifies SilkTest that this is the start of the multithreaded code. Once a spawn method has been called we need to make a call to the individual PC via the setMachine() statement to notify it that test code follows. If you have done any serial remote testing then the setMachine() statement may already be familiar to you. Each machine called in the SetUpMachine() block must have a spawn statement and each spawn statement must have a setMachine() call.

[-] multitestcase Adventure (string sRemotePC1,string sRemotePC2,string sRemotePC3)
  [ ] SetUpMachine(sRemotePC1,sMainWin2,sAppState3)
  [ ] SetUpMachine(sRemotePC2,sMainWin1,sAppState2)
  [ ] SetUpMachine(sRemotePC3,sMainWin3,sAppState1)
   [ ] SetMultiAppStates()
  [ ]
  [-] spawn
    [ ] setMachine(sRemotePC1)
    [ ]
  [-] spawn
    [ ] setMachine(sRemotePC2)
    [ ]
  [-] spawn
    [ ] setMachine(sRemotePC3)
    [ ]


The obvious difference here is that we are running the test code inside the spawn statement. However, there is nothing to stop you from using the same test code structure as we have shown in the parallel statement:
 
[-] multitestcase Adventure (string sRemotePC1,string sRemotePC2,string sRemotePC3)
  [ ] SetUpMachine(sRemotePC1,sMainWin2,sAppState3)
  [ ] SetUpMachine(sRemotePC2,sMainWin1,sAppState2)
  [ ] SetUpMachine(sRemotePC3,sMainWin3,sAppState1)
   [ ] SetMultiAppStates()
  [ ]
  [-] spawn
    [ ] RunMyTest1 (sRemotePC1)
  [-] spawn
    [ ] RunMyTest2 (sRemotePC2)
  [-] spawn
    [ ] RunMyTest3 (sRemotePC3)
//------//   [ ]
  [-] RunMyTest1 (STRING sMachineName)
    [ ] SetMachine(sMachineName)
    [ ]
    [ ]
  [-] RunMyTest2 (STRING sMachineName)
    [ ] SetMachine(sMachineName)
    [ ]
    [ ]
  [-] RunMyTest3 (STRING sMachineName)
    [ ] SetMachine(sMachineName)
    [ ]


The 'rendezvous' Statement

The rendezvous statement notifies SilkTest that this is the end of the multithreading block. As threads will run at different speeds on different machines depending on hardware and load, the rendezvous statement stalls each thread as it returns until the last thread has completed its testing. At this point SilkTest releases the testcase to continue running.

[-] multitestcase Adventure (string sRemotePC1,string sRemotePC2,string sRemotePC3)
  [ ] SetUpMachine(sRemotePC1,sMainWin2,sAppState3)
  [ ] SetUpMachine(sRemotePC2,sMainWin1,sAppState2)
  [ ] SetUpMachine(sRemotePC3,sMainWin3,sAppState1)
   [ ] SetMultiAppStates()
  [ ]
  [-] spawn
    [ ] setMachine(sRemotePC1)
    [ ]
  [-] spawn
    [ ] setMachine(sRemotePC2)
    [ ]
  [-] spawn
    [ ] setMachine(sRemotePC3)
    [ ]
  [ ]
  [ ] rendezvous
  [ ]
  [ ]


If the testcase terminates at the end of the multithreading block then there is no real need to use a rendezvous statement. However, it is recommended to do so if only for the sake of neatness in the script.


Alternative Launch Syntax

There is an alternative syntax that can be used when starting a multitestcase but it is better suited to running identical remote tests. It draws the machine name from a list of string and by way of a loop, launches each remote machine until it reaches the end of the list.

[-] multitestcase runMulti (list of string lsmachines)
  [ ] string sMachine
  [ ]
  [-] for each sMachine in lsMachines
    [-] spawn
      [ ] SetUpMachine(sMachine,"wAppMainWin","sAppstate")
  [ ] rendezvous
  [ ]
  [ ] SetupMultiAppstates()
  [ ]
  [-] for each sMachine in lsMachines
    [-] spawn
      [ ] [sMachine]PerformSomeActivity()
  [ ] rendezvous
  [ ]


Note that the application's main window name and the starting appstate are hard-coded into the SetUpMachine() call. It would be possible to also draw these from a list of string so that different appstates and mainwins can be loaded each time, but at this point it would be a lot easier to use the starting layout defined above.


Thread Control

The following thread control methods apply to both the parallel and multitestcase methods of starting a multithreaded testcase.

The 'share' and 'access' Statements

All threads have equal access to both local and global variables. Uncontrolled access of this type is a bad thing because you can get multiple threads all trying to update a given variable at the same time, leaving it in an undefined state that can wreck the rest of your testcase. Unfortunately we don't have a precise method of controlling access to local variables except by careful coding and paying attention to the scope of the variable. For global variables (those that can be seen by multiple testcases as well as multiple threads) there are the 'share' and 'access' control methods. These two work hand-in-hand to prevent unwanted changes to a variable.

The syntax for the share method is as follows:
 
share =

In this case can be either public or private. If is omitted then it defaults to public. can be of a standard SilkTest type or it can be a user-defined type. The variable must be initialised before you try to use it as if it is not, SilkTest will raise a 'not defined' error message when you try to access it. Once you have declared a variable as shared you will only be able to change or read it by use of the access statement. The access statement controls access to a shared variable by blocking all other threads that require to use the variable until the current controlling thread has finished with it. Once the access script block is exited, SilkTest releases any blocked threads and the variable is available for access again by the next thread that requires it.

[ ] private share integer iTestNum=0
[ ]
[ ] // Function to update the testcase count
[-] void IncrementTestNum()
  [ ] // Block unwanted threading
  [-] access iTestNum
    [ ] // Increment the test count
    [ ] iTestNum++
  [ ]
  [ ] // Exit access control and release the threads
  [ ] return


The 'critical' Statement

The 'critical' statement acts in a similar fashion to the 'access' method but with one important difference; as soon as a thread enters a critical code block all other threads are immediately halted wherever they are in their code execution. The threads are released only when the active thread exists the critical block. Only one thread can have critical status at any one time.
 
[ ] // Function to update the testcase count
[-] void IncrementTestNum()
  [ ] // Halt all threads while this block executes
  [-] critical iTestNum
    [ ] // Increment the test count
    [ ] iTestNum++
  [ ]
  [ ] // Exit critical control and release the halted threads
  [ ] return


The 'semaphore' Datatype

The semaphore is the last method associated with multithreaded testcases. The semaphore datatype has two children - ' acquire' and 'release'. Semaphores are used to control access to a resource or to mutually exclude competing threads. Like the 'shared' method, the number of semaphores needs to be initialized before trying to use them or else an exception will be raised on first access. You cannot directly check for the number of semaphore resources available or in use; to do this you must typecast the number of semaphores and calculate as necessary.
 
[ ] semaphore semControl=10  // Allocate ten semaphore flags
[-] if (semControl==[semaphore]2) // If there are only two semaphores left
    [ ] print("semControl is down to {semControl} unused flags")


'Acquire' is used to allocate a semaphore to a requesting thread. If SilkTest has more threads than semaphores and threads are requesting more semaphores than are available, SilkTest will block all threads until a semaphore is released. When semaphores are released, SilkTest allocates the released semaphores in the order that the threads invoked the 'acquire' method. SilkTest then unblocks the thread that it has just given the newly-released semaphore to. If no threads are waiting for a semaphore when a 'release' statement is executed then the semaphore is put back in the available pile.

'Release' is the keyword to actually release an acquired semaphore..

There is a potentially serious problem with the 'acquire' child method. If the script experiences an exception (as opposed to an expected and trapped error) between the 'acquire' and 'release' statements the script will deadlock with no chance of recovery. To prevent this from happening wrap any code that could cause a potential exception in the 'do' part of a  do-except block and a 'release' statement in the 'except' part. Probably the best use of semaphores is to control access to a database where a limited number of users are allowed online at any one time. In the following example we will assume that the user limit for accessing the database is three users. 

 [ ] semaphore semDBAcess=3 // Declare three semaphores as available
 [-] // Do code here to spawn thread one
   [ ]
   [ ]
   [ ] acquire(semDBAcess)  // Thread one acquires a flag
   [ ] print ("Semaphore requested - {semDBAccess} flags left")
   [ ]
   [ ] print("Beginning database access")
   [-] do
     [ ]
     [ ]
   [-] except
     [ ] // The database has thrown an exception
     [ ] release (semDBAccess)
     [ ] LogWarning ("Unexpected database exception")
     [ ] print ("Semaphore released - {semDBAccess} flags available")
   [ ]
   [ ]
   [ ] release (semDBAccess) // The semaphore is finished with
   [ ] print ("Semaphore released - {semDBAccess} flags available")