need help building execution plan report for automation runs that include tester and root configuration suite

[JamesA99]

Hi, I am trying to report on execution plans that not only include the configuration suite name and execution plan but also the tester that is assigned to it (tester can be set in the deployment tab).

I tried using data mart but cannot see any fields that would contain the tester so not sure if that is best approach for this report?

this is what I have used so far..

SELECT ExecutionPlanName, TestName, BuildName, StatusID as "Current Status ID", PassedCount, FailedCount, NotExecutedCount, ExecutionStartTime, TestDurationInMilliseconds, ExecutionParentFolderID
FROM RV_LatestTestStatus lts
INNER JOIN TM_ExecTreePaths ON lts.ExecutionPlanID = TM_ExecTreePaths.NodeID_pk_fk
WHERE TM_ExecTreePaths.ParentNodeID_pk_fk = ${executionFolderID|2179|Execution Folder ID} and ProjectID = 47
ORDER BY TestName

thanks!

James

[Matthew McGibbon]

Hi James,

Are you using this for manual testing? As currently testing cycles would provide this information, as you can assign a tester to a set of tests which then create an execution plan for those tests and tester which you can then easily report on.

You can apply configurations to testing cycles as well.

Let us know.

Thanks,
Matthew

[JamesA99]

thanks Matthew, this is specifically for automation testing as we are setting up numerous automation execution plans and would like to capture who is assigned to run them ...I cannot seem to find this info in the tables?

thanks for the help!
James

[Matthew McGibbon]

Hi James,

For automation you can locate who started the run from the the [TM_ExecDefinitionRuns] table. In here you have a column which is starter.

This is what you would see when looking at the activites tab, so you can see who has run the execution plan. ExecDefID_fk in this table would be the Execution Plan ID.

The tester association on the deployment tab would only only be for Manual Tests.

As a sample query this could look something like the following:

SELECT edr.ExecDefRunID_pk_fk as RunID, etn.NodeName as 'ExecutionPlan', EDR.BuildName, edr.PassedCount, edr.FailedCount, edr.NotExecutedCount, edr.Starter, edr.ExecutionTimestamp, edr.Duration
FROM RTM_V_ExecTreeNodes etn
JOIN TM_ExecDefinitionRuns edr ON edr.ExecDefID_fk = etn.NodeID_pk
INNER JOIN (SELECT edr2.ExecDefID_fk, MAX(edr2.ExecutionTimestamp) as MaxTime FROM TM_ExecDefinitionRuns edr2 GROUP BY edr2.ExecDefID_fk) as maxRun
ON edr.ExecDefID_fk = maxRun.ExecDefID_fk AND edr.ExecutionTimestamp = maxRun.MaxTime
WHERE etn.ParentFolderID_fk = 24
AND etn.ProjectID_fk = 0
GROUP BY edr.ExecDefRunID_pk_fk, etn.NodeName, edr.PassedCount, edr.FailedCount, edr.NotExecutedCount, edr.Starter, edr.ExecutionTimestamp, edr.Duration, EDR.BuildName

Replace the folderID and projectID's with your own, based on the top-level you are trying to query from.

Any questions let me know.

Thanks,
Matthew

[JamesA99]

Thanks Matthew, this is really great, but it would be good to have the manual tester assigned included as it will help with our planning as we can view who is assigned to the execution plan (like a team lead) and than we would also know who has started the test this is because the person who starts the test might not be the same person who is assigned it as a manual test this will help us with planning and scheduling as some teams might be overloaded with executions to run and the manual tester will also be helping with status updates and giving out who will actually run the execution plan....we can than factor in how long the execution plans have run as well.

I see the TM_Dependencies table but not sure where the AssignedUserID_fk hooks into?

thanks again for your help,
James

[Matthew McGibbon]

Hi James,

By managing load on automated tests for users seems quite strange as it seems to go against the point of automated testing, as in this should require little to no user intervention as possible. You will obviously have to analyze tests results but assigning someone to this is not how this should work.

For automated tests, you either have the person who started it or in most cases, it may be started by a build or schedule that an execution plan may need to follow for example this set of tests runs once a week.

A lot of what you have mentioned does make sense for manual tests, as you do manage who updates status and runs the test but it does seem that you are running automated tests with a manual process. The length of the test is available as the duration in ms so you can convert this to a particular timeframe if required. Duration of automated tests should not place any load on manual testers as they are run by the application independently. Duration does not mean for example the tester is unavailable during this run, in the same way, a manual test would.

If you really need to get this it is available on the table:

TM_ExecNodes_Users

This links users to the Execution Nodes, on the Execution Plan ID.

TM_Dependencies table again would be something different, as you can mark an execution plan as dependent upon another execution plan, this is what this table is referencing.

Thanks,
Matthew

[JamesA99]

thanks Matthew for the info. what is a bit of an issue for us is the lack of functionality to rerun failed manual tests...the automated execution plans seem to handle this better than the manual execution plans and our testers like using the automated execution plans over the manual. there seems to be no mechanism or functional aspect to manual testing where a user can go and select to rerun manual tests...the test cycle is finished and closed off so the user has to build a new test cycle and include the failed tests....in the automated execution plan the tester selects the option to run failed tests and run them manually, a new run ID is created and the results are tracked for the failed tests. we are not to concerned about tracking test cycle dates, that can be easily tracked in the way we structure our folders and we can run report on tests ran between dates or on the folder containing the tests for a certain release or cycle.
if you have some further insight into this please let me know your thoughts...any info is always greatly appreciated.

Thanks,
James

[Matthew McGibbon]

Hi James,

Thanks for the feedback, if you do have suggestions for this just open a ticket for an enhancement and we can log this with our Product team to review.

For running failed tests from testing cycles again, the easiest way to do this would be to select the actions icon on the testing cycle and click duplicate.

In this dialog now you will have a test assignment option:

This allows a user to quickly create a new testing cycle for only failed tests. It will also copy over the testers part of this cycle saving the time to configure this again.

You then can use the built-in dashboard panels for tracking the progress of these testing cycles and getting timeframes for execution.

Thanks,

Matthew

[JamesA99]

thanks Matthew. I guess what it is coming down too is the testers are liking the way the automation execution plan are laid out using tabs and the functionality it uses and also how everything is self-contained in a one stop shop display for the testers to view. also, what happens is that if we use the duplication of test cycles and check failed test only than we will have a more test cycles folders to manage where in the automation execution plan no new folder is created just a new run. thanks again for your help and feedback, over the last few months we have been really pushing everything through silk central so there has been a lot of trial and error in getting everything setup and running.

thanks,
James