Absent Member.
Absent Member.
1067 views

TEST DOCUMENT VIEW not usable/correct after tests rerun due to automatic overlay of 'NOT EXECUTED' status

The mechanism to set each test run just launch automatically to NOT EXECUTED result generates a not correct information in the TEST DOCUMENT VIEW. This is true expecially when you are at second or next runs of same tests and its depends by the test execution dynamic.

In our experience the tests are rexecuted several times in same project, and a test run take days to be closed, so the test manager need to work in 'test result continuity', he need to know always the last consolidated result (set by its Tester, not by SCTM automatically) to be aware about the current 'real' status in the project.

For example: we have 100 tests in total with finished EP and this result: 60 PASSED,  30 FAILED and 10 NOT EXECUTED, as shown in Test Documentation view.  A new build is installed and now the test manager decides to re-run 50 tests, all the FAILED and 5 NOT EXECUTED and a portion of the PASSED (5 Test for regression scope) tests, so immediately the Test Document view shows that 40 test are PASSED and 60 are NOT EXECUTED.What's happen to the previous 'project status' in term of FAILED and PASSED ? disappear!

Question: is it a correct information about the current status of the project ? It is useful ?  what this information says? Help me to understand.. please.

Our opinion (15 test manager opinions) is that this information is incorrect and not useful. The real information that the test manager needs is to understand where is the project 'cross runs and builds', minute by minute, for a angle view and which tests are to be confirmed (because are running again) in another angle views. The currect document view cannot help in any ways.

The test status, when a test is just launched, cannot be 'NOT EXECUTED', maybe we can consider a 'RUNNING' or 'PROGRESS' status, or better NOTHING. Only when the Tester select the test and set a result, it MUST BE consider, never before, as all the other main competitor does (HP QC and IBM QM).

Thanks a lot for your attention.

Gianni

 

Gianni Pistarini

+39 3356756790

0 Likes
1 Reply

Hi,

Deciding which status should be shown at which time is an ongoing discussion topic.

Over the years different approaches have been implemented and released – the current state is that the last status is shown real-time. Hence if you set a status in your manual testing area it is immediately shown in the tests area. This also means that on starting an execution all tests are set to “Not Executed” as this is the initial status for tests within an execution – as the test has not been executed for this specific execution.

Regards,

Florian

Product Owner - Silk
0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.