Idea ID: 2869711

Strong need of an improvement of the new report "Plan vs Actual"

Status : New Idea
4 months ago

After testing this new report functionality my users (and I have to agree) came to the conclusion that the current implementation of the Plan vs Actual – Report is very hard to use.

Our expectation was to see the progress of each period but the system just divides the amount of testcase instances into packages and name with the dates of this period.

What I was not aware of is that data in a previous period will change if you retest the effected testcase instance again. The reason is simple that this report will just count the latest run at all and show it ONLY in the LATEST period where it was executed.

This is valid for the yellow line (actual) too. It will remove the value "current" from a past period if a later execution will be executed.

So from a pure quality perspective it´s interesting to have this report too but I would never show it to a steering committee because it could lead easily to misunderstandings what was done in the past (e.g. If I would simple retest all testcase instances in the last week again ONLY the last period would have data … which would be mega confusing)

In our opinion the expected result should/could be:

If a week is passed you could be interested in

  • Prio1: Focus to “Progress at all”:
    How many testcase instances where executed at the end of this period at all + what was the plan till end of this period.
    Here a different scale between Plan/Actual and the execution-status of this period could be different to make it more readable.
    Example:

       and/or

  • Prio2: Focus to “Progress in each week”:
    How many testcases instances in which status exist at the end of this period + what was the plan till end of this period
    In that case the assumption would be that the bar ends at the yellow line

The „plan“ line (blue) should be kept as it is but the “current” line (yellow) and the bars are confusing currently.

Hope this can be changed / added in the near future.

Labels:

Graph
Reporting
Usability
  • I can't promise when a new graph will be implemented, but we will take your suggestions into consideration.

  • @Mike:
    Yes, this is exactly how we would need it! With this method we could avoid a very common report that has to be currently done outside of ALM.

    Your current method shows something completely different but I would be happy if your current and this new method would be available in parallel, because for both there is a separated use case where they can be worthful. 

  • This is Mike, the feature's functional designer. I'd like to double-check that I'm understanding you correctly. If the following modifications are made:

    1. Even though any test instances are rerun in a later time span, the previous time frame's results will not be modified.
    2. The actual line represents the total number of test instances that have been executed during each time period; No matter if they have been performed once, twice, or multiple times, they will only be counted only once on the line.

    Do you believe they meet your requirements?