After testing this new report functionality my users (and I have to agree) came to the conclusion that the current implementation of the Plan vs Actual – Report is very hard to use.
Our expectation was to see the progress of each period but the system just divides the amount of testcase instances into packages and name with the dates of this period.
What I was not aware of is that data in a previous period will change if you retest the effected testcase instance again. The reason is simple that this report will just count the latest run at all and show it ONLY in the LATEST period where it was executed.
This is valid for the yellow line (actual) too. It will remove the value "current" from a past period if a later execution will be executed.
So from a pure quality perspective it´s interesting to have this report too but I would never show it to a steering committee because it could lead easily to misunderstandings what was done in the past (e.g. If I would simple retest all testcase instances in the last week again ONLY the last period would have data … which would be mega confusing)
In our opinion the expected result should/could be:
If a week is passed you could be interested in
The „plan“ line (blue) should be kept as it is but the “current” line (yellow) and the bars are confusing currently.
Hope this can be changed / added in the near future.
Yes, this is exactly how we would need it! With this method we could avoid a very common report that has to be currently done outside of ALM.
Your current method shows something completely different but I would be happy if your current and this new method would be available in parallel, because for both there is a separated use case where they can be worthful.
This is Mike, the feature's functional designer. I'd like to double-check that I'm understanding you correctly. If the following modifications are made:
Do you believe they meet your requirements?