Tests that span multiple products? - Micro Focus Community

Per _stman · ‎2012-09-25

Hi!

I'm having trouble figuring out the best way to organize our "end-to-end" tests.

These tests consist of scenarios that involve multiple products in our stack (desktop client, web-based applikation, back-end server, mobile app).
Something similar to creating a document in word and publishing it to SkyDrive and viewing it/editing it in Office365.

Tests are stored in a test container that associates to one product, But I have multiple products that these tests relate to. Also, I want to associate runs of execution plans to build numbers through buildinfo files, but since the individual products does not necessarily have the same build number, this becomes an issue.

Anyone have any ideas?

Florian Guenther · ‎2012-09-26

Hi,

I really like your questions! 🙂

Solving this multi-product testing topic floats around in my head for several months now. Unfortunately I have no real answer for you at the moment (in the existing product) that is not connected to duplicating tests, having several test containers, execution plans, etc. - hence increasing administration.

!! Here starts future scenario thinking that could be/but has not to be part of a future release!!:

Maybe we have to remove the product from the tests - maybe moving it to requirements, which would be more logical anyways, as the requirements describe behaviour/features/etc. of a product.

Build information has then to be moved down also on a more granular level ... As you see I have to think more about it.

One more question are these automated or manual tests?

Thanks,

Florian

Product Owner - Silk

Per _stman · ‎2012-09-27

My thought right now is in the direction of having a product that represents our "stack" of products. Maybe add the individual products as components of the stack product.

Then either create versions of this stack product manually that represents collections of builds of the individual products, or have our CI system create a buildinfo file with build number and also generate a mapping between stack product version/buildnumber and the corresponding product version/buildnumber.

A longer term solution would then be to make sure the constituent products share the same build number to avoid the need for this mapping.

Ideally, one could consider enhancing SCTM to handle this. Maybe it would be enough to let components optionally have versions and build numbers that could be represented in the buildinfo file?

Per _stman · ‎2012-09-27

Oh, and to answer your question; these tests will most probably be a mix of automated and manual tests. I can see some of these tests being what we call "soap opera tests", and those are typically geared towards manual/exploratory.