Dynamically distribute test cases between a pool of execution servers

[mojorero]

Good morning fellas,

at our company, we have set up some execution servers, where we execute test suites, one test suite per execution server.

Now the execution of our tests from our main test suite lasts for about 15 hours and we are thinking on ways of splitting this worload (the corresponding tests of this test suite), so each batch of work can be done parallel to the others. The first, easiest-to-do idea which comes to my mind is to split the test suites into little test suites, which would be run simultaneously and then somehow collect all the results from each test suite and with all this info elaborate a single report with the results of all tests.

But this would not be so efficient, and the traceability of the errors could be complex.

So I am thinking of a better, more elegant possibility, which just automates all this stuff.

Does Silk Central Test Manager offer the possibility of having the test cases of a test suite dynamically distributed between our pool of execution servers?

Kind regards and thank you in advance.

[Florian Guenther · Accepted solution]

Hi,

no, Silk Central does not provide any functionality to dynamically distribute one execution plan across several execution servers to have the tests run in parallel. All tests of one execution plan are executed on one execution server.

As you said the only possibility at the moment is to have several smaller execution plans.

It would be interesting to understand if you use physical or virtual execution servers.

In terms of physical execution servers I am not sure if it is a "valid" scenario - can I really say that the execution of test 1 on physical execution server 1 is the same as executing test 1 on physical execution server 2? Can I still compare runs if tests are not always executed on the same execution server and therefore environment?

In the case of virtual execution servers - where the same image is checked out several times - I think it is ok.

What is your oppinion on that?

Thanks,

Florian

[mojorero]

Hi Florian,

thank you for your answer, that was fast.

This is a very needed functionality, I suppose somebody in our situation has already found a workaround. But unfortunately I can't find it.

We use virtual machines as execution servers.

[Florian Guenther]

I have another question - you are saying the suite takes 15 hours to be executed - hence I assume these are no unit tests, correct? What kind of automated tests are these?

Thanks,

Florian

[mojorero]

These are JUnit tests (Silk4J).

[mojorero]

They are around 500 tests

[Per _stman]

My take on this is that it should not be up to the tool to judge this.

I can clearly see a use for being able to parallellize test runs like in this case - it all depends on the context.

If each individual test is crafted so that it is self contained and can be run by itself, and I want to run 500 such tests, it would make perfect sense to be able to parallellize this.

One could be inspired by for example the parallell test executor plugin for Jenkins.

[mojorero]

I don't understand why sould the tool not be in charge of deciding to execute the next test in the next available execution server, which would be chosen from our pool of execution servers.

[Florian Guenther]

Just in case somebody reads this old thread - with Silk Central 18.5 the functionality to distribute tests of one execution plan across all matching execution servers has been introduced.

For more details please see:

community.microfocus.com/.../silk-central-18-5---distributed-test-execution

Regards,
Florian