OK, as this is my first article here, firstly let me introduce myself. I am a long standing employee of Micro Focus, and I have worked on many products in that time. These have included the Net Express line of products and in recent years the various Eclipse based offerings we have, such as Net Express / Server Express for Eclipse and Server Express Remote Development Option.
The Visual COBOL product is particularly exciting to me as it signifies a step change improvement in the depth of our COBOL offerings. Of course, as the product becomes more complex and we do more exciting things with COBOL (.NET, Azure, JVM, cross platform remote development to name but a few), our testing of the product must also become deeper and more complex. And this is where things get interesting. My first blog entry here covers the ‘core’ component – that is the compiler, the file handler and the runtime.
Testing the core product in depth is a time consuming business - there are many hundreds and thousands of routes that a piece of code can take, with many build directives and compiler options as well, so we need to be mindful of this in our testing. For this reason, we have many thousands of unit tests which we continually add to so we can be sure the core component works correctly. Each code commit to source control is accompanied by a corresponding test which exercises and verifies the code change. Stability is vital for the core component so we run these tests across several platforms on a daily basis, meaning that every day we run around 120,000 test cases against our product. There are a number of integration points between the core and the development environments which are also tested.
This isn’t the end of the story though. The tests we run on a daily basis are only run against a subset of our supported platform/chipset/bitism test matrix. When we are happy with these test results the product is submitted into our release management system for further testing and build release activities. This provides testing on a further set of platforms so we end up with a total of around 250,000 test cases. This extended build and test is generally done weekly, but sometimes more often during the release phase of a product rollout.
Of course, even this is not the end of the story – although this process ensures the stability of the 'core' components, Visual COBOL is made up of much more than that including other non ‘core’ features such as SQL, XML and the Debugger, and of course Visual Studio and Eclipse integration. I’ll cover these separately.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.