Having problems with your account or logging in?
A lot of changes are happening in the community right now. Some may affect you. READ MORE HERE

Bug of the Month - April 2012

Matt Schuetze1 Absent Member.
Absent Member.
0 2 1,046

It dawned on me in April that bugs and testing to find bugs reaches infinity pretty fast. In high school math, we are taught that asymptotes stretch to infinity. This is simple to visualize on a graph, but numerically no one has a gut feel for how big (or how far out) infinity is. Later on in college chemistry class, we are taught that moles are the better unit for measuring large numbers of things, like molecules. Avagadro’s number -- 6.02*10^23 -- while certainly a very, very large number, never felt like "infinity". You could hold several dozen moles of various items in a small beaker. In my particular secondary educational background, we talked about the cross section of nuclear interactions. The units were "barns", which amounted to tiny numbers on the order of 10^-24 square meters, so even if something was "as big as a barn" it would still take millions of nuclei to add up to something larger than infinitesimally small. When we throw around units like moles and barns, with numbers of this scale, it remains challenging to not throw in the towel, just call something "infinite", and slink off to the pub to avoid thinking about them over a pint. We use those units to bring back very large numbers to the realm of human-visualized quantities.

Software testing however rapidly reaches "infinity" when the permutations and combinations of linearly independents variables mushroom a test matrix to beyond a scale a human can visualize or comprehend. For a real and raw example, the DevPartner Java support matrix consists of eight mostly independent dimensions to pick one cell in a possible runtime environment. The dimensions are JVM version, JVM Bitness, OS version, OS bitness, AppServer, IDE, Browser, and Browser public JRE. Even though these variables may have a discrete set of values, the combinatorics involved with spanning the space to determine a suitable test matrix rapidly approaches infinity, or in other words beyond the ability for a human to visualize and construct a reasonable test plan, especially since once you've selected values in each of these dimensions, you still have a wide number of tests to run to ensure stability and compatibility. Draw up a short plan, and bugs will fill the remainder of the spanned space of untested variables. Build and execute a completely exhaustive plan, and you'll consume all the energy in the universe to finish it, and likely blow quads of energy uncovering and trying to fix bugs in combinations that no other user will ever run. I like the concept of equivalence class portioning to reduce the number of test cases, but those always seem to be contrived dataflow algorithms under test. I also know that in an infinite number of rooms with an infinite number of monkeys tapping on typewriters that one of them will come up with a copy of Macbeth. I would feel better if a test protocol had a better way to knock infinity in the shins. I certainly don't have budget to consume all the energy in the universe to ensure my code is infinitely clean at all its asymptotes. Do you?

2 Comments
dhalonen Absent Member.
Absent Member.

So then, how does one write medical software? Moon launch control software? How can an automobile be created? Does it require super-humans to compete in this space?

www.fastcompany.com/.../writestuff.html - an older story, but still a great read.

mlevis Absent Member.
Absent Member.

dhalonen,

Moon launch control software or autos have a specific set of hardware, os, bitness, etc that is targeted.  Here we are talking about supporting a very wide range of OS, hardware, bitness, JVM versions off of one codebase. Can you truly test every variation?

Consider

X86 Os's (XP, Win2003, Vista, 2003R2, Win 7, Win 2008)

X64 OS's(XP, Win2003, Vista, 2003R2, Win 7, Win 2008)

SP's for Each (SP1, 2, 3, 4...)

VS that we integrate into ( VS2003, 2005, 2008, 2010, VS11)

SP's for each SP1, 2, 3, 4, 5...

Now test each OS with every possible combination of VS's installed and each variation of SP for OS & combinations of VS.

I think that level of testing is what is being talked about here.

The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.