A Course in Black Box Software Testing
Examples of Combination (Multi-Variable) Testing
Copyright (c) Cem Kaner, 2004
Combination testing refers to tests that involve more than one variable.
A critical problem for multi-variable testing is called combinatorial explosion. The more variables you combine, the higher the number of possible tests. Combination testing is about picking a few tests that can teach us what we need to know about the space of combinations of these variables.
We often plan, document or describe the combinations using combination charts.
Some approaches to filling in the charts are fairly mechanical, such as random combination, combinatorial analysis (such as all-pairs testing), or orthogonal arrays. Each of these starts from the assumption that the variables are (should be) independent of each other. They combine boundaries or other individually interesting values (such as 0), rather than groups of values that have meaning as a combination.
Other approaches rely on meaningful relationships among variables, such as scenario tests, sequential tests, or other risk-focused tests. We study these in other sections of the course.
The following examples illustrate the use of combination tests.
- Some Configurations of the Playlist in Winamp3 Are not Working
- Refreshing Cleared History in FireFox
- FireFox's Command Line Arguments
- FireFox's Extension Manager
- Undo Font Size Bug in OpenOffice.org Writer v1.1.1
Copyright (c) Cem Kaner 2004
This work is licensed under the Creative Commons Attribution-ShareAlike License. To view a copy of this license, visit http://creativecommons.org/licenses/by-sa/2.0/ or send a letter to Creative Commons, 559 Nathan Abbott Way, Stanford, California 94305, USA.
These notes are partially based on research that was supported by NSF Grant EIA-0113539 ITR/SY+PE: "Improving the Education of Software Testers." Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.