| Welcome | Foundations | Bug Advocacy | Test Design | Exploratory Testing | Taking Exams | Policies | Extras | Instructors Course | Metrics | Engineering Ethics |

Test Design: A Survey of Black Box Software Testing Techniques

Overview

This course is a survey of test techniques. We look at a few techniques more closely than the rest but we don't expect this course to make you a skilled practitioner of any technique.

The course takes a fast skim of several dozen techniques, and looks at a few in detail.

The current slide set comes in the course sequence after sections on:

We then skim a group of 10 different techniques, noticing their different objectives, strengths and blind spots.

More info on the Learning Objectives for Test Design: A Survey of Black Box Software Testing Techniques are available on the BBST.info website.

Slides

  • Slide set for all Test Design course lectures (PDF)
  • Videos

    Lecture 1: Function Testing, Tours, & A Taxonomy of Techniques

    This lecture provides a structure for organizing your thinking about the many test techniques, and a focus on testing tours and on function testing. (Why together? The most common tour is an excellent foundation for function testing.)

  • part A (9 mins, 14 secs) (MP4)
  • part B (9 mins, 20 secs) (MP4)
  • part C (3 mins, 32 secs) (MP4)
  • part D (10 mins) (MP4)
  • part E (9 mins, 51 secs) (MP4)
  • part F (4 mins, 38 secs) (MP4)
  • Lecture 2: Risk-Based Testing, Testing Strategy and the Heuristic Test Strategy Model

    This lecture introduces risk-based testing, including failure mode analysis and quicktests. The lecture also examines a few concepts important for developing a testing strategy.

  • part A (9 mins,4 secs) (MP4)
  • part B (9 mins,59 secs) (MP4)
  • part C (9 mins, 15 secs) (MP4)
  • part D (9 mins, 11 secs) (MP4)
  • part E (1 mins, 39 secs) (MP4)
  • Lecture 3: Specification-Based Testing

    This lecture introduces specification-based testing and suggests eight questions that can guide specifications analysis.

  • part A (8 mins, 24 secs) (MP4)
  • part B (9 mins, 35 secs) (MP4)
  • part C (9 mins, 1 sec) (MP4)
  • Lecture 4: Scenarios and Use Cases. Comparatively Evaluating Test Techniques

    This lecture introduces scenario testing, including 17 approaches to creating suites of scenario tests. It also suggests comparing test techniques in terms of their core strengths and describes 18 potential core strengths.

  • part A (9 mins, 49 secs) (MP4)
  • part B (9 mins, 49 secs) (MP4)
  • part C (9 mins, 58 secs) (MP4)
  • part D (10 mins) (MP4)
  • Lecture 5: Domain Testing

    This lecture introduces domain testing, the most popular test technique. The lecture presents a schema for applying domain testing that emphasizes the technique's roots in risk-based testing.

  • part A (9 mins,26 secs) (MP4)
  • part B (9 mins, 16 secs) (MP4)
  • part C (9 mins, 57 secs) (MP4)
  • part D (6 mins, 50 secs) (MP4)
  • part E (9 mins, 57 secs) (MP4)
  • Lecture 6: Combination Testing

    This lecture focuses on ways to test several variables together, emphasizing combinatorial approaches (such as all-pairs).

  • part A (7 mins, 20 secs) (MP4)
  • part B (7 mins, 17 secs) (MP4)
  • part C (9 mins, 49 secs) (MP4)
  • part D (9 mins, 6 secs) (MP4)
  • Readings

    All readings archived on this site are reposted by permission of the copyright holders.

    Required Readings

  • James Bach: Heuristic Test Strategy Model (PDF)
  • Michael Bolton: Blog: Why We Do Scenario Testing (PDF)
  • John M. Carroll: Five Reasons for Scenario-Based Design (PDF)
  • Jacek Czerwonka: Pairwise Testing in the Real World: Practical Extensions to Test-Case Scenarios (PDF)
  • Elisabeth Hendrickson, James Lyndsay, and Dale Emery: Test Heuristics Cheat Sheet (PDF)
  • Cem Kaner: What Is a Good Test Case? (PDF)
  • Cem Kaner: An Introduction to Scenario Testing (updated April 2013) (PDF)
  • Cem Kaner, James Bach, Bret Pettichord: Lessons Learned in Software Testing, Chapter 3: Testing Techniques (PDF)
  • Michael Kelly: Taking a Tour Through Test Country: A Guide to Tours to Take On Your Next Test Project (PDF)
  • Giri Vijayaraghavan and Cem Kaner: Bug Taxonomies: Use Them to Generate Better Tests (PDF)
  • Recommended Readings

  • Adler, M., & Van Doren, C. (1972). How to Read a Book. Touchstone (BOOK)
  • James Bach and Patrick J. Schroeder: Pairwise Testing: A Best Practice That Isn't (PDF)
  • Michael Bolton: Blog: Of Testing Tours and Dashboards (PDF)
  • Michael Bolton: The Factors of Function Testing (PDF)
  • Michael Bolton: Blog: Pairwise Testing (PDF)
  • Hans Buwalda: Soap Opera Testing (PDF)
  • Fiona Charles: Modeling Scenarios using Data (PDF)
  • D. M. Cohen, S. R. Dalal, M. L. Fredman, and G. C. Patton: The AETG System: An Approach to Testing Based on Combinatorial Design (PDF)
  • Ross Collard: Test Design: Developing test cases from use cases (PDF)
  • Copeland, L. (2004). A Practitioner's Guide to Software Test Design. Artech House (BOOK)
  • Gause, D.C., & Weinberg, G.M. (1989). Exploring Requirements: Quality Before Design. Dorset House (BOOK)
  • Dick Hamlet and Ross Taylor: Partition Testing Does Not Inspire Confidence (PDF)
  • Darren McMillan: Tales from the Trenches: Lean Test Cases (PDF)
  • Darren McMillan: Mind Mapping 101 (PDF)
  • NSW Government: Hazardous Industry Planning Advisory Paper No 8: HAZOP Guidelines (PDF)
  • Michael Hunter: You Are Not Done Yet Checklist (PDF)
  • Cem Kaner, Jack Falk, Hung Quoc Nguyen: Testing Computer Software, Second Edition, APPENDIX: COMMON SOFTWARE ERRORS (PDF)
  • Cem Kaner: Teaching the Software Testing Course: A Tutorial (PDF)
  • Cem Kaner and Bob Johnson: Styles of Exploration (PDF)
  • Michael Kelly: Blog: Touring Heuristic (PDF)
  • Myers, G. J. (1979). The Art of Software Testing. Wiley (BOOK)
  • Whittaker, J.A. (2002) How to Break Software, Boston: Addison Wesley (BOOK)
  • Assignments

  • A Lab on Risk-based Testing (PDF)
  • Specification-Based Testing and Analysis (PDF)
  • Feedback on Specification-Based Testing and Analysis assignment (PDF)
  • Test Design Lab (PDF)
  • Domain Testing Lab (PDF)
  • Risk-based Domain Testing and Bug Reporting (PDF)
  • Combination Testing Lab (PDF)
  • Study Guide

  • Superset of Study Guide Questions for Test Design class (PDF)