A key problem in software testing is selecting and evaluating test cases. This chapter introduces basic approaches to test case selection and corresponding adequacy criteria. It serves as a general introduction to the problem and provides a conceptual framework for functional and structural approaches described in subsequent chapters. | Test Case Selection and Adequacy Criteria (c) 2007 Mauro Pezzè & Michal Young Ch 9, slide 1 Learning objectives • Understand the purpose of defining test adequacy criteria, and their limitations • Understand basic terminology of test selection and adequacy • Know some sources of information commonly used to define adequacy criteria • Understand how test selection and adequacy criteria are used (c) 2007 Mauro Pezzè & Michal Young Ch 9, slide 2 Adequacy: We can’t get what we want • What we would like: – A real way of measuring effective testing If the system system passes an adequate suite of test cases, then it must be correct (or dependable) • But that’s impossible! – Adequacy of test suites, in the sense above, is provably undecidable. • So we’ll have to settle on weaker proxies for adequacy – Design rules to highlight inadequacy of test suites (c) 2007 Mauro Pezzè & Michal Young Ch 9, slide 3 Adequacy Criteria as Design Rules • Many design disciplines employ design rules – .: “traces (on a chip, on a circuit board) must be at least wide and separated by at least ” – “The roof must have a pitch of at least _ to shed snow” – “Interstate highways must not have a grade greater than 6% without special review and approval” • Design rules do not guarantee good designs – Good design depends on talented, creative, disciplined designers; design rules help them avoid or spot flaws – Test design is no different (c) 2007 Mauro Pezzè & Michal Young Ch 9, slide 4 Practical (in)Adequacy Criteria • Criteria that identify inadequacies in test suites. – Examples – if the specification describes different treatment in two cases, but the test suite does not check that the two cases are in fact treated differently, we may conclude that the test suite is inadequate to guard against faults in the program logic. – If no test in the test suite executes a particular program statement, the test suite is inadequate to guard against faults in that statement. •
