Whereas test design, even when supported by tools, requires insight and ingenuity in similar measure to other facets of software design, test execution must be sufficiently automated for frequent reexecution without little human involvement. This chapter describes approaches for creating the run-time support for generating and managing test data, creating scaffolding for test execution, and automatically distinguishing between correct and incorrect test case executions. | Test Execution (c) 2007 Mauro Pezzè & Michal Young Ch 17, slide 1 Learning objectives • Appreciate the purpose of test automation – Factoring repetitive, mechanical tasks from creative, human design tasks in testing • Recognize main kinds and components of test scaffolding • Understand some key dimensions in test automation design – Design for testability: Controllability and observability – Degrees of generality in drivers and stubs – Comparison-based oracles and self-checks (c) 2007 Mauro Pezzè & Michal Young Ch 17, slide 2 Automating Test Execution • Designing test cases and test suites is creative – Like any design activity: A demanding intellectual activity, requiring human judgment • Executing test cases should be automatic – Design once, execute many times • Test automation separates the creative human process from the mechanical process of test execution (c) 2007 Mauro Pezzè & Michal Young Ch 17, slide 3 Generation: From Test Case Specifications to Test Cases • Test design often yields test case specifications, rather than concrete data – Ex: “a large positive number”, not 420023 – Ex: “a sorted sequence, length > 2”, not “Alpha, Beta, Chi, Omega” • Other details for execution may be omitted • Generation creates concrete, executable test cases from test case specifications (c) 2007 Mauro Pezzè & Michal Young Ch 17, slide 4 Example Tool Chain for Test Case Generation & Execution • We could combine . – A combinatorial test case generation (like ) to create test data • Optional: Constraint-based data generator to “concretize” individual values, ., from “positive integer” to 42 – DDSteps to convert from spreadsheet data to JUnit test cases – JUnit to execute concrete test cases • Many other tool chains are possible . – depending on application domain Photo: (c) Scott Robinson (clearlyambiguous on Flickr) , creative commons attribution license 17, slide 5 (c) 2007 Mauro Pezzè & Michal .
