Standardized Functional Verification- P14:Every manager who brings a design to tape-out or who purchases IP must eventually face these questions. The ability to answer these questions based on quantitative analysis is both vital and yet elusive. In spite of the enormous technical advances made in IC development and verification software, the answers to these questions are still based largely on guesswork and hand waving. | Tracking Progress 8 115 Tracking Progress 8 The late great writer Douglas Adams was quoted as saying as he labored over his characteristically late manuscripts I love deadlines. I love the whooshing sound they make as they go by. Project managers love deadlines nearly as much as authors and are routinely required to provide updated estimates for when various milestones will be met. The most critical milestone is the one at which the risk of a functional bug is sufficiently low so that final artwork can be prepared for tape-out. How close is our project to completion That question will be addressed in detail in the last two chapters but there are some metrics that provide some indications of progress towards completion. To track progress in reducing the risk of a bug in a verification target the following metrics can provide key insights Code coverage 100 statement coverage see chapter 6 indicates that about one third of the overall work to expose bugs is completed. Fig. illustrates how code coverage might increase as a function of time for a typical project. Bug discovery rate If you stop looking you will stop finding. A non-convergent regression suite or one that has reached the asymptote of its convergence see next chapter will eventually stop looking because it is not exercising previously unexercised functionality. See Fig. for an example of how bug discovery rate might appear for a typical project. Bug counts and severity for risk analysis. Each organization counts bugs and assigns severity somewhat differently so comparison of such data from different organizations is often not reliable. Bug locality Bugs are not typically distributed uniformly within the target but are often clustered by code module or by functionality as characterized by the variables on which faulty behavior is dependent . Tracking the locality of bugs can provide useful indicators on how to focus continued verification activity. Similarly associating bugs with the .
