Chapter 33 EVALUATING THE PREDICTIVE ACCURACY OF MODELS Contents 1. Introduction 2. Numerical solution of nonlinear models 3. Evaluation of ex ante forecasts 4. Evaluation of ex post forecasts 5. An alternative method for evaluating predictive accuracy 6. Conclusion References | Chapter 33 EVALUATING THE PREDICTIVE ACCURACY OF MODELS RAY C. FAIR Contents 1. Introduction 1980 2. Numerical solution of nonlinear models 1981 3. Evaluation of ex ante forecasts 1984 4. Evaluation of ex post forecasts 1986 5. An alternative method for evaluating predictive accuracy 1988 6. Conclusion 1993 References 1994 Handbook of Econometrics Volume III Edited by Z. Griliches and . Intriligator Elsevier Science Publishers BV 1986 1980 R. C. Fair 1. Introduction Methods for evaluating the predictive accuracy of econometric models are discussed in this chapter. Since most models used in practice are nonlinear the nonlinear case will be considered from the beginning. The model is written as fi yt x ai uit z l . f l . T 1 where y is an -dimensional vector of endogenous variables xt is a vector of predetermined variables including lagged endogenous variables a is a vector of unknown coefficients and uit is the error term for equation i for period t. The first m equations are assumed to be stochastic with the remaining u t i m 1 . n identically zero for all t. The emphasis in this chapter is on methods rather than results. No attempt is made to review the results of comparing alternative models. This review would be an enormous undertaking and is beyond the scope of this Handbook. Also as will be argued most of the methods that have been used in the past to compare models are flawed and so it is not clear that an extensive review of results based on these methods is worth anyone s effort. The numerical solution of nonlinear models is reviewed in Section 2 including stochastic simulation procedures. This is background material for the rest of the chapter. The standard methods that have been used to evaluate ex ante and ex post predictive accuracy are discussed in Sections 3 and 4 respectively. The main problems with these methods as will be discussed are that they 1 do not account for exogenous variable uncertainty 2 do not account for the fact that forecast-error
