An immediate generalization of § is to fit a set of data points (xi , yi ) to a model that is not just a linear combination of 1 and x (namely a + bx), but rather a linear combination of any M specified functions of x. | General Linear Least Squares 671 General Linear Least Squares An immediate generalization of is to fit a set of data points xi yf to a model that is not just a linear combination of 1 and x namely a bx but rather a linear combination of any M specified functions of x. For example the functions could be 1 x x2 . xM-1 in which case their general linear combination y x ai a2x a x2 aM xM-1 is a polynomial of degree M - 1. Or the functions could be sines and cosines in which case their general linear combination is a harmonic series. The general form of this kind of model is M y x 2 akXk x k 1 where X1 x . XM x are arbitrary fixed functions of x called the basis functions. Note that the functions Xk x can be wildly nonlinear functions of x. In this discussion linear refers only to the model s dependence on its parameters ak. For these linear models we generalize the discussion of the previous section by defining a merit function N i 1 yi - k 1 akXk xi ai As before ai is the measurement error standard deviation of the ith data point presumed to be known. If the measurement errors are not known they may all as discussed at the end of be set to the constant value a 1. Once again we will pick as best parameters those that minimize 2. There are several different techniques available for finding this minimum. Two are particularly useful and we will discuss both in this section. To introduce them and elucidate their relationship we need some notation. Let A be a matrix whose N x M components are constructed from the M basis functions evaluated at the N abscissas xi and from the N measurement errors ai by the prescription Aij jil The matrix A is called the design matrix of the fitting problem. Notice that in general A has more rows than columns N M since there must be more data points than model parameters to be solved for. You can fit a straight line to two points but not a very meaningful quintic The design matrix is shown .
