The following will be discussed in this chapter: Covariance and correlation, correlation coefficient, ageometric picture, geometric interpretation of correlation coefficient, orthogonality. | Lecture Signals, systems & inference – Lecture 13: Vector picture for first- and second-order statistics; MMSE and LMMSE estimation Vector picture for first- and second-order statistics; MMSE and LMMSE estimation , Spring 2018 Lec 13 1 Covariance and correlation Covariance: eX,Y = E[(X - µX )(Y - µY )] = E[XY ] -µX µY | {z } Correlation rX,Y Shorthand notation: eXY , rXY 2 Correlation coefficient E↵ect of shifting and scaling: If V = ↵(X - β) then µV = ↵(µX -β) , V = ↵ X If W = (Y ) then VW =↵ XY For a shift- and scale-invariant measure: XY ⇢XY = = ⇢V W X Y 3 A geometric picture Think of X and Y as vectors, with inner product E[XY ] µX = E[] : inner product of X and “random variable” 1 E[X 2 ] : squared length of X e = X - µX : vector di↵erence between X and “random variable” µX X e OX : length of X 4 Geometric interpretation of correlation coefficient Y - mY sY u = cos-1r X - mX sX 5 Orthogonality E[XY ] = 0 Correlation is 0, but not uncorrelated! Uncorrelated = zero covariance, ., E[XY ] = E[X]E[Y ] 6 MIT OpenCourseWare Signals, Systems and Inference Spring 2018 For information about citing these materials or our Terms of Use, visit: . 7
