PROBABILITY 11

Covariance and Correlation

Covariance describes how two random variables vary together; correlation gives a dimensionless measure of their linear association.

Covariance

\[\boxed{\operatorname{Cov}(X,Y)=E[(X-E[X])(Y-E[Y])]}\]

An equivalent form is

\[\boxed{\operatorname{Cov}(X,Y)=E[XY]-E[X]E[Y]}.\]

1. Interpretation

Positive covariance means values above the mean of one variable tend to occur with values above the mean of the other. Negative covariance indicates opposite movement.

2. Variance is a covariance

\[\operatorname{Cov}(X,X)=\operatorname{Var}(X).\]

3. Variance of a sum

\[\boxed{\operatorname{Var}(X+Y)=\operatorname{Var}(X)+\operatorname{Var}(Y)+2\operatorname{Cov}(X,Y)}\]

This shows exactly where dependence affects total variability.

4. Independence

If \(X\) and \(Y\) are independent and the relevant moments exist, then

\[\operatorname{Cov}(X,Y)=0.\]

The converse is not generally true: zero covariance does not necessarily imply independence.

5. Correlation

For nonzero standard deviations,

\[\boxed{\rho_{X,Y}=\frac{\operatorname{Cov}(X,Y)}{\operatorname{SD}(X)\operatorname{SD}(Y)}}\]

Correlation satisfies

\[-1\le\rho_{X,Y}\le1.\]

6. Why covariance matters in models

Variables in dynamical systems often interact, so their uncertainties are related. Moment equations for multivariable stochastic models therefore naturally contain covariance terms.