Error Matrix and Correlations

Let's take a closer look at the meaning of the Taylor series expansion of $\chi ^2$ about the minimum. Suppose there are only two parameters. Call them $a_1$ and $a_2$. Let the minimum (best fit values ) be at $a_1^*$ and $a_2^*$. The Taylor series expansion is then

$$\chi^2 = \chi^2(a_1^*,a_2^*) + \frac{1}{2}(a_1 - a_1^*)^2... ...a_2 - a_2^*)\frac{\partial^2\chi^2}{\partial a_1 \partial a_2}$$ (43)

The second partial derivatives give the matrix elements of the Hessian matrix $H$. This matrix is twice the curvature matrix $M$. We used the inverse of the curvature matrix previously to get the errors in the parameters.

$$H_{i,j} = 2M_{i,j} = \frac{\partial^2\chi^2}{\partial a_i \partial a_j}$$ (44)

$$R = M^{-1}$$ (45)

From this expression we can see that the contour lines of constant $\chi ^2$ are concentric ellipses, centered at the best fit value. A one standard deviation change in the parameter values raises $\chi ^2$ one unit above the minimum. This $1 \sigma$ contour is called the error ellipse. If the mixed partial derivative vanishes, then the major and minor axes of the ellipses are parallel to the parameter axes. If not, then the ellipse is rotated with respect to the axes. If the ellipses are particularly eccentric and rotated, as shown in Fig. 2, we see that it takes a large change in parameter values along the major axis in order to get to the $1 \sigma$ contour. This implies that the parameter values are strongly correlated. A measure of this correlation is obtained from the elements of the inverse matrix $R$, which is also called the ``error matrix''.

$$\rho = R_{1,2}/\sqrt{R_{11} R_{22}}.$$ (46)

A small or zero value implies little correlation. A value close to 1 corresponds to a strong correlation.

An example of a situation that leads to such a correlation would be a linear fit to a cluster of $x,y$ points that are all distant from the $y$ axis. See Fig. 3. The $y$ intercept is especially uncertain, because it is far from the points. But the value of the intercept is strongly correlated with the slope, since any reasonably good fit must be a line that passes through the points. A small change in the slope of this line requires a corresponding change in the intercept. If we look at the Hessian matrix for a linear $\chi ^2$ fit, we find that in precisely this situation, the off diagonal terms are large compared with the diagonal terms, so $\rho$ is large.

Figure 2: One sigma error ellipse, showing a strong correlation between the parameters $a_{1}$ and $a_{2}$. The ellipse is obtained from Eq. (43) by setting $\Delta \chi ^2 = \chi ^2 - \chi ^2(a_1^*,a_2^*) = 1$, i.e. one unit above the best value of $\chi ^2$.

Figure 3: A linear fit producing strong correlations between slope and intercept. The two lines represent small departures from the best fit. An increase in slope can be compensated by a decrease in intercept.