17.9 Specifying Correlation Structures
Independence: In standard linear models, the assumption on the residuals \(\epsilon_{i} \sim \mathcal{N}(0, \sigma_{\epsilon}^{2})\) means that
The variance of each observation is \(\sigma_{\epsilon}^{2}\)
The covariance between two different observations \(0\)
Consider \(n=4\) observations, \(y_{1}, \ldots , y_{4}\). Visually the covariance matrix between these four observations would look like this:
\[ \begin{array}{c|cccc} & y_{1} & y_{2} & y_{3} & y_{4}\\ \hline y_{1} & \sigma_{\epsilon}^{2} & 0 & 0 & 0\\ y_{2} & 0 & \sigma_{\epsilon}^{2} & 0 & 0\\ y_{3} & 0 & 0 & \sigma_{\epsilon}^{2} & 0\\ y_{4} & 0& 0 & 0 & \sigma_{\epsilon}^{2} \end{array} \]
We can also write the covariance matrix as \(\sigma_{\epsilon}^{2}\) times the correlation matrix.
\[ \begin{bmatrix} \sigma_{\epsilon}^{2} & 0 & 0 & 0\\ 0 & \sigma_{\epsilon}^{2} & 0 & 0\\ 0 & 0 & \sigma_{\epsilon}^{2} & 0\\ 0& 0 & 0 & \sigma_{\epsilon}^{2} \end{bmatrix} = \sigma_{\epsilon}^2 \begin{bmatrix} 1 & 0 & 0 & 0 \\ & 1 & 0 & 0 \\ & & 1 & 0 \\ & & & 1 \end{bmatrix} \]
- Compound Symmetry or Exchangeable: The simplest covariance structure that includes correlated errors is compound symmetry (CS). Here we see correlated errors between individuals, and note that these correlations are presumed to be the same for each pair of responses, namely \(\rho\).
\[ \sigma_{\epsilon}^{2} \begin{bmatrix} 1 & \rho & \rho & \rho \\ & 1 & \rho & \rho \\ & & 1 & \rho \\ & & & 1 \end{bmatrix} \]
- Autoregressive: Imagine that \(y_{1}, \ldots , y_{4}\) were 4 different time points on the same person. The autoregressive (Lag 1) structure considers correlations to be highest for time adjacent times, and a systematically decreasing correlation with increasing distance between time points. This structure is only applicable for evenly spaced time intervals for the repeated measure.
\[ \sigma_{\epsilon}^{2} \begin{bmatrix} 1 & \rho & \rho^{2} & \rho^{3} \\ & 1 & \rho & \rho^{2} \\ & & 1 & \rho \\ & & & 1 \end{bmatrix} \]
- Unstructured: The Unstructured covariance structure (UN) is the most complex because it is estimating unique correlations for each pair of observations. It is not uncommon to find out that you are not able to use this structure simply because there are too many parameters to estimate.
\[ \begin{bmatrix} \sigma_{1}^{2} & \rho_{12} & \rho_{13} & \rho_{14} \\ & \sigma_{2}^{2} & \rho_{23} & \rho_{24} \\ & & \sigma_{3}^{2} & \rho_{34} \\ & & & \sigma_{4}^{2} \end{bmatrix} \]
- Random Intercept Model
Let \(y_{1}\) and \(y_{2}\) be from group 1, and \(y_{3}\) and \(y_{4}\) be from group 2.
- error terms between groups are uncorrelated (groups are independent)
- two different observations from the same group have covariance \(\sigma_{\alpha}^{2}\)
- individuals now have the error associated with their own observation but also due to the group \(\sigma_{\epsilon}^{2} + \sigma_{\alpha}^{2}\)
\[ \left[ \begin{array}{cc|cc} \sigma_{\epsilon}^{2} + \sigma_{\alpha}^{2} & \sigma_{\alpha}^{2} & 0 & 0\\ \sigma_{\alpha}^{2} & \sigma_{\epsilon}^{2} + \sigma_{\alpha}^{2} & 0 & 0\\ \hline 0 & 0 & \sigma_{\epsilon}^{2} + \sigma_{\alpha}^{2} & \sigma_{\alpha}^{2}\\ 0 & 0 & \sigma_{\alpha}^{2} & \sigma_{\epsilon}^{2} + \sigma_{\alpha}^{2} \end{array} \right] \]