15.2 Factor Model

  • Start with P standardized variables. That is \(\frac{(x_{i}-\bar{x})}{s_{i}}\).
    • So for the rest of these FA notes, understand that each \(X\) written has already been standardized.
  • Express each variable as (its own) linear combination of \(m\) common factors plus a unique factor \(e\).

\[ X_{1} = l_{11}F_{1} + l_{12}F_{2} + \ldots + l_{1m}F_{m} + e_{1} \\ X_{2} = l_{21}F_{1} + l_{22}F_{2} + \ldots + l_{2m}F_{m} + e_{1} \\ \vdots \\ X_{P} = l_{P1}F_{1} + l_{P2}F_{2} + \ldots + l_{Pm}F_{m} + e_{P} \]

  • \(m\) is the number of common factors, typicall \(m << P\). Somemtimes, \(m\) is known in advance.
  • \(X_{i} = \sum l_{ij} F_{j}+ \epsilon_{i}\)
  • \(F_{j}\) = common or latent factors.
    • They are uncorreclated and each having mean 0 and variance 1
  • \(l_{ij}\) = coefficients of common factors = factor loadings
  • \(e_{i}\) = unique factors relating to one of the original variables.
    • \(e_{i}\)’s and \(F_{j}\)’s are uncorrelated

15.2.1 Components of Variance

Recall that \(x_{i}\) is standardized, so \(Var(X)=1\).

Since each response variable \(x_{i}\) is broken into two parts, so is the variance.

  • communality: part due to common factors. Denoted as \(h^{2}_{i}\).
  • specificity: part due to a unique factor. Denoted as \(u^{2}_{i}\).

\[ V(X_{i}) = h^{2}_{i} + u^{2}_{i} \]

If the number \(m\) of common factors is not known (EFA), it is recommended that you start with the default option available in the softare program. Often this is the number of factors with eigenvalues greater than 1.

Since the results are highly dependent on \(m\), you should always try several factors to gain further insight into the data.

15.2.2 Two big steps

  1. The first step is to numerical find estimates of the loadings \(l_{ij}\), and the communalities \(h^{2}_{i}\). This process is called initial factor extraction. There are a number of methods to solve, we will explore three: principal components, iterated components, and maximum likelihood. The mathematical details of each are left in the textbook for interested readers.

  2. The second step is to obtain a new set of factors, called rotated factors which is done to improve interpretation.

We will first explore these steps using simulated data.