15.5 Rotating Factors
- Find new factors that are easier to interpret
- For each \(X\), we want some high/large (near 1) loadings and some low/small (near zero)
- Two common rotation methods: Varimax rotation, and oblique rotation.
Same(ish) goal as PCA, find a new set of axes to represent the factors.
15.5.1 Varimax Rotation
- Restricts the new axes to be orthogonal to each other. (Factors are independent)
- Maximizes the sum of the variances of the squared factor loadings within each factor \(\sum Var(l_{ij}^{2}|F_{j})\)
- Interpretations slightly less clear
Varimax rotation with principal components extraction.
<- principal(stan.dta, nfactors=2, rotate="varimax")
pc.extract.varimax print(pc.extract.varimax)
## Principal Components Analysis
## Call: principal(r = stan.dta, nfactors = 2, rotate = "varimax")
## Standardized loadings (pattern matrix) based upon correlation matrix
## RC1 RC2 h2 u2 com
## X1 0.07 0.94 0.90 0.104 1.0
## X2 0.13 0.94 0.89 0.106 1.0
## X3 0.80 0.02 0.64 0.360 1.0
## X4 0.94 0.11 0.90 0.099 1.0
## X5 0.93 0.23 0.91 0.087 1.1
##
## RC1 RC2
## SS loadings 2.41 1.83
## Proportion Var 0.48 0.37
## Cumulative Var 0.48 0.85
## Proportion Explained 0.57 0.43
## Cumulative Proportion 0.57 1.00
##
## Mean item complexity = 1
## Test of the hypothesis that 2 components are sufficient.
##
## The root mean square of the residuals (RMSR) is 0.08
## with the empirical chi square 12.61 with prob < 0.00038
##
## Fit based upon off diagonal values = 0.97
Varimax rotation with maximum likelihood extraction. Here i’m using the cutoff argument to only show the values of loadings over 0.3.
<- factanal(stan.dta, factors=2, rotation="varimax")
ml.extract.varimax print(ml.extract.varimax, digits=2, cutoff=.3)
##
## Call:
## factanal(x = stan.dta, factors = 2, rotation = "varimax")
##
## Uniquenesses:
## X1 X2 X3 X4 X5
## 0.37 0.00 0.63 0.06 0.04
##
## Loadings:
## Factor1 Factor2
## X1 0.79
## X2 0.99
## X3 0.61
## X4 0.97
## X5 0.96
##
## Factor1 Factor2
## SS loadings 2.26 1.64
## Proportion Var 0.45 0.33
## Cumulative Var 0.45 0.78
##
## Test of the hypothesis that 2 factors are sufficient.
## The chi square statistic is 0.2 on 1 degree of freedom.
## The p-value is 0.652
Communalities are unchanged after varimax (part of variance due to common factors). This will always be the case for orthogonal (perpendicular) rotations.
15.5.2 Oblique rotation
- Same idea as varimax, but drop the orthogonality requirement
- less restrictions allow for greater flexibility
- Factors are still correlated
- Better interpretation
- Methods:
- quartimax or quartimin minimizes the number of factors needed to explain each variable
- direct oblimin standard method, but results in diminished interpretability of factors
- promax is computationally faster than direct oblimin, so good for very large datasets
<- principal(stan.dta, nfactors=2, rotate="quartimin")
pc.extract.quartimin <- factanal(stan.dta, factors=2, rotation="promax") ml.extract.promax
par(mfrow=c(2,3))
plot(pc.extract.norotate, title="PC + norotate")
plot(pc.extract.varimax, title="PC + Varimax")
plot(pc.extract.quartimin, title="PC + quartimin")
<- ml.extract.norotate$loadings[,1:2]
load plot(load, type="n", main="ML + norotate")
text(load, labels=rownames(load))
<- ml.extract.varimax$loadings[,1:2]
load plot(load, type="n", main="ML + Varimax")
text(load, labels=rownames(load))
<- ml.extract.promax$loadings[,1:2]
load plot(load, type="n", main= "ML + Promax")
text(load, labels=rownames(load))
Varimax vs oblique here doesn’t make much of a difference, and typically this is the case. You almost always use some sort of rotation. Recall, this is a hypothetical example and we set up the variables in a distinct two-factor model. So this example will look nice.