7.3 Least Squares Regression

The Least Squares Method finds the estimates for the intercept \(b_{0}\) and slope \(b_{1}\) that minimize the SSE. Let’s see how that works:

See https://paternogbc.shinyapps.io/SS_regression/

Initial Setup

  • Set the sample size to 50
  • Set the regression slope to 1
  • Set the standard deviation to 5

Partitioning the Variance using the Sum of Squares

  • SS Total- how far are the points away from \(\bar{y}\)? (one sample mean)
  • SS Regression - how far away is the regression line from \(\bar{y}\)?.
  • SS Error - how far are the points away from the estimated regression line?

Looking at it this way, we are asking "If I know the value of \(x\), how much better will I be at predicting \(y\) than if I were just to use \(\bar{y}\)?

Increase the standard deviation to 30. What happens to SSReg? What about SSE?

Here is a link to another interactive app where you can try to fit your own line to minimize the SSE.

RMSE is the Root Mean Squared Error. In the PMA textbook this is denoted as \(S\), which is an estimate for \(\sigma\).

\[ S = \sqrt{\frac{SSE}{N-2}}\]