The Assumptions of Simple Linear Regression

The key to creating any statistical model is to verify if the model actually explains the data. In addition to simple visual inspection, residuals provide a pathway for making a rigorous estimate of model accuracy when applying any form of regression. Read this overview of how residuals are applied.

Overview

How do we evaluate a model? How do we know if the model we are using is good? One way to consider these questions is to assess whether the assumptions underlying the simple linear regression model seem reasonable when applied to the dataset in question. Since the assumptions relate to the (population) prediction errors, we do this through the study of the (sample) estimated errors, the residuals.

We focus in this lesson on graphical residual analysis. When we revisit this topic in the context of multiple linear regression in Lesson 7, we'll also study some statistical tests for assessing the assumptions. We'll consider various remedies for when linear regression model assumptions fail throughout the rest of the course, particularly in Lesson 9.

Objectives

Upon completion of this lesson, you should be able to:

  • Understand why we need to check the assumptions of our model.
  • Know the things that can go wrong with the linear regression model.
  • Know how we can detect various problems with the model using a residuals vs. fits plot.
  • Know how we can detect various problems with the model using residuals vs. predictor plots.
  • Know how we can detect a certain kind of dependent error terms using residuals vs. order plots.
  • Know how we can detect non-normal error terms using a normal probability plot.

Source: The Pennsylvania State University, https://online.stat.psu.edu/stat501/lesson/4
Creative Commons License This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 License.