Linear Regression

At various points throughout the course, it will be necessary to build a statistical model that can be used to estimate or forecast a value based on a set of given values. When one has data where a dependent variable is assumed to depend upon a set of independent variables, linear regression is often applied as the first step in data analysis. This is because parameters for the linear model are very tractable and easily interpreted. You will be implementing this model in various ways using Python.

Prediction (Optional)

Recall the third exam/final exam example (Example 12.5).

We found the equation of the best-fit line for the final exam grade as a function of the grade on the third exam. We can now use the least-squares regression line for prediction.

Suppose you want to estimate, or predict, the mean final exam score of statistics students who received a 73 on the third exam. The exam scores (x values) range from 65 to 75. Since 73 is between the x values 65 and 75, substitute x = 73 into the equation. Then,

\widehat y =−173.51+4.83(73)=179.08 .

We predict that statistics students who earn a grade of 73 on the third exam will earn a grade of 179.08 on the final exam, on average.


Example 12.10

Recall the third exam/final exam example.

a. What would you predict the final exam score to be for a student who scored a 66 on the third exam?

Solution 1
a. 145.27

b. What would you predict the final exam score to be for a student who scored a 90 on the third exam?

Solution 2
b. The x values in the data are between 65 and 75. 90 is outside the domain of the observed x values in the data (independent variable), so you cannot reliably predict the final exam score for this student. Even though it is possible to enter 90 into the equation for x and calculate a corresponding y value, the y value that you get will not be reliable.

To understand how unreliable the prediction can be outside the x values observed in the data, make the substitution x = 90 into the equation:

ŷ=–173.51+4.83(90)=261.19.

The final exam score is predicted to be 261.19. The most points that can be awarded for the final exam are 200.


Try It 12.10

Data are collected on the relationship between the number of hours per week practicing a musical instrument and scores on a math test. The line of best fit is as follows:

ŷ = 72.5 + 2.8x.

What would you predict the score on a math test will be for a student who practices a musical instrument for five hours a week?