Inferential Statistics for b and r
Site: | Saylor Academy |
Course: | MA121: Introduction to Statistics |
Book: | Inferential Statistics for b and r |
Printed by: | Guest user |
Date: | Thursday, June 8, 2023, 10:23 AM |
Description
This section starts with assumptions on the errors that are necessary for statistical inference. Then, it gives an example of a significance test for the slope. Finally, it talks about constructing confidence intervals for the slope and closes with a significance test for the correlation.
Inferential Statistics for b and r
Learning Objectives
- State the assumptions that inferential statistics in regression are based upon
- Identify heteroscedasticity in a scatter plot
- Compute the standard error of a slope
- Test a slope for significance
- Construct a confidence interval on a slope
- Test a correlation for significance
This section shows how to conduct significance
tests and compute confidence intervals for the regression slope
and Pearson's correlation. As you will see, if the regression
slope is significantly different from zero, then the correlation
coefficient is also significantly different from zero.
Assumptions
Although no assumptions were needed to determine the best-fitting straight line, assumptions are made in the calculation of inferential statistics. Naturally, these assumptions refer to the population, not the sample.
- Linearity: The relationship between the two variables is linear.
- Homoscedasticity: The variance around the regression line
is the same for all values of
. A clear violation of this assumption is shown in Figure 1. Notice that the predictions for students with high high-school GPAs are very good, whereas the predictions for students with low high-school GPAs are not very good. In other words, the points for students with high high-school GPAs are close to the regression line, whereas the points for low high-school GPA students are not.
Figure 1. University GPA as a function of High School GPA.
- The errors of prediction are distributed normally. This means
that the deviations from the regression line
are normally distributed. It does not mean that
or
is normally distributed.
Significance Test for the Slope (
)
Recall the general formula for a test:
As applied here, the statistic is the sample value
of the slope () and the hypothesized value is 0. The number of degrees
of freedom for this test is:
where is the number of pairs of scores.
The
estimated standard error of is computed using the following
formula:
where is the estimated
standard error of
,
is the standard
error of the estimate, and
is the sum of squared deviations of
from the mean of
.
is calculated as
where
is the mean of
. As shown previously, the standard error of
the estimate can be calculated as
These formulas are illustrated with the data shown
in Table 1. These data are reproduced
from the introductory section. The column
has the values of the predictor variable and
the column
has the values of the criterion variable.
The third column,
, contains the differences between the
values of column
and the mean of
. The fourth
column,
, is the square of
the
column. The fifth column,
, contains
the differences between the values of column
and the mean of
. The last
column,
, is simply square of the
column.
Table 1. Example data.
X | Y | x | x2 | y | y2 | |
---|---|---|---|---|---|---|
1.00 | 1.00 | -2.00 | 4 | -1.06 | 1.1236 | |
2.00 | 2.00 | -1.00 | 1 | -0.06 | 0.0036 | |
3.00 | 1.30 | 0.00 | 0 | -0.76 | 0.5776 | |
4.00 | 3.75 | 1.00 | 1 | 1.69 | 2.8561 | |
5.00 | 2.25 | 2.00 | 4 | 0.19 | 0.0361 | |
Sum | 15.00 | 10.30 | 0.00 | 10.00 | 0.00 | 4.5970 |
The computation of the standard
error of the estimate () for these
data is shown in the section on the standard error of the estimate. It
is equal to 0.964.
is the sum of squared deviations from the mean
of
. It is, therefore, equal to the sum of the
column and is
equal to 10.
We now have all the information to compute the
standard error of :
As shown previously, the slope () is 0.425. Therefore,
The value for a two-tailed
test is 0.26. Therefore,
the slope is not significantly different from 0.
Confidence Interval for the Slope
The method for computing a confidence interval for the population slope is very similar to methods for computing other confidence intervals. For the 95% confidence interval, the formula is:
where is the value of
to use for the
95% confidence interval.
The values of to be used in a confidence interval
can be looked up in a table of the t distribution. A small version
of such a table is shown in Table 2. The first column,
, stands
for degrees of freedom.
Table 2. Abbreviated t table.
df | 0.95 | 0.99 |
---|---|---|
2 | 4.303 | 9.925 |
3 | 3.182 | 5.841 |
4 | 2.776 | 4.604 |
5 | 2.571 | 4.032 |
8 | 2.306 | 3.355 |
10 | 2.228 | 3.169 |
20 | 2.086 | 2.845 |
50 | 2.009 | 2.678 |
100 | 1.984 | 2.626 |
You can also use the "inverse
distribution" calculator to find the t values to
use in a confidence interval.
Applying these formulas to the example data,
Significance Test for the Correlation
The formula for a significance test of Pearson's correlation is shown below:
where is the number of pairs of scores. For
the example data,
Notice that this is the same value obtained
in the
test of
. As in that test, the degrees of freedom is
.
Source: David M. Lane , https://onlinestatbook.com/2/regression/inferential.html This work is in the Public Domain.
Video
Questions
Question 1 out of 4.
Which of the following are assumptions made in the calculation of regression inferential statistics?
The errors of prediction are normally distributed.
X is normally distributed.
Y is normally distributed.
The variance around the regression line is the same for all values of X.
The relationship between X and Y is linear.
Question 2 out of 4.
The slope of a regression line is 0.8, and the standard error of the
slope is 0.3. The sample used to compute this regression line consisted
of 12 participants. Compute the 95% confidence interval for the slope.
Type the upper limit of the confidence interval in the box below.
Question 3 out of 4.
In a sample of 20, the correlation between two variables is .5.
Determine if this correlation is significant at the .05 level by
calculating the t value.
Question 4 out of 4.
Calculate the lower limit of the 95% confidence interval for the correlation of .75 (N = 25).
Answers
- The assumptions are linearity, homoscedasticity, and normally distributed errors. See the text for more information.
- Use the table in this section or the inverse t distribution calculator
to find that the critical value is t(N-2) = t(10) = 2.23. The upper
limit of the 95% CI = b + (t)(sb) = .8 + 2.23(.3) = 1.47.
- t = (r) sqrt(N-2)/sqrt(1-r2) = (0.5) sqrt(18)/sqrt(1-.25) = 2.45 (This is significant at the .05 level.)
- First, convert r to z' (so .75 -> .973). The standard error of z' is 1/sqrt(N-3) = .213. Lower limit of CI = .973 - 1.96(.213) = 0.556. Now convert back from z' to r. r = .505