Regression Basics

If the estimated regression line fits well into the data, the model can then be used for predictions. Using the above estimated simple regression model, we can predict the price of an apartment a given distance to downtown. This is known as the prediction interval or P.I. Alternatively, we may predict the mean price of the apartment, also known as the confidence interval or C.I., for the mean value.

In predicting intervals for the price of an apartment that is six kilometres away from downtown, we simply set $x=6$ , and substitute it back into the estimated equation:

$y=71.84-5.38 \times 6=\$ 39.56$

You should pay attention to the scale of data. In this case, the dependent variable is measured in $1000s. Therefore, the predicted value for an apartment six kilometres from downtown is 39.56*1000=$39,560. This value is known as the point estimate of the prediction and is not reliable, as we are not clear how close this value is to the true value of the population.

A more reliable estimate can be constructed by setting up an interval around the point estimate. This can be done in two ways. We can predict the particular value of $y$ for a given value of x, or we can estimate the expected value (mean) of $y$, for a given value of $x$. For the particular value of $y$, we use the following formula for the interval:

$Y \pm t \alpha / 2, n-2 * S . E . \text { of the prediction }$

where the standard error, S.E., of the prediction is calculated based on the following formula:

$\text { S. E. of the prediction }=s \sqrt{1+\frac{1}{n}+\frac{\left(x^{*}-\bar{x}\right)}{\sum(x-\bar{x})^{\wedge} 2}}$

In this equation, x^* is the particular value of the independent variable, which in our case is 6, and sis the standard error of the regression, calculated as:

$s=\sqrt{\frac{S S E}{n-1}}$

From the Excel printout for the simple regression model, this standard error is estimated as 7.02.

The sum of squares of the independent variable,

$\sum_{i=1}^{12}(x-\bar{x})^{\wedge} 2$

can also be calculated as shown in Figure 8.9.

Figure 8.9

All these calculated values can be substituted back into the formula for the S.E. of the prediction:

$\text { S.E. of C.I. }=7.02 \sqrt{\frac{1}{12}+\frac{(6-2.325)^{2}}{17.3275}}=6.52$

Now that the S.E. of the confidence interval has been calculated, you can pick up the cut-off point from the t-table. Given the degrees of freedom 12-2=10, the appropriate value from the t-table is 2.23. You use this information to calculate the margin of error as 6.52*2.23=14.54. Finally, construct the prediction interval for the particular value of the price of an apartment located six kilometres away from downtown as:

$39.56 \mp 14.54$

This is a compact version of the prediction interval. For a more general version of any confidence interval for any given confidence level of alpha, we can write:

$\text { P[Point Estimate }-\text { M.E. < population value < Point Estimate }+M . E .]=1-\alpha$

Intuitively, for say a .05 level of confidence, we are 95% confident that the true parameter of the population will be within these two lower and upper limits:

$P[39.56-14.54 \text { < True Population Value < } 39.56+14.54]=0.95$

Based on our simple regression model that only includes distance as a significant factor in predicting the price of an apartment, and for a particular apartment six kilometres away from downtown, we are 95% confident that the true price of an apartments in Nelson, BC, is between $25,037 and $54,096, with a width of $29,059. One should not be surprised there is such a wide width, given the fact that the coefficient of determination of this model was only 50%, and the fact that we have selected a distance far away from the mean distance from downtown. We can always improve these numbers by adding more explanatory variables to our simple regression model. Alternatively, we can predict only for the numbers as much as possible close to the downtown area.

Now we estimate the expected value (mean) of $y$ for a given value of $x$, the so-called prediction interval. The process of constructing intervals is very similar to the previous case, except we use a new formula for S.E. and of course we set up the intervals for the mean value of the apartment price (i.e., =59.33).

$\text { S.E. of P.I. }=7.02 \sqrt{1+\frac{1}{12}+\frac{(6-2.325)^{2}}{17.3275}}=9.58$

You should be very careful to note the difference between this formula and the one introduced earlier for S.E. for predicting the particular value of $y$ for a given value of $x$.They look very similar but this formula comes with an extra 1 inside the radical!

The margin of error is then calculated as 2.179*3.82=8.32. We use this to set up directly the lower and upper limits of the estimates:

$P[39.56-21.36 \text { < True Population Value < } 39.56+21.36]=0.95$

Thus, for the average price of apartments located in Nelson, BC, six kilometres away from downtown, we are 95% confident that this average price will be between $18,200 and $60,920, with a width of $47,720. Compared with the earlier width for C.I., it is obvious that we are less confident in predicting the average price. The reason is that the S.E. for the prediction is always larger than the S.E. for the confidence interval.

This process can be repeated for all different levels of x, to calculate the associated confidence and prediction intervals. By doing this, we will have a range of lower and upper levels for both P.I.s and C.I.s. All these numbers can be reproduced within the interactive Excel template shown in Figure 8.8. If you use a statistical software such as Minitab, you will directly plot a scatter diagram with all P.I.s and C.I.s as well as the estimated linear regression line all in one diagram. Figure 8.10 shows such a diagram from Minitab for our example.

Figure 8.10 Minitab Plot for C.I. and P.I.

Figure 8.10 indicates that a more reliable prediction should be made as close as possible to the mean of our observations for $x$. In this graph, the widths of both intervals are at the lowest levels closer to the means of $x$ and $y$.

You should be careful to note that Figure 8.10 provides the predicted intervals only for the case of a simple regression model. For the multiple regression model, you may use other statistical software packages, such as SAS, SPSS, etc., to estimate both P.I. and C.I. For instance, by selecting $x1=3$, and $x2=300$, and coding these figures into Minitab, you will see the results as shown in Figure 8.11. Alternatively, you may use the interactive Excel template provided in Figure 8.8 to estimate your multiple regression model, and to check for the significance of the estimated parameters. This template can also be used to construct both the P.I. and C.I. for the given values of $x1=3$, and $x2=300$ or any other values of your choice. Furthermore, this template enables you to test if the estimated multiple regression model is overall significant. When the estimated multiple regression model is not overall significant, this template will not provide the P.I. and C.I. To practice this case, you may want to change the yellow columns of $x1$ and $x2$ with different random numbers that are not correlated with the dependent variable. Once the estimated model is not overall significant, no prediction values will be provided.

Figure 8.11

We have just given you some rough ideas about how the basic regression calculations are done. We left out other steps needed to calculate more detailed results of regression without a computer on purpose, for you will never compute a regression without a computer (or a high-end calculator) in all of your working years. However, by working with these interactive templates, you will have a much better chance to play around with any data to see how the outcomes can be altered, and to observe their implications for the real-world business decision-making process.