MA121: Introduction to Statistics

The conceptual basis of our testing procedure is that we reject $H_{0}$ only if the data that we obtained would constitute a rare event if $H_{\mathrm{O}}$ were actually true. The level of significance $\alpha$ specifies what is meant by "rare". The observed significance of the test is a measure of how rare the value of the test statistic that we have just observed would be if the null hypothesis were true. That is, the observed significance of the test just performed is the probability that, if the test were repeated with a new sample, the result of the new test would be at least as contrary to $H_{\mathrm{O}}$ and in support of $H_{a}$ as what was observed in the original test.

Definition

The observed significance or $\boldsymbol{p}$-value of a specific test of hypotheses is the probability, on the supposition that $H_{\mathrm{O}}$ is true, of obtaining a result at least as contrary to $H_{\mathrm{O}}$ and in favor of $H_{a}$ as the result actually observed in the sample data.

Think back to Note 8.27 "Example 4- in Section 8.2 "Large Sample Tests for a Population Mean" concerning the effectiveness of a new pain reliever. This was a left-tailed test in which the value of the test statistic was $-1.886$. To be as contrary to $H_{\mathrm{O}}$ and in support of $H_{a}$ as the result $Z=-1.886$ actually observed means to obtain a value of the test statistic in the interval $(-\infty,-1.886]$. Rounding $-1.886$ to $-1.89$, we can read directly from Figure 12.2 "Cumulative Normal Probability" that $P(Z \leq-1.89)=0.0294$. Thus the $p$-value or observed significance of the test in Note 8.27 "Example 4". is 0.0294 or about 3%. Under repeated sampling from this population, if $H_{\mathrm{O}}$ were true then only about $3 \%$ of all samples of size 50 would give a result as contrary to $H_{\mathrm{O}}$ and in favor of $H_{a}$ as the sample we observed. Note that the probability 0.0294 is the area of the left tail cut off by the test statistic in this left-tailed test.

Analogous reasoning applies to a right-tailed or a two-tailed test, except that in the case of a twotailed test being as far from $\mathrm{o}$ as the observed value of the test statistic but on the opposite side of $\mathrm{o}$ is just as contrary to $H_{\mathrm{O}}$ as being the same distance away and on the same side of $\mathrm{o}$, hence the corresponding tail area is doubled.

Computational Definition of the Observed Significance of a Test of Hypotheses

The observed significance of a test of hypotheses is the area of the tail of the distribution cut off by the test statistic (times two in the case of a two-tailed test).

EXAMPLE 6

Compute the observed significance of the test performed in Note 8.28 "Example 5" in Section 8.2 "Large Sample Tests for a Population Mean".

Solution:

The value of the test statistic was $z=2.490$, which by Figure 12.2 "Cumulative Normal Probability" cuts off a tail of area $0.0064$, as shown in Figure 8.7 "Area of the Tail for". Since the test was two-tailed, the observed significance is $2 \times 0.0064=0.0128$.

Figure 8.7

Area of the Tail for Note 8.34 "Example 6"