First, this section discusses whether rejection of the null hypothesis should be an all-or-none proposition. Then, it discusses how to interpret non-significant results; for example, it explains why the null hypothesis should not be accepted or should be accepted with caution. It also describes how a non-significant result can increase confidence that the null hypothesis is false.
Interpreting Significant Results
Question 1 out of 3.
Which of the following probability values gives you the most confidence that the null hypothesis is false?
Question 2 out of 3.
You are testing the difference between high school freshmen and seniors on SAT performance. The null hypothesis is that the population mean SAT score of the seniors is equal to the population mean SAT score of the freshmen. You randomly sample 20 students in each grade and have them take the SAT. You find that the sample mean of the seniors is significantly higher than the sample mean of the freshmen. Which alternative hypothesis is accepted?
- The population mean SAT score of the seniors is less than the population mean SAT score of the freshmen.
- The population mean SAT score of the seniors is greater than the population mean SAT score of the freshmen.
- You cannot be sure which alternative hypothesis to accept. You just know that the null hypothesis was rejected.
Question 3 out of 3.
If you are already certain that a null hypothesis is false, then:
- Significance testing provides no useful information since all it does is reject a null hypothesis.
- Significance testing is informative because you still need to know whether an effect is significant even if you know the null hypothesis is false.
- When a difference is significant you can draw a confident conclusion about the direction of the effect.