Steps and Confidence Intervals in Hypothesis Testing

Site: Saylor Academy
Course: MA121: Introduction to Statistics
Book: Steps and Confidence Intervals in Hypothesis Testing
Printed by: Guest user
Date: Thursday, May 23, 2024, 6:11 PM

Description

This section lists four key steps in hypothesis testing and explains the close relationship between confidence intervals and significance tests.

Steps in Hypothesis Testing

Learning Objectives

  1. Be able to state the null hypothesis for both one-tailed and two-tailed tests
  2. Differentiate between a significance level and a probability level
  3. State the four steps involved in significance testing

  1. The first step is to specify the null hypothesis. For a two-tailed test, the null hypothesis is typically that a parameter equals zero although there are exceptions. A typical null hypothesis is \mu_{1}-\mu_{2}=0 which is equivalent to \mu_{1} =\mu_{2}. For a one-tailed test, the null hypothesis is either that a parameter is greater than or equal to zero or that a parameter is less than or equal to zero. If the prediction is that \mu_{1} is larger than \mu_{2}, then the null hypothesis (the reverse of the prediction) is \mu_{2}-\mu_{1} \geq 0. This is equivalent to \mu_{1} \leq \mu_{2}.
  2. The second step is to specify the α level which is also known as the significance level. Typical values are 0.05 and 0.01.
  3. The third step is to compute the probability value (also known as the p value). This is the probability of obtaining a sample statistic as different or more different from the parameter specified in the null hypothesis given that the null hypothesis is true.
  4. Finally, compare the probability value with the α level. If the probability value is lower then you reject the null hypothesis. Keep in mind that rejecting the null hypothesis is not an all-or-none decision. The lower the probability value, the more confidence you can have that the null hypothesis is false. However, if your probability value is higher than the conventional α level of 0.05, most scientists will consider your findings inconclusive. Failure to reject the null hypothesis does not constitute support for the null hypothesis. It just means you do not have sufficiently strong data to reject it.


Source: David M. Lane, https://onlinestatbook.com/2/logic_of_hypothesis_testing/steps.html
Public Domain Mark This work is in the Public Domain.

Video

 

 

Questions

Question 1 out of 2.

True/false: First you decide on the null hypothesis. Then you analyze the data and calculate the probability value. You look at this probability value, and depending on what it is, you then choose an appropriate alpha level. Then you decide whether you can reject the null hypothesis.

  • True
  • False


Question 2 out of 2.

True/false: The goal of research is to prove that the null hypothesis is true.

Answers


  1. You want to select the alpha level before you calculate the probability value. You compare your probability value to your previously-selected alpha level when deciding whether or not you can reject the null hypothesis.

  2. Researchers generally specify a null hypothesis that is the opposite of what they are predicting. Getting a p value lower than the alpha level allows you to reject the null hypothesis, but getting a p value greater than the alpha level does not prove that the null hypothesis is true.

Significance Testing and Confidence Intervals

Learning Objectives

  1. Determine from a confidence interval whether a test is significant
  2. Explain why a confidence interval makes clear that one should not accept the null hypothesis

There is a close relationship between confidence intervals and significance tests. Specifically, if a statistic is significantly different from 0 at the 0.05 level, then the 95% confidence interval will not contain 0. All values in the confidence interval are plausible values for the parameter, whereas values outside the interval are rejected as plausible values for the parameter. In the Physicians' Reactions case study, the 95% confidence interval for the difference between means extends from 2.00 to 11.26. Therefore, any value lower than 2.00 or higher than 11.26 is rejected as a plausible value for the population difference between means. Since zero is lower than 2.00, it is rejected as a plausible value and a test of the null hypothesis that there is no difference between means is significant. It turns out that the p value is 0.0057. There is a similar relationship between the 99% confidence interval and significance at the 0.01 level.

Whenever an effect is significant, all values in the confidence interval will be on the same side of zero (either all positive or all negative). Therefore, a significant finding allows the researcher to specify the direction of the effect. There are many situations in which it is very unlikely two conditions will have exactly the same population means. For example, it is practically impossible that aspirin and acetaminophen provide exactly the same degree of pain relief. Therefore, even before an experiment comparing their effectiveness is conducted, the researcher knows that the null hypothesis of exactly no difference is false. However, the researcher does not know which drug offers more relief. If a test of the difference is significant, then the direction of the difference is established because the values in the confidence interval are either all positive or all negative.

If the 95% confidence interval contains zero (more precisely, the parameter value specified in the null hypothesis), then the effect will not be significant at the 0.05 level. Looking at non-significant effects in terms of confidence intervals makes clear why the null hypothesis should not be accepted when it is not rejected: Every value in the confidence interval is a plausible value of the parameter. Since zero is in the interval, it cannot be rejected. However, there is an infinite number of other values in the interval (assuming continuous measurement), and none of them can be rejected either.

Video

 

 

Questions

Question 1 out of 4.

The null hypothesis for a particular experiment is that the mean test score is 20. If the 99% confidence interval is (18, 24), can you reject the null hypothesis at the .01 level?

  • Yes
  • No


Question 2 out of 4.

Select all that apply. Which of these 95% confidence intervals for the difference between means represent a significant difference at the .05 level?

  • (-4.6, -1.8)
  • (-0.2, 8.1)
  • (-5.1, 6.7)
  • (3.0, 10.9)


Question 3 out of 4.

If a 95% confidence interval contains 0, so will the 99% confidence interval.

  • True
  • False


Question 4 out of 4.

Select all that apply. A person is testing whether a coin that a magician uses is biased. After analyzing the results from his coin flipping, the p value ends up being .21, so he concludes that there is no evidence that the coin is biased. Based on this information, which of these is/are possible 95% confidence intervals on the population proportion of times heads comes up?

  • (.43, .55)
  • (.32, .46)
  • (.48, .64)
  • (.76, .98)
  • (.81, 1.33)

Answers


  1. You cannot reject the null hypothesis because the confidence interval shows that 20 is a plausible value of the population parameter.

  2. This study is testing the difference between means, and significant differences would be either larger or smaller than 0. Thus, confidence intervals that do not contain 0 represent statistically significant findings.

  3. The 99% confidence interval contains all of the values that the 95% confidence interval has, but it extends farther at both ends and has other values, too. If something is not significant at the .05 level, it is also non-significant at the .01 level.

  4. Because the p value was .21, we know that the 95% confidence interval contains the null hypothesis parameter, .5. Thus, both of the confidence intervals that contain .5 are possible confidence intervals that this researcher could have computed.