Overview
This lecture explains the purpose of hypothesis testing, the meaning of statistical significance, and how increasing sample size impacts the likelihood of detecting significant results.
Purpose of Hypothesis Testing
- Hypothesis testing aims to make decisions about populations by evaluating whether to reject the null hypothesis in favor of the alternative.
- Statistical power is the probability of correctly rejecting the null when the alternative is true.
- A statistically significant result indicates sufficient evidence against the null hypothesis.
Statistical Significance and P-values
- Statistical significance means the result is unlikely due to random chance, based on a chosen significance level (usually 0.05).
- If the P-value is less than the significance level, we reject the null hypothesis and declare significance.
- If the P-value is greater, we fail to reject the null and state there is not enough evidence.
Coin Flip Example and Sample Size Impact
- Null hypothesis: coin lands heads or tails 50/50; alternative: probability is higher for starting face (51%).
- With 700 flips, P-value = 0.298; since 0.298 > 0.05, we fail to reject the null hypothesis, deeming the result not significant.
- With 7,000 flips, P-value = 0.471; since 0.471 < 0.05, we reject the null hypothesis, result is significant (Note: This seems like a lecture slip; normally, 0.471 > 0.05, but the main lesson focuses on larger sample size leading to lower P-values and significance).
- Increasing sample size decreases P-value, making it easier to detect statistical significance.
Importance of Sample Size
- A larger sample size enhances the test's power and likelihood of finding significant results.
- Regardless of method, using a large enough sample increases the chance of rejecting the null hypothesis when appropriate.
Key Terms & Definitions
- Null Hypothesis (H₀) — The default assumption, e.g., no difference or effect (coin is 50/50).
- Alternative Hypothesis (H₁) — The claim that challenges the null (coin is more likely to land on starting face).
- P-value — Probability of obtaining results as extreme as observed, under the null hypothesis.
- Significance Level (α) — Threshold to declare significance, commonly 0.05.
- Statistical Significance — Determination that observed results are unlikely due to random variation.
- Statistical Power — Probability of correctly rejecting a false null hypothesis.
Action Items / Next Steps
- Practice calculating P-values and interpreting their relationship to significance level.
- Review how sample size affects statistical power and significance in hypothesis tests.