Overview
This lecture explains the concept of significance level in hypothesis testing, its role in measuring error, and how to interpret it within real-world and statistical contexts.
Statistical Inference & Error
- Statistical inference involves estimating unknown values, but always includes some error.
- Confidence intervals express this uncertainty by providing a range with a certain level of confidence.
Significance Level in Hypothesis Testing
- The significance level (alpha) is the probability of making a mistake in hypothesis testing.
- Specifically, it measures the risk of rejecting the null hypothesis when it is actually true.
- In court system analogy, the null hypothesis represents innocence.
- Rejecting a true null (convicting an innocent person) is considered the worst error in this context.
- The significance level quantifies the chance of making this error and should be kept as low as possible.
Choosing and Interpreting Significance Levels
- Standard significance level is typically 5% (alpha = 0.05), but it can be made lower for situations with serious consequences.
- Less serious consequences may allow for a higher significance level.
- The significance level is always a small percentage, such as 5% or 10%.
- Interpretation template: "There is a [significance level]% chance of concluding the alternative hypothesis is true when there is actually no difference."
Examples of Interpretation
- For alpha = 0.05: "There is a 5% chance of concluding West Valley's math success rate is higher than statewide, when actually there is no difference."
- For alpha = 0.10: "There is a 10% chance of concluding that the percent of concerned parents is different from 1994, when in fact there is no difference."
Key Terms & Definitions
- Significance Level (alpha) — The probability of rejecting the null hypothesis when it is true.
- Null Hypothesis (H₀) — The default assumption, typically representing no effect or no difference.
- Alternative Hypothesis (H₁) — The statement we seek evidence for, indicating a difference or effect.
- Type I Error — Rejecting the null hypothesis when it is actually true (controlled by the significance level).
Action Items / Next Steps
- Practice writing interpretations of significance levels using the provided template for current and future hypothesis tests.