Stat Quest: Statistical Power

Introduction

• Presenter: Josh Starburns
• Topic: Statistical Power
• Assumptions: Familiarity with p-values and the normal distribution

Comparing Two Distributions

• Example: Weights of mice on special diet vs. normal diet
• Observation: Little overlap between distributions, easy to see differences
• Small Sample Set:
  • Collect small measurements from both groups
  • Calculate p-value (e.g., 0.0004)
  • Small p-value (< 0.05) → reject null hypothesis

Probability of Correctly Rejecting Null Hypothesis

• High Probability: Repeating experiment shows high chance of small p-value
• Large p-value Scenario:
  • Occasional overlap in data
  • Large p-value (> 0.05) → fail to reject null hypothesis

Defining Power

• Power: Probability of correctly rejecting the null hypothesis
• High Power: Little overlap between distributions → high probability of small p-value

No Difference Between Diets

• Same Distribution: No difference in diets, null hypothesis true → power does not apply

Small Overlap Scenario

• Example: Special diet slightly helps mice lose weight
• Small Sample Size:
  • Measurements from three mice on each diet
  • P-value (e.g., 0.34) → fail to reject null hypothesis
• Low Power: High overlap and small sample size → low probability of small p-value

Increasing Power

• Increasing Measurements: Collect more data to increase power
• Power Analysis: Determines number of measurements needed for sufficient power

Summary

• Power: Probability of correctly rejecting null hypothesis
• High Power: Little overlap and large sample size
• Low Power: High overlap and small sample size
• Power Analysis: Tells required measurements for good power

Closing

• Call to Action: Subscribe, support via Patreon, buy merchandise
• Outro: Quest on