📊

ANOVA and Post Hoc Analysis

Jul 25, 2025

Overview

This lecture explains how to interpret results from ANOVA, introduces post hoc procedures to identify specific group differences, and works through an example involving different background sounds and student learning.

ANOVA & Post Hoc Procedures

  • ANOVA identifies if there is an overall association between groups, but not where differences occur.
  • A post hoc procedure is used after rejecting the null hypothesis (finding a statistically significant result) to locate specific group differences.
  • Post hoc means "after this" and is conducted only if the ANOVA test is significant.
  • Examples of post hoc tests include Tukey's HSD and two-sample T-intervals.

Example: Background Sound and Learning

  • A study divided 24 students into three groups (constant sound, unpredictable sound, no sound), each group had 8 students.
  • Each group studied for 30 minutes with their assigned sound condition and then took a 10-point multiple choice test.
  • Variable of interest: test score (quantitative); group assignment (categorical).
  • Null hypothesis: No difference in learning effectiveness between sound groups (all means are equal).
  • Alternative hypothesis: At least one group learns significantly better or worse than others.
  • ANOVA test produces an F-statistic and p-value; in this example, p = 0.0454.
  • Since p < 0.05, the null hypothesis is rejected—at least one group differs significantly.
  • Summary stats suggest constant sound yields a higher mean score than random or no sound.

Interpreting Post Hoc Results

  • Tukey HSD (StatCrunch output) and two-sample intervals compare pairs of group means.
  • An interval containing zero means no significant difference; intervals fully positive or negative indicate a significant difference.
  • In this example, only the comparison between constant sound and no sound shows a significant difference; constant sound group scores are higher.
  • These results are more meaningful due to the controlled nature of the experiment, but further replication is needed.

Key Terms & Definitions

  • ANOVA (Analysis of Variance) — a statistical test comparing means across three or more groups.
  • Post hoc procedure — additional testing done after finding significant ANOVA results to identify which groups differ.
  • Null hypothesis (H₀) — assumption that all group means are equal (no effect).
  • Alternative hypothesis (Hₐ) — at least one group mean differs from the others.
  • Tukey HSD — a common post hoc test for comparing all possible pairs of means.

Action Items / Next Steps

  • Practice interpreting ANOVA and post hoc results using summary statistics and intervals.
  • Consider testing study conditions with different background sounds for personal learning.

Full transcript