Understanding External Validity and Generalizability

Sep 10, 2024

External Validity and Generalizability

External validity refers to whether the hypothesized relation holds across different persons, settings, and times.

Threats to External Validity

  1. History Threat

    • The observed effect may not generalize to other time periods.
    • Example: Compliance study in the 1950s showed extreme willingness to comply with unethical authority.
      • Results may differ today due to increased education and reduced sensitivity to authority.

  2. Setting Threat

    • The observed effect is specific to a particular setting and does not generalize.
    • Example: Relation between violent imagery and aggression in children.
      • Aggression observed on the playground may not occur at home under caregiver supervision.

    Artificiality of Research Setting

    • Two specific setting threats: Pretesting and Reactivity.

    • Pretesting Threat

      • Effect observed only when a pretest is performed.
      • Example: Depression treatment study with a pretest that increases participant receptiveness.
        • Threatens both internal and external validity as it relies on the pretest for effectiveness.

    • Reactivity Threat

      • Participants or experimenters react to their awareness of being part of a study.
      • Examples include participant expectancy and altered behavior due to nervousness.
        • Example: Math teaching method effectiveness may increase due to students knowing they are being studied.

  3. Selection Threat

    • Hypothesized relation holds only for a specific subset of people.
    • Results may be biased due to over or under-representation of certain groups.
    • Example: Study on depression therapy with volunteers may lead to overestimation of effectiveness, as volunteers are more proactive.
    • Example: Opinion on women's right to vote gathered from a university campus may not generalize to the broader public.

Reducing Threats to External Validity

  1. Replicating Studies
    • Conduct studies in different time periods or settings to verify generalizability.
    • Repeat studies in more natural environments to reduce artificiality threats.
  2. Random Sampling
    • Use probability sampling to ensure diverse representation in the research sample, reducing selection threats.