Bayes' Theorem in Medical Testing

Overview

This transcript explains Bayes' Theorem through a medical testing example, its historical origins, iterative updating, applications like spam filtering, and reflections on priors, certainty, and experimentation.

Medical Testing Example and Bayes' Theorem

• Scenario: Rare disease affects 0.1% of population; test sensitivity 99%, false positive rate 1%.
• Common mistake: Equating test accuracy with probability of actually having the disease.
• Bayes' Theorem updates belief after observing a positive test result.

Disease Test Numbers

• Prior probability of disease: 0.1% before testing.
• Sensitivity (P(positive | disease)): 99%.
• False positive rate (P(positive | no disease)): 1%.
• Posterior after one positive: ~9% chance of actually having the disease.
• Intuition with 1000 people: ~1 true positive and ~10 false positives → 1 in 11 ≈ 9%.

Two Independent Positive Tests

• Use posterior from first test (9%) as new prior for second independent test.
• Result after two positives: ~91% chance of having the disease.
• Note: Still below single-test reported accuracy; independence assumed by using different labs.

Bayes' Theorem: Concept and Use

• Purpose: Probability of a hypothesis given evidence, by updating prior beliefs.
• Components:
  • Prior: Belief before evidence (often hard to set).
  • Likelihood: Probability of evidence if hypothesis is true.
  • Evidence probability: Total probability of observing the evidence.
• Designed for repeated use: Update with each new piece of evidence.

Structured Summary of the Medical Example

Historical Context and Thought Experiments

• Bayes did not publish his theorem; found posthumously by Richard Price.
• Price discovered the work while reviewing Bayes' papers at relatives’ request.
• Bayes’ table and balls thought experiment: Iteratively narrows location by accumulating relative position reports.
• Richard Price’s analogy: A man leaving a cave gains confidence as the Sun rises each day.

Applications

• Spam filtering: Evaluate probability an email is spam given the presence of certain words.
• General insight: Update beliefs with evidence; accuracy improved with iterative data.

Priors, Certainty, and Debate

• Bayes' Theorem cannot set priors; people may hold 0% or 100% priors.
• With extreme priors, no amount of evidence changes beliefs.
• Cited perspective: Debates between 0% and 100% priors are futile; no convergence.

Practical Reflections on Learning and Action

• Concern: People may over-internalize past outcomes and become overly certain about immutability.
• Mandela quote invoked: Everything seems impossible until done; priors can be zero until evidence appears.
• Actions influence outcomes; beliefs can become self-fulfilling through repeated behavior.
• Implication: Experimentation is essential to change outcomes and update beliefs.

Key Terms & Definitions

• Prior probability: Belief in a hypothesis before observing new evidence.
• Likelihood: Probability of observed evidence if the hypothesis is true.
• Posterior probability: Updated probability of the hypothesis after incorporating evidence.
• Sensitivity: Probability a test correctly identifies a true condition (true positive rate).
• False positive rate: Probability a test incorrectly signals a condition when absent.

Action Items / Next Steps

• When faced with test results, consider base rates and false positives using Bayes' Theorem.
• Seek independent repeat tests to refine posterior probabilities.
• Avoid absolute priors; remain open to belief updates with new evidence.
• If stuck in repeating outcomes, design experiments to change actions and gather new evidence.