Overview
This lecture explores why people are often poor at predicting the future, the flaws in expert forecasting, and how "superforecasting" methods can dramatically improve predictive accuracy through specific habits, training, and feedback.
Why Prediction Fails
- Many experts lack accountability and have no incentives to be accurate with their public predictions.
- Overconfidence and dogmatism—the reluctance to change one's mind—are common traits in poor forecasters.
- Vague language in forecasts ("fair chance") leads to misunderstanding and poor decisions.
- Most people underestimate how much forecasting is embedded in everyday decisions.
The Good Judgment Project & Forecasting Tournaments
- The Good Judgment Project, led by Philip Tetlock, participated in a major forecasting tournament run by IARPA.
- Questions in tournaments were about real, complex geopolitical issues, and thousands of volunteers could participate.
- Crowdsourcing and combining diverse perspectives often beats individual expert predictions.
- Training, teaming up, and frequent feedback improved forecasting accuracy considerably.
Characteristics of Superforecasters
- Superforecasters are humble, open-minded, and ready to update their beliefs with new evidence.
- They think probabilistically, use numerical estimates, and avoid vague statements.
- They balance inside knowledge (details about the case) with outside knowledge (base rates or statistical patterns).
- Above-average intelligence and curiosity help, but attitude and methods are more important than expertise.
Methods for Better Prediction
- Break complex questions into parts and seek opposing causal explanations.
- Start estimates with base rates and adjust with case-specific information.
- Regularly recalibrate and learn from mistakes with honest feedback.
- Use teamwork and share reasoning to improve collective accuracy.
The Ten Commandments for Aspiring Superforecasters
- Focus effort where it matters, break problems down, balance inside/outside views, and adjust carefully to evidence.
- Learn from errors, be neither overconfident nor indecisive, refine your uncertainty, collaborate, and learn by doing.
Real-World Impact and Applications
- Superforecasting can outperform traditional intelligence analysis even without special access to information.
- Making prediction accuracy public and measurable could improve public debate and decision-making in society.
Key Terms & Definitions
- Forecasting — Making predictions about future events based on evidence and reasoning.
- Superforecaster — A person who consistently makes more accurate predictions by following certain habits and methods.
- Base Rate — The general statistical frequency of an event, used as a starting point for predictions.
- Vague Verbiage — Non-numerical language in predictions that leads to ambiguity.
- Crowdsourcing — Combining input from many people to improve prediction accuracy.
Action Items / Next Steps
- Practice making explicit, numerical predictions and tracking their accuracy.
- Study the "Ten Commandments" for superforecasting and try applying them to real-world questions.
- Read "Superforecasting" by Philip Tetlock and Dan Gardner for detailed methods.