Estimation and Psychometrics in Software Development

Jan 23, 2025

Lecture on Estimation and Psychometrics

Introduction

  • Presenter: Joseph, a retired psychologist with expertise in IOB methods and process automation.
  • Background: Nearly 30 years in psychology, helping companies improve customer satisfaction.
  • Focus: Applying psychology to agile methods and estimation.

Key Themes

  • Estimation in Software Development: Common frustration with exact estimates from managers.
  • Expertise and Estimation: Experts may become myopic and conservative.
  • Estimation Across Fields: Economists, financial advisors, meteorologists, etc., all engage in estimation.

Estimation Challenges

  • Echo Chamber in Software Development: Developers often find themselves in a feedback loop, leading to a 'cult' of estimation.
  • Psychometrics as a Solution: Science of estimating through measuring latent constructs.

Main Topics Covered

The Problem of Estimation

  • Kobashi Maru Scenario: Estimation likened to a no-win situation.
  • Political Consequences: Political impact of estimating in time can be problematic.

Estimation Fallacies

  1. Strategic Misrepresentation: Deliberate under/overestimation.
  2. Relativity Fallacy: Perception of time varies with circumstances.
  3. Planning Fallacy: Over-detailing leads to underestimation.
  4. Completeness Fallacy: Lack of full information affects estimates.
  5. Distribution Fallacy: Estimates are log-normal, not Gaussian.
  6. Groundhog Fallacy: Repetitive cycles in estimation lead to predictability issues.
  7. Objectivity Fallacy: Cognitive biases affect estimation.

Psychometric Approach to Estimation

  • Equation for Estimation: True Value + Systemic Error + Random Error.
  • Factors Affecting Estimates: Work, people, tools, and load factors.

Techniques for Better Estimation

Complex Numbers in Estimation

  • Real vs. Imaginary Parts: Real part is the number; imaginary part is the anxiety.
  • Rationalizing Anxiety: Convert anxiety into risk.
  • Quantifying Risk: Use a Likert scale for comfort level.

Reference Class Forecasting

  • Planning Fallacy Solution: Use historical data to predict outcomes.

Unpacking

  • Task Breakdown: Avoid the 'oh' effect by detailing steps.

Bayesian Inferencing

  • Statistical Probability Distribution: Provides a range instead of a single estimate.

Final Thoughts

  • Estimates Have a Shelf Life: Estimates and task breakdowns need regular review.

Q&A Highlights

  • Origin of Story Points: Discussion of the creation of story points from "ideal weeks" and gummy bears.
  • Relevance of "The Mythical Man-Month": Still holds valuable insights despite age.
  • Communicating with Project Managers: Challenges with shifting risk responsibility to developers.

These notes capture the essence of Joseph's lecture on estimation, focusing on psychological and psychometric principles to improve accuracy and understanding in software development.