Interpretable vs. Explainable Machine Learning

Introduction

• Field of Interpretable Machine Learning (IML) is still developing.
• Ongoing debates about terminology: IML vs. Explainable AI (XAI).
• Aim to clarify definitions and differences between interpretable and explainable models.

Definitions

• Interpretable Model:

  • Capable of being understood by humans without additional methods.
  • Example Models:
    • Decision Trees: Easy to traverse and understand predictions.
    • Linear Regression: Clear relationships between parameters (e.g., income and age in predicting loan size).

• Explainable Model:

  • Too complex for humans to understand directly (often referred to as "black box" models).
  • Requires additional techniques to interpret.
  • Example Models:
    • Random Forests: Comprised of multiple decision trees, making it difficult to understand.
    • Neural Networks (e.g., AlexNet): Complex with millions of parameters, not interpretable through weights alone.

Techniques for Understanding Explainable Models

• Model-Specific Techniques:

  • E.g., DeepLift for neural networks.

• Model-Agnostic Techniques:

  • Can be applied to any model.
  • Examples:
    • LIME
    • SHAP
    • Partial Dependence Plots (PDPs)
    • Individual Conditional Expectation (ICE) plots
  • These techniques provide approximations and come with their own assumptions and limitations.

Interpretability Spectrum

• Interpretability exists on a spectrum rather than a binary classification.
• Example of Interpretability Levels:
  • Convolutional Neural Networks (CNNs) < Random Forest < Decision Tree
• Disagreement on where to classify models can arise.
  • E.g., Random Forest with 2 trees may be interpretable, but one with 100 may not.

Challenges with Current Definitions

• Human comprehension varies based on technical skills and experience.
• No formal way to measure interpretability.
• Use of additional techniques complicates classification:
  • E.g., using a correlation matrix for linear regression raises questions about its interpretability status.

Conclusion

• Goal of IML is to understand and explain models, regardless of classification.
• Choice of methods depends on model type and questions being addressed.
• There's no consensus on definitions; open to opinions on better ways to classify models.

Additional Resources

• For those interested in IML:
  • Python SHAP course available.
  • Free access for newsletter sign-ups.