Risk Extrapolation (Rex) and Invariant Risk Minimization (IRM)

Introduction

• Rex (Risk Extrapolation) and IRM (Invariant Risk Minimization) are methods for causal discovery using deep learning
• Inspired by the work of Arjhovsky et al.

Core Idea

• Causal Discovery
  • In prediction tasks, it's often unclear which inputs cause the target and which do not
  • Both Rex and IRM aim to identify causal inputs and predict based solely on these causes

Advantages of Rex and IRM

• Generalization to New Domains
  • If encountering a new domain where x has been intervened on, but the mechanism for y is preserved, the prediction rule remains effective
  • This is different from typical out-of-domain generalization, which assumes that P(Y|X) is preserved
  • Here, the assumption is that the mechanism for Y is preserved

Advantages of Rex over IRM

• Covariate Shift
  • Rex addresses covariate shift and robustly handles changes in the distribution of the causes of y
• Potential Disadvantages
  • Can be sensitive when different inputs have varying levels of predictability