A simple yet lightweight module for enhancing domain generalization through relative representation

Domain Generalization (DG) learns a model from multiple source domains to combat individual domain differences and ensure generalization to unseen domains. Most existing methods focus on learning domain-invariant absolute representations. However, we empirically observe that such representations often suffer from notable distribution divergence, leading to unstable performance in diverse unseen domains. In contrast, relative representations, constructed w.r.t. a set of anchors, naturally capture geometric relationships and exhibit intrinsic stability within a dataset. Despite this potential, their application to DG remains largely unexplored, due to their common transductive assumption that anchors require access to target-domain data, which is incompatible with the inductive setting of DG. To address this issue, we design Re2SL, a simple and lightweight plug-in module that follows a pre-trained encoder and constructs anchors solely from source-domain prototypes, thereby ensuring a completely inductive design. To our knowledge, Re2SL is the first to explore relative representation for DG. This design is inspired by the insight that ReSidual differences between absolute and domain-specific representations can spontaneously seek stable representations within the same distribution shared across all domains. Leveraging these stable representations, we construct cross-domain ReLative representation to enhance stability and transferability without accessing any target data during training or anchor computation. Empirical studies show that our constructed representation exhibits minimal $H$-divergence, confirming its stability. Notably, Re2SL achieves up to 4.3 % improvement while reducing computational cost by 90 %, demonstrating its efficiency.