RelPose-TTA: Energy-based relative pose correction for test-time adaptation of category-level object pose estimation

Abstract

Category-level object pose estimation is fundamental for robotic grasping and manipulation, yet models trained on synthetic data often generalize poorly to real-world environments due to substantial domain gaps. Test-time adaptation (TTA) offers a promising solution to address this challenge, but existing methods frequently depend on noisy pseudo-labels or complex optimization, which can lead to performance degradation and error accumulation over time. In this paper, we propose RelPose-TTA, a test-time adaptation framework that improves the generalization and long-term stability for category-level object pose estimation in previously unseen real-world environments. The core idea is to exploit the relative motion between consecutive frames, which is typically more stable and reliable than single-frame absolute pose estimation, and to use it as a self-supervisory signal during inference. Concretely, RelPose-TTA introduces an energy-based relative pose corrector to model inter-frame motion and mitigate ambiguities induced by occlusions, object symmetries, and large viewpoint changes. During test-time adaptation, the corrector is updated online via contrastive learning and is tightly coupled with point cloud registration, so that refined relative pose estimates can effectively guide absolute pose refinement. Extensive experiments demonstrate that RelPose-TTA consistently outperforms prior TTA methods in unseen real-world settings, while substantially reducing long-term drift and maintaining stable pose predictions.