ReCAP2: Rectified and context-aware polarization prompting for robust depth enhancement

Abstract

Accurate depth perception is fundamental for numerous computer vision applications, yet depth maps acquired from commodity sensors often suffer from artifacts and inaccuracies, necessitating effective enhancement techniques. Polarization imaging, capturing rich geometric cues robust to illumination variations, offers a promising modality to guide this process. However, effectively integrating these cues within learning-based depth enhancement frameworks remains challenging. Existing methods often overlook the inherent representational gap between depth and polarization features and employ context-agnostic fusion mechanisms, incapable of generating prompts adaptive to cross-modal relationships and local context. To address these limitations, we propose a novel Rectified and Context-Aware Polarization Prompting (ReCAP${}^2$) framework for depth enhancement models. The ReCAP${}^2$ first performs initial feature rectification across both channel and spatial dimensions to bridge the modality gap. Subsequently, it generates fine-grained polarization prompts by leveraging dual-level context: utilizing cross-modal context ensures the prompts encode pertinent inter-modality relationships, while processing spatial neighborhood context yields prompts spatially tailored to regional content. Consequently, these dual-context aware prompts provide precise, adaptive guidance for the foundation model, facilitating more robust depth enhancement. Extensive experiments demonstrate the effectiveness of our method. On the multi-modal HAMMER dataset, our method shows superior accuracy and robustness across diverse sensor types in indoor scenes under both full fine-tuning and prompt tuning settings. Furthermore, cross-domain evaluations on the challenging CroMo dataset validate its strong generalization to outdoor environments.