ULDepth: Transform Self-Supervised Depth Estimation to Unpaired Multi-Domain Learning

Abstract

This paper introduces a general plug-in framework designed to enhance the robustness and cross-domain generalization of self-supervised depth estimation models. Current models often struggle with real-world deployment due to their limited ability to generalize across diverse domains, such as varying lighting and weather conditions. Single-domain models are optimized for specific scenarios while existing multi-domain approaches typically rely on paired images, which are rarely available in real-world datasets. Our framework addresses these limitations by training directly on unpaired real images from multiple domains. Daytime images serve as a reference to guide the model in learning consistent depth distributions across these diverse domains through adversarial training, eliminating the need for paired images. To refine regions prone to artifacts, we augment the discriminator with positional encoding, which is combined with the predicted depth maps. We also incorporate a dynamic normalization mechanism to capture shared depth features across domains, removing the requirement for separate domain-specific encoders. Furthermore, we introduce a new benchmark designed for a more comprehensive evaluation, encompassing previously unaddressed real-world scenarios. By focusing on unpaired real data, our framework significantly improves the generalization capabilities of existing models, enabling them to better adapt to the complexities and authentic data encountered in real-world environments.