Domain consistency learning for continual test-time adaptation in image semantic segmentation

In the open-world scenario, the challenge of distribution shift persists. Test-time adaptation adjusts the model during test-time to fit the target domain’s data, addressing the distribution shift between the source and target domains. However, test-time adaptation methods still face significant challenges with continuously changing data distributions, especially since there are few methods applicable to continual test-time adaptation in image semantic segmentation. Furthermore, inconsistent semantic representations across different domains result in catastrophic forgetting in continual test-time adaptation. This paper focuses on the problem of continual test-time adaptation in semantic segmentation tasks and proposes a method named domain consistency learning for continual test-time adaptation. We mitigate catastrophic forgetting through feature-level and prediction-level consistency learning. Specifically, we propose domain feature consistency learning and class awareness consistency learning to guide model learning, enabling the target domain model to extract generalized knowledge. Additionally, to mitigate error accumulation, we propose a novel value-based sample selection method that jointly considers the pseudo-label confidence and style representativeness of the test images. Extensive experiments on widely-used semantic segmentation benchmarks demonstrate that our approach achieves satisfactory performance compared to state-of-the-art methods.