Continual Test-Time Adaptation for Robust Remote Photoplethysmography Estimation

Abstract

Remote photoplethysmography (rPPG) estimation has made considerable progress by leveraging deep learning, yet its performance remains highly susceptible to the domain shifts caused by lighting, skin tone and movement, particularly during inference. Moreover, continuous adaptation across multiple domains is also challenging due to the dynamic environmental changes such as lighting transitions or continuous motions. Domain adaptation has been widely investigated, mostly focusing on classification tasks with labels. Prior arts to address domain shifts in rPPG estimation, which is a regression task, rely on labeled data, require pretraining on target domains, and focus on single-domain test-time adaptation (TTA). However, there are still remaining challenges in TTA for their applicability of rPPG estimation in real-world scenarios such as no label during inference, continuous adaptation over multiple domains, and potential catastrophic forgetting when re-adapting to the source domain. In this work, we recast an rPPG TTA problem as a continual learning and propose an efficient continual TTA method that mitigates significant domain shifts in multiple target domains without labels by leveraging the non-contrastive unsupervised learning loss with selective updates of the batch normalization layers only as well as alleviates catastrophic forgetting in source domain by adopting the learning without forgetting (LwF) regularization in the frequency domain. Our method without target labels consistently yielded improved performance in challenging continual adaptation scenarios, including adapting to multiple new domain datasets over several cycles. This approach not only mitigates catastrophic forgetting in the source domain, but also ensures robust performance across different domains.