Syn-rPPG: Improving unsupervised remote photoplethysmography extraction with synthesized videos using generative models

Remote photoplethysmography (rPPG) is a non-contact technology used to capture cardiac activity from the face, providing measurements of physiological parameters. Current unsupervised methods for rPPG tasks often focus on contrastive learning, which highlights relationships between samples but struggles with a lack of diverse training data, particularly in terms of varying skin colors and motion types. This limits model effectiveness in complex real-world scenarios. Generative models offer a potential solution by creating synthetic samples to enrich the training data. In this study, we explore the impact of using synthetic videos generated by style transfer and motion transfer techniques to enhance unsupervised rPPG tasks. We generate two types of synthetic videos: skin color synthetic videos and motion synthetic videos. These address the key challenges in rPPG, namely skin color variations and motion artifacts. Our analysis shows that these synthetic videos provide valuable physiological information, improving the performance and robustness of unsupervised models. Additionally, we propose a novel lightweight rPPG network, Style-Aware rPPG Fusion Net (SAFNet), based on an encoder–decoder structure, which is optimized for joint training with synthetic videos. By incorporating a feature fusion approach, SAFNet captures richer spatiotemporal information, resulting in superior performance and robustness. Extensive experiments on four public benchmark datasets demonstrate that our method achieves excellent results, particularly in challenging conditions, proving the effectiveness of using synthetic data to enhance remote physiological measurements.