Physiological navigation amplifier for remote extracting PPG signals from face video clips

Abstract

Remote photoplethysmography (rPPG) signal measurement based on face video clips is one of the essential methods for non-contact monitoring of human heart health. However, the subtle variations in face color associated with the rPPG signal can be easily contaminated by external noise, leading to a limited cross-domain generalization of existing models. In the context of short face video clips, we have observed that the influence of factors such as background, illumination, and other variables results in significant variation in noise between samples, while those within the sample typically exhibit similar patterns. Therefore, those distinctive features can be utilized to model noise variation. This paper presents a physiological navigation amplifier with a self-adaptive differential feature representation structure for rPPG signal measurement in challenging video clips. First, a noise representation encoder is proposed for self-adaptive differential feature representation. Second, a physiological navigation amplifier is designed to extract the subtle rPPG signal from facial feature sets by disentangling rPPG and noise features in the spatiotemporal space. Finally, to solve the problem of measured signal degradation caused by drastic external disturbances that result in network performance decline, a learnable signal rectification matrix is employed to reconstruct the measured rPPG signal. The experimental results, obtained through publicly available intra-datasets and cross-dataset validation, demonstrate that our proposed method reduces the mean absolute error (MAE) in rPPG signal and heart rate measurements by more than 17 % (OBF), 13 % (COHFACE), and 24 % (UBFC) compared to state-of-the-art methods.