PulseFormer: Continuous Remote Heart Rate Measurement Through Zoomed Time-Spectral Attention

Abstract

Despite the recent advances in remote heart rate measurement, most improvements primarily focus on recovering the rPPG signal, often overlooking the inherent challenges of estimating heart rate (HR) from the derived signal. Furthermore, most existing methods adopt the average HR per video to assess model performance, thus relying on rather large temporal windows to produce a single estimate; this hampers their applicability to scenarios in which the continuous monitoring of a patient’s physiological status is crucial. Besides, this evaluation approach can also lead to biased performance assessments due to low continuous precision, as it considers only the mean value of the entire video. In this paper, we present the PulseFormer, a novel continuous deep estimator for remote HR. Our proposed method utilizes a time-frequency attention block that leverages the enhanced resolution properties of the Chirp-Z Transform (CZT) to accurately estimate HR from the recovered low-resolution signal using a reduced temporal window size. We validate the effectiveness of our model on the large-scale Vision-for-Vitals (V4V) benchmark, designed for continuous physiological signals estimation from facial videos. The results reveal outstanding frame-to-frame HR estimation capabilities, establishing the proposed approach as a robust and versatile estimator that could be used with any rPPG method.