A feasibility study of a video-based heart rate estimation method with convolutional neural networks

Remote photoplethysmography (rPPG) is a kind of video-based heart rate (HR) estimation technique which has widely potential applications in health monitoring and human- computer interaction. However, the accuracy of conventional rPPG methods is easily disturbed by motion and illumination artifacts. Recently, some deep learning based rPPG methods have attracted many attentions due to its good performance and robustness to noise. This paper proposes a new rPPG scheme using a convolutional neural network (CNN) to map the pulse accumulated image to corresponding true heart rate value, where the spatial-temporal input images are constructed with raw pulses from conventional rPPG methods. In order to check the feasibility and ideal performance of this method, synthetic rPPG pulses are built using real electrocardiograph (ECG) or blood volume pulse (BVP) signals through a modified Akima cubic Hermite interpolation. We test the proposed method in three cases, subject dependent, subject independent, and also a cross-dataset one. The experimental results show that our method performs well in heart rate value estimation with synthetic rPPG pulses even for the cross-dataset case (mean absolute error HRmae = 4.36 BPM, root mean square error HRrmse = 6.26 BPM, mean error rate percentage HRmer = 5.46%). This pilot study verifies the feasibility of the proposed method and provides a solid foundation for the follow-up research with real rPPG pulses.