Remote Optical Sensing of Heart Sounds for Biometric Identification Using Information Fusion

Machine learning (ML) has extensive applications in healthcare for the classification of vital and health signals from raw data. This method both improves the existing measurement techniques and provides opportunities for developing new approaches. In this work, we present two different information fusion methods for classifying the heart sound signals recorded by a new-generation optical stethoscope. The optical stethoscope consists of a green laser diode and a high-speed industrial camera; the laser is pointed to the human neck and chest area and the camera receives the reflected laser speckles. The proposed fusion schemes then combine the output of VGG-19 and Bi-LSTM neural networks at the feature and decision levels and fuse the data from different measurement points on the body. For the decision-level fusion, a weighted sum function is utilized to merge the information of two networks, whereas, for the feature level, a fully connected layer block is implemented to cascade the output vectors from the last layer of VGG-19 and Bi-LSTM. Both methods show the subsequent improvement of 10% or more compared to the single network thanks to the difference in extracted information between convolution and recurrent neural networks. When training and testing the proposed fusion methods on the combination of neck and chest data, the mitral and tricuspid valve auscultation positions demonstrate better average classification performance than the aortic and pulmonary valve positions, with validation via ten iterations of training and testing of the ML models.