End-to-end Scalable and Low Power Multi-modal CNN for Respiratory-related Symptoms Detection

Abstract

With the onset of the highly contagious COVID-19 pandemic, early-stage and clinic-independent machine assistance is essential for initial disease diagnosis based on its symptoms such as fever, dry cough, fatigue, and dyspnea. This paper proposes a scalable and low power architecture based on end-to-end Convolutional Neural Networks (CNN) for respiratory-related symptoms (cough and dyspnea) detection. The CNN-based model will be part of the final product running on general computing processors that can assess patients similar to what doctors do at triage and telemedicine using passively recorded audio and other information. The proposed model consists of 1D-convolutions to extract audio features and combinations of 2D-convolutions and fully-connected neurons for classification. The architecture achieves a detection accuracy of 87.5% for cough and 87.3% for dyspnea respectively. The proposed work involves extensive optimization of parameters in order to develop a model architecture that can be implemented on highly constrained power budget devices while maintaining high classification accuracy. This optimization allows us to achieve the model size of 960 KB for cough detection which is 193x smaller than the related works employing the end-to-end CNN architecture. The hardware architecture is designed to provide more versatility in terms of the number of input channels, filters, data width and processing engine (P.E.) in a parameterized manner with the target of proposing a reconfigurable hardware. The proposed architecture is fully synthesized and placed-and-routed on Xilinx Artix-7 FPGA. At 47.6 MHz operating frequency, our cough detection hardware architecture consumes 211 mW of power. On the other hand, dyspnea detection hardware architecture consumes 207 mW power at an operating frequency of 50 MHz. In addition, the proposed hardware architecture meets the latency deadline of 1s needed for the efficient operation of hardware while still being energy-effective compared to related work.