Non-invasive diagnosis of lung diseases via multimodal feature extraction from breathing audio and chest dynamics

Abstract

Early and accurate diagnosis of lung diseases is crucial for effective treatment. While traditional methods have limitations, audio analysis offers a promising non-invasive approach. However, existing studies often rely solely on acoustic features, neglecting valuable information contained in visual cues like chest wall dynamics. This research proposes a novel multimodal approach that integrates both audio and visual modalities to enhance lung disease detection. By extracting and fusing features from both modalities, we aim to capture a more comprehensive representation of lung health. The proposed deep learning model, trained on a dataset of audio and video recordings, achieved a validation accuracy of 92.02 %. The model effectively leverages features such as pitch, MFCCs, and breathing audio envelopes, along with visual cues from chest wall dynamics, to accurately classify different lung disease categories. This multimodal approach offers several advantages, including improved accuracy, robustness to noise and variability, and the potential for early disease detection. By addressing the limitations of single-modality approaches, this research contributes to the development of more effective and accessible lung disease diagnostic tools.