A comprehensive validation study on the influencing factors of cough-based COVID-19 detection through multi-center data with abundant metadata

Abstract

Objective:

In recent years, COVID-19 has placed enormous burdens on healthcare systems. Currently, hundreds of thousands of new cases are reported monthly. World Health Organization is managing COVID-19 as a long-term disease, indicating that an efficient and low-cost detection method remains necessary. Previous studies have shown competitive results on cough-based COVID-19 detection combined with deep learning methods. However, most studies have focused only on improving classification performance on single-source data while neglecting the impact of various factors in real-world applications.

Methods:

To this end, we collected clinical and large-scale crowdsourced cough audios with abundant metadata to comprehensively validate the performance differences among different groups. Specifically, we leveraged self-supervised learning for pre-training and fine-tuned the model with data from different sources. Then based on the metadata, we compared the effects of factors such as cough types, symptoms, and infection stages on detection performance. Moreover, we recorded clinical indicators of viral load and antibody levels and observed the correlation between predicted probabilities and indicator values for the first time. Several open-source datasets were tested to verify the model generalizability.

Results:

The area under receiver operating characteristic curve is 0.79 for clinical data and 0.69 for crowdsourced data, indicating differences between clinical validation and real-world application. The performance in detecting symptomatic COVID-19 subjects is usually better than detecting asymptomatic COVID-19 subjects. The prediction results show weak correlation with clinical indicators on a small number of clinical data. Poor detection performance in recovery individuals and open-source datasets shows a limitation of existing cough-based detection models.

Conclusion:

Our study validated the model performance and limitations using multi-source data with abundant metadata, which helped researchers evaluate the feasibility of cough-based COVID-19 detection model in practical applications.