Explainable, robust and fair user-centric AI system for the diagnosis and prognosis of severe pneumonia

The COVID-19 pandemic has markedly exacerbated the complexities surrounding the diagnosis and prognosis of diverse severe pneumonia types, posing extraordinary challenges to healthcare systems worldwide. While previous AI-based approaches primarily targeted COVID-19 severe pneumonia and sought to enhance machine learning accuracy, they often neglected critical aspects such as distinguishing diagnostic and prognostic features among COVID-19 infectious, non-COVID infectious, and non-infectious severe pneumonia, as well as the explainability and fairness of user-centric AI assist decisions. This study es the need for robust, fair, and reliable diagnosis and prognosis of severe pneumonia within the context of the COVID-19 pandemic. This paper introduces a user-centric framework that first employs a GaussianCopula-based data augmentation method to enhance fairness by addressing small imbalanced sample sets. Following this, the framework introduces an explainable AI system designed to classify three types of severe pneumonia using demographic and physiological indicators, offering transparent decision-making processes and an understandable analysis of prognosis risk factors. Our fair system utilizes transparent models exclusively, which enables healthcare practitioners to access intelligent and reliable medical services such as pre-diagnosis and prognosis analysis (the likelihood of death) of severe pneumonia. The results show the data augmentation method efficiently reduces data bias and enhances fairness, reaching 70.70% distribution similarity. Our transparent model-based severe pneumonia classification module achieves 98.88% F1-scores on a real-world dataset. The transparent mechanism reveals that the four most significant features for classifying severe pneumonia types are ‘Interleukin_6’, ‘Albumin’, ‘D_Dimer’, and ‘CD4_absolute_count’. Meanwhile, the explainable statistical analysis identifies critical mortality risk factors for each pneumonia category: ‘Blood platelet’ and ‘Creatinine’ for COVID-19 severe pneumonia, ‘Hemameba’, ‘Interleukin-6’, and ‘Uric Acid’ for non-COVID-19 infectious severe pneumonia, and ‘Hemameba’, ‘BNP’, ‘Cholesterol’, and ‘PT’ for non-infectious severe pneumonia. Our study highlights the potential of transparent machine learning algorithms for accurate diagnosis and Cox proportional regression for transparent risk trend prediction. These analytical tools and medical results can facilitate early and appropriate management of pneumonia patients for doctors, potentially revolutionizing diagnostic processes and patient care strategies to improve clinical outcomes.