Towards human-AI collaboration in radiology: a multidimensional evaluation of the acceptability of AI for chest radiograph analysis in supporting pulmonary tuberculosis diagnosis.

Abstract

Objective: Artificial intelligence (AI) technology promises to be a powerful tool in addressing the global health challenges posed by tuberculosis (TB). However, evidence for its real-world impact is lacking, which may hinder safe, responsible adoption. This case study addresses this gap by assessing the technical performance, usability and workflow aspects, and health impact of implementing a commercial AI system (qXR by Qure.ai) to support Australian radiologists in diagnosing pulmonary TB.

Materials and methods: A retrospective diagnostic accuracy evaluation was conducted to establish the technical performance of qXR in detecting TB compared to a human radiologist and microbiological reference standard. A qualitative human factors assessment was performed to investigate the user experience and clinical decision-making process of radiologists using qXR. A task productivity analysis was completed to quantify how the radiological screening turnaround time is impacted.

Results: qXR displays near-human performance satisfying the World Health Organization's suggested accuracy profile. Radiologists reported high satisfaction with using qXR based on minimal workflow disruptions, respect for their professional autonomy, and limited increases in workload burden despite poor algorithm explainability. qXR delivers considerable productivity gains for normal cases and optimizes resource allocation through redistributing time from normal to abnormal cases.

Discussion and conclusion: This study provides preliminary evidence of how an AI system with reasonable diagnostic accuracy and a human-centered user experience can meaningfully augment the TB diagnostic workflow. Future research needs to investigate the impact of AI on clinician accuracy, its relationship with efficiency, and best practices for optimizing the impact of clinician-AI collaboration.