Value assessment of augmentative artificial intelligence for assessment of pulmonary emboli on CT - a meta-analysis comprising 15,963 CT scans.

Purpose: Artificial Intelligence (AI) algorithms in radiology are currently deployed as tools to augment radiologists rather than autonomous readers. An augmentative tool should improve performance above and beyond the baseline performance achieved by the user alone. We conducted a meta-analysis to elucidate the added value of augmentative AI to radiologists for detecting Pulmonary Embolism (PE) on CT scan.

Methods: Using PRISMA guidelines, studies in which both AI and Human Interpreter (HI) assessed CT scans for pulmonary emboli were selected. Data extracted from these studies were used to compare diagnostic performance of AI and HI with an emphasis on the performance of AI above and beyond that of HI.

Results: Both HI and AI performed similarly with no statistically significant difference in the pooled estimates of sensitivity, specificity, PPV, NPV and accuracy. Subsequent analysis focusing on the differences between performance of AI and HI within each study, followed by pooled estimate, also did not demonstrate any significant difference (p < 0.05).

Conclusions: In a meta-analysis of nearly sixteen thousand CTs, AI and HI had similar performance for detection of pulmonary emboli. On one hand, this buttresses AI's use for triaging and for second reads. On the other hand, the outcomes may or may not be different when AI is added-on. The findings of this meta-analysis can be used to re-examine the use-scenarios of AI and to re-calibrate its value proposition.