Efficacy of a deep learning-based software for chest X-ray analysis in an emergency department.

Abstract

Purpose: The purpose of this study was to evaluate the efficacy of a deep learning (DL)-based computer-aided detection (CAD) system for the detection of abnormalities on chest X-rays performed in an emergency department setting, where readers have access to relevant clinical information.

Materials and methods: Four hundred and four consecutive chest X-rays performed over a two-month period in patients presenting to an emergency department with respiratory symptoms were retrospectively collected. Five readers (two radiologists, three emergency physicians) with access to clinical information were asked to identify five abnormalities (i.e., consolidation, lung nodule, pleural effusion, pneumothorax, mediastinal/hilar mass) in the dataset without assistance, and then after a 2-week period, with the assistance of a DL-based CAD system. The reference standard was a chest X-ray consensus review by two experienced radiologists. Reader performances were compared between the reading sessions, and interobserver agreement was assessed using Fleiss' kappa test.

Results: The dataset included 118 occurrences of the five abnormalities in 103 chest X-rays. The CAD system improved sensitivity for consolidation, pleural effusion, and nodule, with respective absolute differences of 8.3 % (95 % CI: 3.8-12.7; P < 0.001), 7.9 % (95 % CI: 1.7-14.1; P = 0.012), and 29.5 % (95 % CI: 19.8-38.2; P < 0.001), respectively. Specificity was greater than 89 % for all abnormalities and showed a minimal but significant decrease with DL for nodules and mediastinal/hilar masses (-1.8 % [95 % CI: -2.7 - -0.9]; P < 0.001 and -0.8 % [95 % CI: -1.5 - -0.2]; P = 0.005). Inter-observer agreement improved with DL, with kappa values ranging from 0.40 [95 % CI: 0.37-0.43] for mediastinal/hilar mass to 0.84 [95 % CI: 0.81-0.87] for pneumothorax.

Conclusion: Our results suggest that DL-assisted reading increases the sensitivity for detecting important chest X-ray abnormalities in the emergency department, even when clinical information is available to the radiologist.