Impact of artificial intelligence assistance on bone scintigraphy diagnosis.

Abstract

Bone scintigraphy is an important tool for detecting bone lesions. This study aimed to improve and evaluate the performance of our previously-developed deep learning-based model called MaligNet in helping nuclear medicine (NM) physicians interpret bone scan. Bone scintigraphy of 553 patients with imaging data from six-month follow-up records were split into training, validation, and test sets in a ratio of 353:100:100 to re-train MaligNet. Seven nuclear medicine physicians, including two junior and five senior physicians, were asked to segment and classify lesions in the test set images without and with AI assistance, which was the prediction of MaligNet. The improved performance of MaligNet was evaluated using the precision-recall (PR) and receiver operating characteristic (ROC) curves for lesion-based and patient-based classifications, respectively. The impact of AI assistance on physician reading was evaluated using reading time per case and malignancy diagnostic performance metrics. The re-trained MaligNet yielded considerably higher area under the PR curve (0.334 vs. 0.225) and higher area under the ROC curve (0.881 vs. 0.789) than the original model. For patient-based classification, AI assistance improved the average accuracy, sensitivity, specificity, and precision of the physician by 2.14%, 0.89%, 2.38%, and 1.97%, respectively, while reducing the average reading time by 31.14%. For lesion-based classification, it improved physicians' average precision by 2.95%, but did not improve sensitivity. With AI assistance, junior physicians achieved diagnostic performances comparable to those of senior physicians. AI assistance with MaligNet improved bone scintigraphy diagnostic performance and showed promise in clinical practice.