Impact of spectrum bias on deep learning–based stroke MRI analysis

Abstract

Purpose

To evaluate spectrum bias in stroke MRI analysis by excluding cases with uncertain acute ischemic lesions (AIL) and examining patient, imaging, and lesion factors associated with these cases.

Materials and Methods

This single-center retrospective observational study included adults with brain MRIs for suspected stroke between January 2020 and April 2022. Diagnostic uncertain AIL were identified through reader disagreement or low certainty grading by a radiology resident, a neuroradiologist, and the original radiology report consisting of various neuroradiologists. A commercially available deep learning tool analyzing brain MRIs for AIL was evaluated to assess the impact of excluding uncertain cases on diagnostic odds ratios. Patient-related, MRI acquisition-related, and lesion-related factors were analyzed using the Wilcoxon rank sum test, χ2 test, and multiple logistic regression. The study was approved by the National Committee on Health Research Ethics.

Results

In 989 patients (median age 73 (IQR: 59–80), 53% female), certain AIL were found in 374 (38%), uncertain AIL in 63 (6%), and no AIL in 552 (56%). Excluding uncertain cases led to a four-fold increase in the diagnostic odds ratio (from 68 to 278), while a simulated case-control design resulted in a six-fold increase compared to the full disease spectrum (from 68 to 431). Independent factors associated with uncertain AIL were MRI artifacts, smaller lesion size, older lesion age, and infratentorial location.

Conclusion

Excluding uncertain cases leads to a four-fold overestimation of the diagnostic odds ratio. MRI artifacts, smaller lesion size, infratentorial location, and older lesion age are associated with uncertain AIL and should be accounted for in validation studies.