External validation of an RSNA 2023 Abdominal Trauma AI Challenge high performing machine learning model in the detection and grading of splenic injuries on CT.

Purpose: This study aims to validate the performance of an award-winning machine learning (ML) model from the Radiological Society of North America (RSNA) 2023 Abdominal Trauma AI Challenge in detecting splenic injuries on CT scans using a large, geographically and temporally distinct external dataset.

Method: A single-center retrospective study was conducted using an external dataset comprising 1216 CT scans (608 positive and 608 negative for splenic injuries). The ML model, trained on the RSNA Abdominal Traumatic Injury CT (RATIC) dataset, employs a multi-component pipeline including 2D MaxVit, 2.5D CoatNet with LSTM for study-level predictions. Model performance was evaluated using sensitivity, specificity, PPV, NPV, accuracy, F1 score, and AUC.

Results: The ML model achieved an AUC of 0.931 (95% CI: 0.917, 0.945) for binary classification of splenic injuries, with an accuracy of 0.849 (95% CI: 0.827, 0.868), sensitivity of 0.747 (95% CI: 0.711, 0.780), and specificity of 0.951 (95% CI: 0.930, 0.965). For high-grade splenic injuries, the model achieved an AUC of 0.950 (95% CI: 0.932, 0.968), accuracy of 0.928 (95% CI: 0.912, 0.941), sensitivity of 0.719 (95% CI: 0.643, 0.784), and specificity of 0.958 (95% CI: 0.944, 0.968).

Conclusion: The ML model shows strong, reliable performance and generalizability in detecting and grading splenic injuries on CT scans. This supports its potential clinical application, particularly for quick and accurate diagnosis in splenic trauma patients, and highlights the value of RSNA AI challenges in advancing clinical research and applications in medical imaging.