Validation of a Novel Data-Driven Algorithm to Detect Atypical Prescriptions in Radiation Therapy

Abstract

Purpose

Erroneous radiation therapy (RT) prescriptions (Rx) can lead to injury or death of patients. A novel data-driven model that uses similarity learning to identify atypical Rx was recently published. In that study, prototype analysis was conducted within a single institution with a single treatment site. The present study sets out to validate the robustness of the model by applying the model to multiple disease sites using a different institution’s data.

Methods and Materials

A query was conducted of Brown University Health RT treatment records for thoracic and brain cancer patients from 1995 to 2021 to create historical databases used for training. The query included records containing data on the Rx and patient-specific features. Simulated anomalies were created to mimic potential errors and were used in the training and testing of the model. Model performance was evaluated using F1 score.

Results

F1 scores for the brain site are 99% for intensity modulated RT, 90% for stereotactic radiation therapy/ radiosurgery/SRT, and 94% for 3-dimensional RT. F1 scores for the thoracic site are 95%, 90%, and 95% for the 3 techniques, respectively. Statistical analysis shows no significant differences between the model’s prediction and ground truth.

Conclusions

The model performance shows feasibility for application to various disease sites across different institutions. This model can be used alongside physicians and physicists during peer review chart rounds to aid in the detection of potential RT Rx errors.