Using simulation and machine learning to maximise the benefit of intravenous thrombolysis in acute stroke in England and Wales: the SAMueL modelling and qualitative study

Abstract

Stroke is a common cause of adult disability. Expert opinion is that about 20% of patients should receive thrombolysis to break up a clot causing the stroke. Currently, 11–12% of patients in England and Wales receive this treatment, ranging between 2% and 24% between hospitals. We sought to enhance the national stroke audit by providing further analysis of the key sources of inter-hospital variation to determine how a target of 20% of stroke patients receiving thrombolysis may be reached. We modelled three aspects of the thrombolysis pathway, using machine learning and clinical pathway simulation. In addition, the project had a qualitative research arm, with the objective of understanding clinicians’ attitudes to use of modelling and machine learning applied to the national stroke audit. Anonymised data were collected for 246,676 emergency stroke admissions to acute stroke teams in England and Wales between 2016 and 2018, obtained from the Sentinel Stroke National Audit Programme. Use of thrombolysis could be predicted with 85% accuracy for those patients with a chance of receiving thrombolysis (i.e. those arriving within 4 hours of stroke onset). Machine learning models allowed prediction of likely treatment choice for each patient at all hospitals. A clinical pathway simulation predicted hospital thrombolysis use with an average absolute error of 0.5 percentage points. We found that about half of the inter-hospital variation in thrombolysis use came from differences in local patient populations, and half from in-hospital processes and decision-making. Three changes were applied to all hospitals in the model: (1) arrival to treatment in 30 minutes, (2) proportion of patients with determined stroke onset times set to at least the national upper quartile and (3) thrombolysis decisions made based on majority vote of a benchmark set of 30 hospitals. Any single change alone was predicted to increase national thrombolysis use from 11.6% to between 12.3% and 14.5% (with clinical decision-making having the most effect). Combined, these changes would be expected to increase thrombolysis to 18.3% (and to double the clinical benefit of thrombolysis, as speed increases also improve clinical benefit independently of the proportion of patients receiving thrombolysis); however, there would still be significant variation between hospitals depending on local patient population. For each hospital, the effect of each change could be predicted alone or in combination. Qualitative research with 19 clinicians showed that engagement with, and trust in, the model was greatest in physicians from units with higher thrombolysis rates. Physicians also wanted to see a machine learning model predicting outcome with probability of adverse effect of thrombolysis to counter a fear that driving thrombolysis use up may cause more harm than good. Models may be built using data available in the Sentinel Stroke National Audit Programme only. Not all factors affecting use of thrombolysis are contained in Sentinel Stroke National Audit Programme data and the model, therefore, provides information on patterns of thrombolysis use in hospitals, but is not suitable for, or intended as, a decision aid to thrombolysis. Machine learning and clinical pathway simulation may be applied at scale to national audit data, allowing extended use and analysis of audit data. Stroke thrombolysis rates of at least 18% look achievable in England and Wales, but each hospital should have its own target. Future studies should extend machine learning modelling to predict the patient-level outcome and probability of adverse effects of thrombolysis, and apply co-production techniques, with clinicians and other stakeholders, to communicate model outputs. This project was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme and will be published in full in Health and Social Care Delivery Research; Vol. 10, No. 31. See the NIHR Journals Library website for further project information.