ICU Clinician Perspectives on Machine Learning and the Implementation of a Mechanical Ventilation Prediction Tool: A Single Center Survey Study

Abstract

Rationale Although there is considerable interest in machine learning (ML) algorithms to improve patient care, implementation of these algorithms into practice has been limited. Our team developed and validated a deep learning algorithm to predict respiratory failure requiring mechanical ventilation in patients in the intensive care unit (ICU), including those with COVID-19. To help optimize implementation of this tool, we developed and disseminated a survey assessing ICU physician perspectives on the acceptability and feasibility of this tool at our institution. Methods We distributed an 8-item survey to 99 critical care trainees and faculty at our institution via email. The survey consisted of 6 multiple choice and 2 free response questions, with an ordinal scale of 1-5 used in perception-based questions. The survey was designed in accordance with international recommendations for web-based surveys. Our survey was reviewed for completeness by a team of critical care, machine learning, and implementation science experts. Data were collected over a 2- week period in May of 2021. This survey was anonymous and exempt from IRB review. Results Fifty-three critical care physicians (53.5% of providers contacted) started the survey, and of these, 88.7% (47/53) completed the survey. Fifty-nine percent (n=31) of respondents were attendings, 36% (n=19) fellows, and 3.7% (n=2) residents. Baseline knowledge of ML was low (mean= 2.40/5), with only 7.5% (n=4) of respondents rating their knowledge as a 4 or 5. Fifteen percent (n=8) had knowingly used an ML-based tool in their clinical practice. Confidence in predicting the need for mechanical ventilation due to COVID-19 (mean=3.57/5) was slightly lower than for respiratory failure due to all other causes (mean=3.89/5). Overall willingness to utilize an ML-based algorithm was favorable (mean=3.32/5). Factors most likely to increase likelihood of utilization were “high quality evidence that it outperformed trained clinicians” (mean=4.28/5), “transparency of the data utilized” (mean= 4.13/5), and “limited workflow interruption” (mean=4.09/5). Shared concerns from participants included “alarm fatigue” and “workflow interruption.” Conclusion The results suggest that ICU physicians have had limited exposure to ML-based tools, but feel such a tool would be beneficial in the context of predicting need for mechanical ventilation in ICU patients and those with COVID-19. Evidence of the tool's efficacy and data transparency were high priorities for respondents, and there was concern over workflow interruptions. This survey provided a baseline assessment of physician acceptance of a novel ML-based tool, which will be crucial in optimizing its implementation into clinical practice at our institution.