Deep reinforcement learning extracts the optimal sepsis treatment policy from treatment records

Abstract

Sepsis is one of the most life-threatening medical conditions. Therefore, many clinical trials have been conducted to identify optimal treatment strategies for sepsis. However, finding reliable strategies remains challenging due to limited-scale clinical tests. Here we tried to extract the optimal sepsis treatment policy from accumulated treatment records. In this study, with our modified deep reinforcement learning algorithm, we stably generated a patient treatment artificial intelligence model. As training data, 16,744 distinct admissions in tertiary hospitals were used and tested with separate datasets. Model performance was tested by t test and visualization of estimated survival rates. We also analyze model behavior using the confusion matrix, important feature extraction by a random forest decision tree, and treatment behavior comparison to understand how our treatment model achieves high performance. Here we show that our treatment model’s policy achieves a significantly higher estimated survival rate (up to 10.03%). We also show that our models’ vasopressor treatment was quite different from that of physicians. Here, we identify that blood urea nitrogen, age, sequential organ failure assessment score, and shock index are the most different factors in dealing with sepsis patients between our model and physicians. Our results demonstrate that the patient treatment model can extract potential optimal sepsis treatment policy. We also extract core information about sepsis treatment by analyzing its policy. These results may not apply directly in clinical settings because they were only tested on a database. However, they are expected to serve as important guidelines for further research. Sepsis is one of the most life-threatening medical conditions. It can be challenging to select the best treatment strategy for individual patients. We developed a computational model to identify optimal treatment strategies and applied it to a large amount of data obtained from patients with sepsis. We identified particular types of information about the patients that can be used to decide on the best medication and dose to treat sepsis. Further development of our treatment model could potentially enable it to be used to improve the survival of patients with sepsis. Also, the results we obtained could be used to improve the general guidance followed when treating people with sepsis. Choi et al. develop and validate a sepsis treatment model based on deep reinforcement learning using patients’ treatment records. The model increases patients’ estimated survival rate and analysis of the treatment model’s strategies also suggests potential future sepsis research directions.