Reinforcement learning using neural networks in estimating an optimal dynamic treatment regime in patients with sepsis

Objective:

Early fluid resuscitation is crucial in the treatment of sepsis, yet the optimal dosage remains debated. This study aims to determine the optimal multi-stage fluid resuscitation dosage for sepsis patients.

Methods:

We propose a reinforcement learning algorithm with neural networks (RL-NN), utilizing the flexibility of deep learning architectures to mitigate model misspecification. We use cross-validation and random search for hyperparameter tuning to further enhance model robustness and generalization.

Results:

Simulation results demonstrate that our method outperforms existing methods in terms of both the percentage of correctly classified optimal treatments and the predicted counterfactual mean outcome. Applying this method to the sepsis cohort from the Medical Information Mart for Intensive Care III (MIMIC-III), we recommend that all sepsis patients receive adequate fluid resuscitation ($\ge$ 30 mL/kg) within the first 3 h of admission to the MICU. Our approach is expected to significantly reduce the mean SOFA score by 23.71%, enhancing patient outcomes.

Conclusion:

Our RL-NN method offers an accurate, real-time approach to optimizing sepsis treatment and aligns with the ’Surviving Sepsis Campaign’ guidelines. It also has the potential to be integrated with existing electronic health record (EHR) systems, guiding clinical decision-making and thereby improving patient prognosis.