Q-learning via deep learning-based Buckley-James method for non-linear censored data.

Abstract

In healthcare, personalized treatment strategies are vital for improving patient outcomes, especially under right-censored survival data. We propose Dynamic Deep Buckley-James Q-Learning, a novel counterfactual Q-learning algorithm that integrates deep learning with the Buckley-James method to simultaneously address censoring and nonlinear modeling challenges. By explicitly capturing complex, nonlinear interactions between covariates and treatment effects, the algorithm robustly estimates optimal dynamic treatment regimes. Leveraging a counterfactual framework, we define and estimate potential survival outcomes under hypothetical treatment sequences, enabling unbiased Q-function estimation even in the presence of time-dependent covariates and right censoring. The algorithm maximizes the expected imputed survival reward under these counterfactual scenarios. Simulation studies and real-world data analysis demonstrate its superior performance in predictive accuracy and treatment decision-making, offering a powerful framework for individualized care in complex clinical settings.