Online outcome weighted learning with general loss functions

Abstract

The pursuit of individualized treatment rules in precision medicine has generated significant interest due to its potential to optimize clinical outcomes for patients with diverse treatment responses. One approach that has gained attention is outcome weighted learning, which is tailored to estimate optimal individualized treatment rules by leveraging each patient's unique characteristics under a weighted classification framework. However, traditional offline learning algorithms, which process all available data at once, face limitations when applied to high-dimensional electronic health records data due to its sheer volume. Additionally, the dynamic nature of precision medicine requires that learning algorithms can effectively handle streaming data that arrives in a sequential manner. To overcome these challenges, we present a novel framework that combines outcome weighted learning with online gradient descent algorithms, aiming to enhance precision medicine practices. Our framework provides a comprehensive analysis of the learning theory associated with online outcome weighted learning algorithms, taking into account general classification loss functions. We establish the convergence of these algorithms for the first time, providing explicit convergence rates while assuming polynomially decaying step sizes, with (or without) a regularization term. Our findings present a non-trivial extension of online classification to online outcome weighted learning, contributing to the theoretical foundations of learning algorithms tailored for processing streaming input-output-reward type data.