Long Run Non-Transitivity of Win Ratio and How to Rule It Out.

Abstract

The win ratio (WR) has emerged as an increasingly used method for analyzing composite endpoints in randomized controlled trials (RCTs). Its growing popularity stems from its ability to accommodate a hierarchy of event types within a composite primary endpoint in an RCT. However, as a relatively new measure, the WR exhibits some surprising properties. Oakes and other authors have noted that for a given triplet of participants, there are realizations of event times and hierarchy of event types when "wins" form a loop. They term this property "intransitivity." In this paper, we report and study a different phenomenon, a violation of long-run transitivity of WRs, for example, when in the long run $\text{WR}\left(T_A>T_B\right)>1$ , $\text{WR}\left(T_B>T_C\right)>1$ and paradoxically $\text{WR}\left(T_C>T_A\right)>1$ , where $T_A,T_B,T_C$ denote the time-to-event in trial arms A, B, and C (or placebo, standard of care and novel treatment, respectively). We refer to it as Efron-type "non-transitivity." Unlike Oakes-type intransitivity, Efron-type non-transitivity is an inherent property of the underlying random variables. As such, it cannot be resolved by a longer observation period or larger sample size. Lumley and Gillen showed that transitivity can be guaranteed only for univariate statistics (e.g., mean, median, etc.), whereas head-to-head comparisons can be non-transitive. WR and win proportion (WP) are head-to-head comparisons and may be non-transitive, in which case the best treatment cannot be determined. Therefore, the non-transitivity property challenges the utility of all head-to-head comparisons as efficacy measures in RCTs. In this paper, we explore the scope of this problem, focusing on identifying conditions that rule out non-transitivity for WR, WP, and hazard ratio (HR). We apply our findings to establish transitivity in two case studies using published RCTs. This work highlights an important phenomenon that must be recognized when the WR is used as an efficacy measure in randomized controlled trials.