Bayesian probabilistic statements for communicating clinical trial results.

Abstract

Background: Clinical trials are often analyzed using frequentist methods and results are interpreted according to their statistical significance. However, p-values and effect measures such as odds or hazard ratios are poorly understood by researchers and policymakers, and even less by patients and clinicians. Moreover, communicating uncertainty around treatment effects is challenging. Probabilistic statements from Bayesian analyses (e.g., the probability that a drug is more effective than placebo) offer an intuitive way to convey results and uncertainty, may provide insight, and support clinical decision-making. We aimed to (i) empirically evaluate the utility of probabilistic statements by re-analyzing data from previously published trials using Bayesian methods, and (ii) provide instructional examples of how probabilistic statements can be used to communicate and interpret results from clinical trials.

Methods: We analyzed data on primary outcomes obtained from trials previously published in leading medical journals, using Bayesian methods. We included 169 primary outcomes from 130 negative trials (p-values > 0.05) across various medical fields, and 234 outcomes from 225 positive trials (p-value < 0.05) in oncology. We used uninformative priors for illustration, supplementing them with skeptical priors in sensitivity analyses. We estimated the probabilities that active treatments were more effective or safe than controls, for each outcome. We also estimated probabilities that the treatment effects were above specific thresholds for clinical significance. We described three of the included trials in more detail and additionally analyzed one trial in neurology as didactical case studies, to illustrate the use of probabilistic statements.

Results: So-called negative studies may sometimes yield clinically relevant evidence of an effect: in 26/169 (15%) cases we found > 90% probability that the experimental treatment is better than control; for 12/169 (7%) of them, this probability was > 95%. Conversely, positive trials may not always show evidence of clinically meaningful effects: in 25/234 (11%) of the positive oncology trials, there was less than 50% probability that treatment reduces hazards by 20% or more. Our case studies illustrate how to use probabilistic statements in medical decision making in practice.

Conclusions: Relying on statistical significance without considering clinical importance can mislead decision-making and potentially harm patients. Probabilistic statements from Bayesian analyses can complement traditional approaches to communicating clinical trial results.