Lowering the p value threshold in recently published Respiratory Medicine RCTs

Abstract

Introduction: The use and misuse of p values remains controversial, with increasing support for lowering the threshold of statistical significance from 0.05 to 0.005, to reduce false positive results. A recent evaluation of randomised controlled trials (RCTs) determined that 70% would maintain significance if the p value threshold was Methods: RCTs published in the American Journal of Respiratory and Critical Care Medicine (AJRCCM), European Respiratory Journal (ERJ), Chest, Lancet Respiratory Medicine and Thorax (2017-2018) were identified. Pooled analyses and RCTs using Bayesian or noninferiority analyses were excluded. Data extracted included whether each RCT was multicentre, multinational, a drug trial and sample size. Results: 125 studies were included. 62/125 (49.6%) reported a significant primary end point with p value Discussion: Of statistically significant primary end points published in respiratory medicine RCTs, only 50% would maintain statistical significance with a threshold of 0.005. This may be because treatment effects are not large enough to be assessed at this level of significance. Lowering the p value threshold would have an important impact on the clinical applicability of RCT findings and alternative methods such as Bayesian analysis should be explored as potential solutions to the controversy of p values. Reference: [1] JAMA. 2016;315(11):1141-1148.