Some superiority trials with non-significant results published in high impact factor journals correspond to non-inferiority situations: a research-on-research study.

Abstract

Objective: Many negative randomized controlled trials (RCTs) report spin in their conclusions to highlight the benefits of the experimental arm, which could correspond to a non-inferiority (NI) objective. We aimed to evaluate whether some negative superiority RCTs comparing two active interventions could correspond to an NI situation and to explore associated trial characteristics.

Study design and setting: We searched PubMed for superiority RCTs comparing two active interventions with non-statistically significant results for the primary outcome that were published in 2021 in the 5 journals with the highest impact factor in each medical specialty. Three reviewers independently evaluated whether trials could correspond to an NI situation (i.e., an evaluation of efficacy as the primary outcome, with the experimental intervention presenting advantages including better safety profile, ease of administration, or decreased cost as compared with the control intervention).

Results: Of the 147 trials included, 19 (12.9%, 95% CI [7.9%, 19.4%]) corresponded to a potential NI situation, and as compared with trials not in a potential NI situation, they were published in a journal with a lower impact factor (median impact factor 8.7 vs 15.6), were more frequently rated at high or some concerns regarding risk of bias (n=14, 73.7% vs. n=69, 53.9%) and reported spin in the article conclusions (n=11, 57.9% vs. n=24, 18.8%).

Conclusion: A non-negligible proportion of superiority negative trials comparing two active interventions could correspond to an NI situation. These trials seemed at increased risk of bias and frequently reported spin in the conclusions, which may distort the interpretation of results.