Misinterpretation of statistical nonsignificance as a sign of potential bias: Hydroxychloroquine as a case study.

Abstract

The term "statistical significance," ubiquitous in the medical literature, is often misinterpreted, as is the "p-value" from which it stems. This article explores the implications of results that are numerically positive (e.g., those in the treatment arm do better on average) but not statistically significant. This lack of statistical significance is sometimes interpreted as strong, even decisive, evidence against an effect without due consideration of other factors. Three influential articles on hydroxychloroquine (HCQ) as a treatment for COVID-19 are illustrative. They all involve numerically positive results that were not statistically significant that were misinterpreted as strong evidence against HCQ's efficacy. These and related considerations raise concerns regarding the reliability of academic/medical reasoning around COVID-19 treatments, as well as more generally, and regarding the potential for bias stemming from conflicts of interest.