A reproducibility crisis for clinical metabolomics studies

Abstract

Cancer is a leading cause of world-wide death and a major subject of clinical studies focused on the identification of new diagnostic tools. An in-depth meta-analysis of 244 clinical metabolomics studies of human serum samples highlights a reproducibility crisis. A total of 2,206 unique metabolites were reported as statistically significant across the 244 studies, but 72% (1,582) of these metabolites were identified by only one study. Further analysis shows a random disparate disagreement in reported directions of metabolite concentration changes when detected by multiple studies. Statistical models revealed that 1,867 of the 2,206 metabolites (85%) are simply statistical noise. Only 3–12% of these metabolites reach the threshold of statistical significance for a specific cancer type. Our findings demonstrate the absence of a detectable metabolic response to cancer and provide evidence of a serious need by the metabolomics community to establish widely accepted best practices to improve future outcomes.