Confirmatory and Exploratory Analyses in Omics Studies with Particular Focus on Multiple Testing and P -value

Abstract

In this article, we discuss the role of P values in multiple testing to associate a large number of genetic or molecular features with a phenotypic variable of interest in biomedical omics studies. For multiple tests in such association analyses, we distinguish those conducted for confirmatory purpose, as seen in genome-wide association studies to determine disease-associated variants, from those for exploratory screening of associated features. For the latter, exploratory analysis, we discuss application of the ROC curve analysis used in diagnostic medicine, as an alternative, but more relevant framework, rather than the standard framework based on multiple testing that controls false positives only. Finally, partly based on arguments made in the field of omics studies, we make some comments on future endeavors by statisticians to disseminate discussions given in the ASA’s Statement on P -Values (Wasserstein and Lazar, 2016, The American Statistician, 70, 129-133) to improve statistical practice in various scientific fields.