Extending Multiple Testing With Unknown Test Dependency via the CoCo Test: With Applications to Cancer Studies.

Abstract

Multiple testing issues are common in clinical and scientific research, particularly in clinical trials involving multple endpoints. The central challenge lies in controlling the type I error rate ( $\alpha$ -control). The behavior of multiple testing procedures for $\alpha$ -control when the tests are independent or dependent but with a known joint distribution is relatively well known. When the joint distribution of test statistics is unknown, one can still guarantee the $\alpha$ -control, if the positive dependency through stochastic ordering (PDS) condition is satisfied. Despite the frequent occurrence of unknown test dependency in multiple testing and the importance of the PDS condition in endorsing its validity, little do we know about how to verify this condition. Here, we develop a new nonparametric statistical test, called the CoCo test, based on ranked correlation coefficients and a simple, yet effective, algebraic arrangement of the Spearman's $\rho$ and Kendall's $\tau$ , that can validate the PDS condition, through which one can control for $\alpha$ regardless of the prior knowledge of the dependency between test statistics. Simulation studies show that the CoCo test can faithfully detect the violation of the PDS condition or lack thereof. To further evaluate the efficacy of the CoCo test, we apply it to investigate three meta-analyses. Our simulation studies and data analyses encourage one to evaluate the PDS condition during multiple testing, especially when one is uncertain about the relationship between tests, and the proposed CoCo test provides both methodological insights into and a technical device for doing so. An R package cocotest to implement the proposed methodology is available at CRAN.