Conditional Hypothesis Testing

Abstract

When testing multiple hypotheses, conventional techniques used for reducing false positives require all tests to be pre-specified and do not account for correlation between p-values. This makes them incompatible with sequential modelling techniques, where models are built one-at-a-time and future models benefit from the insight of previous testing. We propose here a technique for adjusting future tests to in-corporate prior information and show that this reduces to replacing the likelihood function with the conditional likelihood. A numerical algorithm is also developed that uses Monte Carlo integration to efficiently compute conditional acceptance regions from conditional sizes.