Optimising the Trade-Off Between Type I and Type II Errors: A Review and Extensions

Abstract

In clinical studies upon which decisions are based there are two types of errors that can be made: a type I error arises when the decision is taken to declare a positive outcome when the truth is in fact negative, and a type II error arises when the decision is taken to declare a negative outcome when the truth is in fact positive. Commonly the primary analysis of such a study entails a two-sided hypothesis test with a type I error rate of 5% and the study is designed to have a sufficiently low type II error rate, for example 10% or 20%. These values are arbitrary and often do not reflect the clinical, or regulatory, context of the study and ignore both the relative costs of making either type of error and the sponsor's prior belief that the drug is superior to either placebo, or a standard of care if relevant. This simplistic approach has recently been challenged by numerous authors both from a frequentist and Bayesian perspective since when resources are constrained there will be a need to consider a trade-off between type I and type II errors. In this paper we review proposals to utilise the trade-off by formally acknowledging the costs to optimise the choice of error rates for simple, point null and alternative hypotheses and extend the results to composite, or interval hypotheses, showing links to the Probability of Success of a clinical study.