Why area under the curve in hypothesis testing?

Abstract

To compare two different hypothesis testing techniques, researchers use the following heuristic idea: for each technique, they form a curve describing how the probabilities of type I and type II errors are related for this technique, and then compare areas under the resulting curves. In this paper, we provide a justification for this heuristic idea. 1 Formulation of the Problem Type I and type II errors. There are many different techniques for hypothesis testing, i.g., for deciding, based on the observation, whether the original (null) hypothesis is valid or whether this hypothesis has to be rejected (and the alternative hypothesis has to be considered true); see, e.g., [3]. In hypothesis testing, we can have two different types of errors: • a type I error (also known as False Negative) is when the correct null hypothesis is erroneously rejected, while • a type II error (also known as False Positive) is when the false null hypothesis is erroneously accepted. The probability of the type I error is usually denoted by α and the probability of the type II error is usually denoted by β. In different situations, we have different requirements on the allowed probabilities of these two errors. For example, in early cancer diagnostics, when the null hypothesis means no cancer, type I error is not that critical – it simply