Statistical Power and Search Intensity in Hit Rates Tests of Discrimination

Abstract

The statistical power of a hit rates test of taste-based discrimination varies markedly with the parameters of the application. For the presentation of multiple tests, power should be reported alongside p-values. Tests should also adjust for differences in search intensity whenever possible. If not, differential search intensity across groups will bias results. Theoretical bounds on the bias are wider than commonly observed differences in hit rates, but a simple empirical adjustment provides a valid test when data contain a discrete indicator of search intensity.