Telephone Games: How Issues With Common Citation Practices Established the Inverse Efficiency Score.

Abstract

In many experimental-psychological studies, either the speed or the accuracy of responses could be the primary dependent variable of interest. Often, there is no straightforward choice and this decision is particularly difficult if there are signs of variation in speed-accuracy tradeoff across conditions and/or participants. An apparently well-validated solution to this conundrum is the inverse efficiency score (IES). The general application of IES has never actually been endorsed in the cited sources and recent validation studies show that it does not serve well any of the major purposes to which it has been applied. Its popularity may even have hampered progress on the important methodological issue of how to best combine speed and accuracy measures. The present article traces the history of IES via a literature review and with the aid of contemporary witnesses. This exercise reveals a flaw in the academic culture that could be improved by increasing the value of either checking the original sources or instead being honest about the reliance on second-hand information. The story of IES also exemplifies how ignoring fundamental methodological problems makes practitioners resort to the most accessible solutions, often relying on a simplified appeal to authority and little else.