The Quality of Expertise

Abstract

Policy-makers and managers often turn to experts when in need of information: because they are more informed than others of the content and quality of current and past research, they should provide the best advice. I show, however, that we should expect experts to be systematically biased, potentially to the point that they are less reliable sources of information than non-experts. This is because the decision to research a question implies a belief that research will be fruitful. If priors about the impact of current work are correct, on average, then those who select into researching a question are optimistic about the quality of current work. In areas that are new, or feature new research technologies (e.g., data sources, technical methods, or paradigms), the selection problem is less important than the benefit of greater knowledge: experts will indeed be experts. In areas that are old and lack new research technologies, there will be significant bias. Furthermore, consistent with a large body of empirical research, this selection problem implies that experts who express greater confidence in their beliefs will be, on average, less accurate. This paper provides many empirical implications for expert accuracy, as well as mechanism design implications for hiring, task assignment, and referee assignment.