Strategic Manipulation of Empirical Tests

Theories can be produced by experts seeking a reputation for having knowledge. Hence, a tester could anticipate that theories may have been strategically produced by uninformed experts who want to pass an empirical test. We show that, with no restriction on the domain of permissible theories, strategic experts cannot be discredited for an arbitrary but given number of periods, no matter which test is used (provided that the test does not reject the actual data-generating process). Natural ways around this impossibility result include (1) assuming that unbounded data sets are available and (2) restricting the domain of permissible theories (opening the possibility that the actual data-generating process is rejected out-of-hand). In both cases, it is possible to dismiss strategic experts but only to a limited extent. These results show significant limits on what data can accomplish when experts produce theories strategically.