An empirical investigation of judgment feedback and computerized decision support in a prediction task

Abstract

This study examines the effects on judgment accuracy of cognitive and outcome feedback provided using a computerized decision support tool. Five feedback conditions were examined in a two-stage experiment utilizing 294 participants: an outcome feedback condition, two cognitive feedback conditions (judgment policy feedback and model predictions feedback), and two joint feedback conditions (judgment policy plus outcome feedback, and model predictions plus outcome feedback). In the first stage, decision makers specified the judgment policies (i.e. cue weights and function forms) that they believed they would use in making their earnings predictions. They were then asked to forecast earnings per share for several companies based on average earnings for the last three years, current year gross margin percentage, quick ratio and eamings yield. Using appropriately modified end-user software, feedback was then provided to all participants, except those receiving outcome feedback only. Judgment policy feedback consisted of informing decision makers of the cue weights and function forms underlying their actual predictions, while model predictions feedback consisted of earnings predictions generated from the decision makers' stated judgment policies. In the second stage, decision makers revised or retained their original judgment policies and then made another set of earnings predictions. Outcome feedback, consisting of information about the actual earnings attained by the companies, was then provided to participants in the outcome feedback and joint feedback conditions. This process was then repeated for a new set of companies to determine how the various forms of feedback influenced judgment accuracy. Results indicated that providing decision makers with either type of cognitive feedback, relative to providing outcome feedback, contributed to improvements in judgment accuracy. There were no significant differences between the judgment accuracy of the cognitive feedback conditions and of the respective joint feedback conditions, indicating that adding outcome feedback did not enhance judgment accuracy. Results also suggested that model predictions feedback may be more effective than judgment policy feedback, which in turn is superior to outcome feedback. All cognitive feedback conditions, relative to outcome feedback only, also demonstrated convergence between stated model predictions and actual predictions. These results are discussed in terms of implications for the design of decision support systems for individual judgment tasks.