Evaluating credit applications: A validation of multiattribute utility weight elicitation techniques

Multiattribute Utility Measurement (MAUM) provides a set of tools and procedures that are designed to aid the decision maker who is faced with decision problems of such complexity and ambiguity that unaided, intuitive judgment is likely to lead to the selection of suboptimal alternatives. Attempts to validate MAUM procedures have been primarily of three types: (1) behavioral tests of axiom systems derived from assumptions about what constitutes reasonable behavior; (2) convergent validation, in which the results of different procedures or even different subjects are compared; and (3) criterion validation, in which judgments and their resultant decisions are compared with some external criterion. From a behavioral point of view, the last of these, criterion validity, is by far the strongest. Past efforts at criterion validation of MAUM have suffered from three limitations: the subjects were not experts, alternative weight elicitation procedures were not compared, and the strength of the criterion used in each case is open to question. The purpose of this experiment was to provide an empirical comparison of a number of alternative MAUM weight elicitation procedures in a situation that offered a meaningful external crtierion along with subjects expert in its use. High quality decisions resulted from weight judgments provided in response to all weight elicitation procedures as long as single dimensions were first individually scaled and then weighted for aggregation. A procedure in which alternatives were rated holistically and weights and single dimension utility functions derived statistically showed poorer quality decisions. Thus, the “divide and conquer” theme of MAUM was upheld.