Sample allocation for multiple attribute selection problems

Abstract

Prior to making a multiple attribute selection decision, a decision-maker may collect information to estimate the value of each attribute for each alternative. In this work, we consider a fixed experimental sample budget and address the problem of how best to allocate this budget across three attributes when the attribute value estimates have a normally distributed measurement error. We illustrate that the allocation choice impacts the decision-maker's ability to select the true best alternative. Through a simulation study we evaluate the performance of a common allocation approach of uniformly distributing the sample budget across the three attributes. We compare these results to the performance of several allocation rules that leverage the decision-maker's preferences. We found that incorporating the decision-maker's preferences into the allocation choice improves the probability of selecting the true best alternative.