Psychometric modeling of decision making via game play

Abstract

We build a model for the kind of decision making involved in games of strategy such as chess, making it abstract enough to remove essentially all game-specific contingency, and compare it to known psychometric models of test taking, item response, and performance assessment. Decisions are modeled in terms of fallible agents Z faced with possible actions ai whose utilities Ui=U (ai) are not fully apparent. The three main goals of the model are prediction, meaning to infer probabilities Pi for Z to choose ai; intrinsic rating, meaning to assess the skill of a person's actual choices ait over various test items t; and simulation of the distribution of choices by an agent with a specified skill set. We describe and train the model on large data from chess tournament games of different ranks of players, and exemplify its accuracy by applying it to give intrinsic ratings for world championship matches.