AI vs humans in the AUT: Simulations to LLMs

Abstract

This paper reviews studies of proposed creative machines applied to a prototypical creative task, i.e., the Alternative Uses Task (AUT). Although one system (OROC) did simulate some aspects of human strategies for the AUT, most recent attempts have not been simulation-oriented, but rather have used Large Language Model (LLM) systems such as GPT-3 which embody extremely large connectionist networks trained on huge volumes of textual data. Studies reviewed here indicate that LLM based systems are performing on the AUT at near or somewhat above human levels in terms of scores on originality and usefulness. Moreover, similar patterns are found in the data of humans and LLM models in the AUT, such as output order effects and a negative association between originality and value or utility. However, it is concluded that GPT-3 and similar systems, despite generating novel and useful responses, do not display creativity as they lack agency and are purely algorithmic. LLM studies so far in this area have largely been exploratory and future studies should guard against possible training data contamination.