Semantic Distance and the Alternate Uses Task: Recommendations for Reliable Automated Assessment of Originality

ABSTRACT Semantic distance is increasingly used for automated scoring of originality on divergent thinking tasks, such as the Alternate Uses Task (AUT). Despite some psychometric support for semantic distance – including positive correlations with human creativity ratings – additional work is needed to optimize its reliability and validity, including identifying maximally reliable items (objects) for AUT administration. We identify a set of 13 AUT items based on a systematic item-selection strategy (belt, brick, broom, bucket, candle, clock, comb, knife, lamp, pencil, pillow, purse, sock). This item-set resulted in acceptable reliability estimates and was found to be moderately related to both human creativity ratings and a creative personality factor (Study 1). These results replicated in a new sample of Participants (Study 2). We conclude with the following recommendations for reliable and valid assessment of AUT originality using semantic distance: 1) make choices based on theoretical/practical considerations, 2) administer (some or all of) the 13 items from this study; 3) if other items must be used, avoid compound words as AUT items (e.g., guitar string); 4) include as many AUT items as time permits; 5) instruct participants to “be creative”; and 6) address fluency confounds that conflate idea quantity and quality (e.g., via max scoring).