Human or Not?: An Experiment With Chatbot Manipulations to Test Machine Heuristics and Political Self-Concepts

Abstract

Chatbots have a growing role to play in political discourse, including in political campaigns, voter mobilization ventures, and dissemination of political news, though chatbots in the political domain are relatively understudied. While testing the machine heuristics and political self-concepts frameworks, we carried out a 2 × 2 experiment where both perceived conversational partner (i.e., bot, human) and topic (i.e., political, casual) were manipulated ( N = 126). During the experiment, participants exchanged chat messages with trained research confederates for 30 min. In support of the machine heuristics and political self-concepts frameworks, participants assigned to human partners reported more positive relationships and higher political interest. Through moderation analysis, liking the partner was found to differ between the perceived partner conditions, with perceived political knowledge varying more in the human conditions. Thus, the experimental findings add nuance to interpersonal (i.e., impression management and social identity theory) and human-computer interaction theories (i.e., machine heuristics and Computers Are Social Actors), and have broader implications for online political interactions and for decisionmakers of online political discourse spaces.