Student Reactions to Bots on Course Q&A Platform

Motivation Bots can alleviate the workload of instructors supporting students in large course Q&A platforms, but it's not clear whether students will be receptive to the use of automated assistants in this setting. Objectives We aim to observe student reactions when they encounter bot-generated follow-ups to Q&A board posts. We investigate the effect of revealing that a bot, rather than a human, is suggesting that the current post is a duplicate. Methods Our bot revealed or hid its bot identity when suggesting duplicate posts, with the condition selected randomly. We observed students' reactions in both conditions. A post-course survey was distributed to collect students' demographic data, previous experiences with bots, and attitudes toward our bot. Results We observed a slight increase in students' response rate when the bot hid its identity. We compared the positive response rate in both conditions and did not find evidence suggesting that students had less trust in bot-generated answers. From the survey, we only saw minimal direct evidence that students might mistrust the bot: 7 of 59 students reported worries about receiving an inaccurate bot-generated answer. Other students were concerned that they would not receive attention from an instructor. Discussion We did not find evidence that revealing the bot's identity has a negative impact on student reactions. However, future bot design should consider the emotional impact of deploying a bot as there may be negative emotional effects to receiving a bot-generated response.