Chatbots in education: Outperforming students but perceived as less trustworthy

IF 3.9 1区心理学 Q1 PSYCHOLOGY, EDUCATIONAL

Contemporary Educational Psychology Pub Date : 2025-04-22 DOI:10.1016/j.cedpsych.2025.102373

Martin Laun , Leonard Puderbach , Katharina Hirt , Eva L. Wyss , Daniel Friemert , Ulrich Hartmann , Fabian Wolff

{"title":"Chatbots in education: Outperforming students but perceived as less trustworthy","authors":"Martin Laun , Leonard Puderbach , Katharina Hirt , Eva L. Wyss , Daniel Friemert , Ulrich Hartmann , Fabian Wolff","doi":"10.1016/j.cedpsych.2025.102373","DOIUrl":null,"url":null,"abstract":"<div><div>In recent years, chatbots have emerged as potential individual assistance systems in educational settings. However, their successful implementation depends on their ability to deliver high-quality outcomes and gain students’ trust. Against this background, this study investigated the perceived trustworthiness of chatbots in nursing education and whether this trust was justified in light of chatbots’ actual performance in care plan creation. We conducted a study with 189 vocational nursing students who created medical care plans under two conditions: without ChatGPT (GPT-4) and with ChatGPT assistance. Additionally, ChatGPT was used to develop care plans without student involvement. Expert evaluations of these care plans allowed us to compare quality across conditions. We also examined students’ trust by having them evaluate another care plan before and after experimentally manipulating information about its source (chatbot or peer-generated). Statistical analyses revealed that source disclosure significantly affected students’ trust: Care plans believed to be chatbot-generated experienced a significant decrease in trust, while those attributed to peers showed no significant change. However, analyses of expert evaluations showed that ChatGPT-generated plans were of higher quality than those created by students, even when students used ChatGPT assistance. This discrepancy between perceived trustworthiness and actual performance of chatbots indicates that students’ skepticism toward chatbot-generated content is, to some extent, exaggerated. While ChatGPT can enhance students’ care plan quality, our findings emphasize the importance of addressing trust issues in educational settings where chatbots are implemented.</div></div>","PeriodicalId":10635,"journal":{"name":"Contemporary Educational Psychology","volume":"81 ","pages":"Article 102373"},"PeriodicalIF":3.9000,"publicationDate":"2025-04-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Contemporary Educational Psychology","FirstCategoryId":"102","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0361476X25000384","RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"PSYCHOLOGY, EDUCATIONAL","Score":null,"Total":0}

引用次数: 0

Abstract

In recent years, chatbots have emerged as potential individual assistance systems in educational settings. However, their successful implementation depends on their ability to deliver high-quality outcomes and gain students’ trust. Against this background, this study investigated the perceived trustworthiness of chatbots in nursing education and whether this trust was justified in light of chatbots’ actual performance in care plan creation. We conducted a study with 189 vocational nursing students who created medical care plans under two conditions: without ChatGPT (GPT-4) and with ChatGPT assistance. Additionally, ChatGPT was used to develop care plans without student involvement. Expert evaluations of these care plans allowed us to compare quality across conditions. We also examined students’ trust by having them evaluate another care plan before and after experimentally manipulating information about its source (chatbot or peer-generated). Statistical analyses revealed that source disclosure significantly affected students’ trust: Care plans believed to be chatbot-generated experienced a significant decrease in trust, while those attributed to peers showed no significant change. However, analyses of expert evaluations showed that ChatGPT-generated plans were of higher quality than those created by students, even when students used ChatGPT assistance. This discrepancy between perceived trustworthiness and actual performance of chatbots indicates that students’ skepticism toward chatbot-generated content is, to some extent, exaggerated. While ChatGPT can enhance students’ care plan quality, our findings emphasize the importance of addressing trust issues in educational settings where chatbots are implemented.

查看原文本刊更多论文

教育领域的聊天机器人：表现优于学生，但被认为不那么值得信赖

近年来，聊天机器人已经成为教育环境中潜在的个人辅助系统。然而，他们的成功实施取决于他们提供高质量成果和获得学生信任的能力。在此背景下，本研究调查了聊天机器人在护理教育中的感知可信度，以及根据聊天机器人在护理计划制定中的实际表现，这种信任是否合理。我们对189名职业护理专业的学生进行了一项研究，他们在两种情况下制定了医疗保健计划：没有ChatGPT （GPT-4）和有ChatGPT帮助。此外，ChatGPT被用于制定没有学生参与的护理计划。对这些护理计划的专家评估使我们能够比较不同条件下的质量。我们还通过实验操纵信息来源（聊天机器人或同伴生成）之前和之后，让学生评估另一个护理计划，来检验学生的信任。统计分析显示，来源披露显著影响了学生的信任：被认为是聊天机器人生成的护理计划的信任度显著下降，而被认为是同伴生成的护理计划的信任度没有显著变化。然而，专家评估的分析表明，即使学生使用ChatGPT辅助，ChatGPT生成的计划也比学生创建的计划质量更高。聊天机器人的感知可信度与实际表现之间的差异表明，学生对聊天机器人生成内容的怀疑在某种程度上被夸大了。虽然ChatGPT可以提高学生护理计划的质量，但我们的研究结果强调了在实施聊天机器人的教育环境中解决信任问题的重要性。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Contemporary Educational Psychology PSYCHOLOGY, EDUCATIONAL-

CiteScore

16.50

自引率

3.90%

发文量

期刊介绍： Contemporary Educational Psychology is a scholarly journal that publishes empirical research from various parts of the world. The research aims to substantially advance, extend, or re-envision the ongoing discourse in educational psychology research and practice. To be considered for publication, manuscripts must be well-grounded in a comprehensive theoretical and empirical framework. This framework should raise critical and timely questions that educational psychology currently faces. Additionally, the questions asked should be closely related to the chosen methodological approach, and the authors should provide actionable implications for education research and practice. The journal seeks to publish manuscripts that offer cutting-edge theoretical and methodological perspectives on critical and timely education questions. The journal is abstracted and indexed in various databases, including Contents Pages in Education, Australian Educational Index, Current Contents, EBSCOhost, Education Index, ERA, PsycINFO, Sociology of Education Abstracts, PubMed/Medline, BIOSIS Previews, and others.