Credibility and altered communication styles of AI graders in the classroom

IF 5.1 2区教育学 Q1 EDUCATION & EDUCATIONAL RESEARCH

Journal of Computer Assisted Learning Pub Date : 2024-04-04 DOI:10.1111/jcal.12979

Bryan Abendschein, Xialing Lin, Chad Edwards, Autumn Edwards, Varun Rijhwani

{"title":"Credibility and altered communication styles of AI graders in the classroom","authors":"Bryan Abendschein, Xialing Lin, Chad Edwards, Autumn Edwards, Varun Rijhwani","doi":"10.1111/jcal.12979","DOIUrl":null,"url":null,"abstract":"<div>\n \n \n <section>\n \n <h3> Background</h3>\n \n <p>Education is often the primary arena for exploring and integrating new technologies. AI and human-machine communication (HMC) are prevalent in the classroom, yet we are still learning how student perceptions of these tools will impact education.</p>\n </section>\n \n <section>\n \n <h3> Objectives</h3>\n \n <p>We sought to understand student perceptions of credibility related to written feedback attributed to a human or an AI grader (Study One). We also investigated how corrective messages containing verbal immediacy and social support influenced student perceptions of an AI grader's credibility based on feedback in an evaluated essay (Study Two).</p>\n </section>\n \n <section>\n \n <h3> Methods</h3>\n \n <p>We used an online experimental design to assess the perceived credibility of a grader. In Study One, we randomly assigned students (<i>N</i> = 155) to a condition that contained a paragraph they were told was evaluated by a human or an AI grader. In Study Two (<i>N</i> = 222), we investigated ways of increasing perceptions of an AI grader's credibility by writing messages with higher/lower levels of immediacy and social support.</p>\n </section>\n \n <section>\n \n <h3> Results</h3>\n \n <p>In Study One, the students rated both the human and AI grader as credible (yet rated the AI grader lower on goodwill). The data suggest that students in Study Two attributed more goodwill (i.e., caring) to the AI grader when the feedback included more verbal immediacy.</p>\n </section>\n \n <section>\n \n <h3> Conclusions</h3>\n \n <p>Our results highlight the importance of student perceptions and communication styles when integrating technology into education. The two studies imply that students viewed the human and AI graders as competent, caring, and trustworthy, specifically when feedback included more immediacy cues.</p>\n </section>\n </div>","PeriodicalId":48071,"journal":{"name":"Journal of Computer Assisted Learning","volume":"40 4","pages":"1766-1776"},"PeriodicalIF":5.1000,"publicationDate":"2024-04-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Computer Assisted Learning","FirstCategoryId":"95","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1111/jcal.12979","RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}

引用次数: 0

Abstract

Background

Education is often the primary arena for exploring and integrating new technologies. AI and human-machine communication (HMC) are prevalent in the classroom, yet we are still learning how student perceptions of these tools will impact education.

Objectives

We sought to understand student perceptions of credibility related to written feedback attributed to a human or an AI grader (Study One). We also investigated how corrective messages containing verbal immediacy and social support influenced student perceptions of an AI grader's credibility based on feedback in an evaluated essay (Study Two).

Methods

We used an online experimental design to assess the perceived credibility of a grader. In Study One, we randomly assigned students (N = 155) to a condition that contained a paragraph they were told was evaluated by a human or an AI grader. In Study Two (N = 222), we investigated ways of increasing perceptions of an AI grader's credibility by writing messages with higher/lower levels of immediacy and social support.

Results

In Study One, the students rated both the human and AI grader as credible (yet rated the AI grader lower on goodwill). The data suggest that students in Study Two attributed more goodwill (i.e., caring) to the AI grader when the feedback included more verbal immediacy.

Conclusions

Our results highlight the importance of student perceptions and communication styles when integrating technology into education. The two studies imply that students viewed the human and AI graders as competent, caring, and trustworthy, specifically when feedback included more immediacy cues.

查看原文本刊更多论文

AI 年级学生在课堂上的可信度和改变的交流方式

背景教育通常是探索和整合新技术的主要领域。我们试图了解学生对人类或人工智能评分员的书面反馈的可信度的看法（研究一）。我们还调查了包含口头即时性和社会支持的纠正信息如何影响学生根据评价文章中的反馈对人工智能评分员可信度的看法（研究二）。方法我们采用在线实验设计来评估评分员的可信度。在 "研究一 "中，我们将学生（人数= 155）随机分配到一个条件中，其中包含一段他们被告知由人类或人工智能评分员进行评估的文章。在研究二（N = 222）中，我们研究了通过编写即时性和社会支持程度较高/较低的信息来提高人工智能评分员可信度的方法。研究结果在研究一中，学生对人类和人工智能评分员的评分都是可信的（但对人工智能评分员的好感度评分较低）。数据表明，在研究二中，当反馈包含更多的口头即时性时，学生对人工智能评分员的好感（即关心）更高。这两项研究表明，学生认为人类和人工智能评分员都是称职的、有爱心的和值得信赖的，特别是当反馈包含更多即时性提示时。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Journal of Computer Assisted Learning EDUCATION & EDUCATIONAL RESEARCH-

CiteScore

9.70

自引率

6.00%

发文量

116

期刊介绍： The Journal of Computer Assisted Learning is an international peer-reviewed journal which covers the whole range of uses of information and communication technology to support learning and knowledge exchange. It aims to provide a medium for communication among researchers as well as a channel linking researchers, practitioners, and policy makers. JCAL is also a rich source of material for master and PhD students in areas such as educational psychology, the learning sciences, instructional technology, instructional design, collaborative learning, intelligent learning systems, learning analytics, open, distance and networked learning, and educational evaluation and assessment. This is the case for formal (e.g., schools), non-formal (e.g., workplace learning) and informal learning (e.g., museums and libraries) situations and environments. Volumes often include one Special Issue which these provides readers with a broad and in-depth perspective on a specific topic. First published in 1985, JCAL continues to have the aim of making the outcomes of contemporary research and experience accessible. During this period there have been major technological advances offering new opportunities and approaches in the use of a wide range of technologies to support learning and knowledge transfer more generally. There is currently much emphasis on the use of network functionality and the challenges its appropriate uses pose to teachers/tutors working with students locally and at a distance. JCAL welcomes: -Empirical reports, single studies or programmatic series of studies on the use of computers and information technologies in learning and assessment -Critical and original meta-reviews of literature on the use of computers for learning -Empirical studies on the design and development of innovative technology-based systems for learning -Conceptual articles on issues relating to the Aims and Scope