Unveiling Patterns of Interaction With Automated Feedback in Writing Mentor and Their Relationships With Use Goals and Writing Outcomes

IF 4.6 2区教育学 Q1 EDUCATION & EDUCATIONAL RESEARCH

Journal of Computer Assisted Learning Pub Date : 2025-02-25 DOI:10.1111/jcal.70014

Yang Jiang, Beata Beigman Klebanov, Jiangang Hao, Paul Deane, Oren E. Livne

{"title":"Unveiling Patterns of Interaction With Automated Feedback in Writing Mentor and Their Relationships With Use Goals and Writing Outcomes","authors":"Yang Jiang, Beata Beigman Klebanov, Jiangang Hao, Paul Deane, Oren E. Livne","doi":"10.1111/jcal.70014","DOIUrl":null,"url":null,"abstract":"<div>\n \n \n <section>\n \n <h3> Background</h3>\n \n <p>Writing is integral to educational success at all levels and to success in the workplace. However, low literacy is a global challenge, and many students lack sufficient skills to be good writers. With the rapid advance of technology, computer-based tools that provide automated feedback are being increasingly developed. However, mixed results have been obtained so far on the effects of writing feedback, and little is known about how various types of feedback are used by users who access feedback with different goals.</p>\n </section>\n \n <section>\n \n <h3> Objectives</h3>\n \n <p>Leveraging both process and response data, the present study investigated students' interactions with feedback in Writing Mentor (WM), an app designed to support academic writing through automated feedback and NLP techniques.</p>\n </section>\n \n <section>\n \n <h3> Methods</h3>\n \n <p>We identified 1857 documents submitted by middle- and high-school students among <i>users-in-the-wild</i> and explored the activities that students engaged in while using WM and the quality of their submitted written products. Students' interaction processes and behaviours were compared across groups of students with different goals for using WM. In addition, sequential pattern mining was applied to identify interesting sequential patterns of student interactions with the writing feedback features that are differentially frequent across goal groups.</p>\n </section>\n \n <section>\n \n <h3> Results and Conclusions</h3>\n \n <p>Results indicated that students with different goals for using WM displayed different patterns of interaction. Furthermore, the effects of the various types of feedback on writing quality vary by students' purposes for using the tool. Findings from this study shed light on the design of computer-based writing tools and automated writing feedback for learners of various needs.</p>\n </section>\n </div>","PeriodicalId":48071,"journal":{"name":"Journal of Computer Assisted Learning","volume":"41 2","pages":""},"PeriodicalIF":4.6000,"publicationDate":"2025-02-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Computer Assisted Learning","FirstCategoryId":"95","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1111/jcal.70014","RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}

引用次数: 0

Abstract

Background

Writing is integral to educational success at all levels and to success in the workplace. However, low literacy is a global challenge, and many students lack sufficient skills to be good writers. With the rapid advance of technology, computer-based tools that provide automated feedback are being increasingly developed. However, mixed results have been obtained so far on the effects of writing feedback, and little is known about how various types of feedback are used by users who access feedback with different goals.

Objectives

Leveraging both process and response data, the present study investigated students' interactions with feedback in Writing Mentor (WM), an app designed to support academic writing through automated feedback and NLP techniques.

Methods

We identified 1857 documents submitted by middle- and high-school students among users-in-the-wild and explored the activities that students engaged in while using WM and the quality of their submitted written products. Students' interaction processes and behaviours were compared across groups of students with different goals for using WM. In addition, sequential pattern mining was applied to identify interesting sequential patterns of student interactions with the writing feedback features that are differentially frequent across goal groups.

Results and Conclusions

Results indicated that students with different goals for using WM displayed different patterns of interaction. Furthermore, the effects of the various types of feedback on writing quality vary by students' purposes for using the tool. Findings from this study shed light on the design of computer-based writing tools and automated writing feedback for learners of various needs.

查看原文本刊更多论文

揭示写作导师与自动反馈的交互模式及其与使用目标和写作结果的关系

写作对于各级教育的成功和职场的成功都是不可或缺的。然而，低文化水平是一个全球性的挑战，许多学生缺乏足够的技能来成为优秀的作家。随着技术的快速发展，提供自动反馈的基于计算机的工具正在日益发展。然而，到目前为止，关于写作反馈的效果，已经获得了好坏参半的结果，对于不同目标的用户如何使用不同类型的反馈，我们知之甚少。利用过程和响应数据，本研究调查了学生在写作导师（WM）中与反馈的互动，WM是一款旨在通过自动反馈和NLP技术支持学术写作的应用程序。方法在野外用户中选取了1857份初高中学生提交的文件，探讨了学生在使用WM时所从事的活动及其提交的书面产品的质量。学生的互动过程和行为在使用WM的不同目标的学生组之间进行比较。此外，顺序模式挖掘应用于识别学生与写作反馈特征交互的有趣顺序模式，这些模式在不同目标组之间的频率不同。结果与结论结果表明，不同WM使用目标的学生表现出不同的互动模式。此外，各种类型的反馈对写作质量的影响因学生使用该工具的目的而异。本研究的研究结果揭示了基于计算机的写作工具的设计和针对不同需求的学习者的自动写作反馈。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Journal of Computer Assisted Learning EDUCATION & EDUCATIONAL RESEARCH-

CiteScore

9.70

自引率

6.00%

发文量

116

期刊介绍： The Journal of Computer Assisted Learning is an international peer-reviewed journal which covers the whole range of uses of information and communication technology to support learning and knowledge exchange. It aims to provide a medium for communication among researchers as well as a channel linking researchers, practitioners, and policy makers. JCAL is also a rich source of material for master and PhD students in areas such as educational psychology, the learning sciences, instructional technology, instructional design, collaborative learning, intelligent learning systems, learning analytics, open, distance and networked learning, and educational evaluation and assessment. This is the case for formal (e.g., schools), non-formal (e.g., workplace learning) and informal learning (e.g., museums and libraries) situations and environments. Volumes often include one Special Issue which these provides readers with a broad and in-depth perspective on a specific topic. First published in 1985, JCAL continues to have the aim of making the outcomes of contemporary research and experience accessible. During this period there have been major technological advances offering new opportunities and approaches in the use of a wide range of technologies to support learning and knowledge transfer more generally. There is currently much emphasis on the use of network functionality and the challenges its appropriate uses pose to teachers/tutors working with students locally and at a distance. JCAL welcomes: -Empirical reports, single studies or programmatic series of studies on the use of computers and information technologies in learning and assessment -Critical and original meta-reviews of literature on the use of computers for learning -Empirical studies on the design and development of innovative technology-based systems for learning -Conceptual articles on issues relating to the Aims and Scope