Coexisting with ChatGPT: Evaluating a tool for AI-based paper revision

Q1 Arts and Humanities

Computers and Composition Pub Date : 2025-03-11 DOI:10.1016/j.compcom.2025.102923

Matt Manierre, Lisa Propst, Alex Cohen, JoAnn Rogers

{"title":"Coexisting with ChatGPT: Evaluating a tool for AI-based paper revision","authors":"Matt Manierre, Lisa Propst, Alex Cohen, JoAnn Rogers","doi":"10.1016/j.compcom.2025.102923","DOIUrl":null,"url":null,"abstract":"<div><div>AI based tools such as ChatGPT have presented many challenges to educators since they entered the scene in 2022. We present our effort to coexist with ChatGPT in the classroom, developing an exercise for first year writing students to use ChatGPT while revising papers. The effectiveness of this activity was determined using pretest/posttest surveys (<em>n</em> = 64 and 53) and one- page reflective essays. Survey results indicated that students had largely positive appraisals of the different elements of the exercise, describing them as useful without reducing their appreciation of writing as an essential skill for the future. Yet, student writing self-efficacy also did not improve after working with ChatGPT. Qualitative responses were often positive but students frequently reported frustrations with ChatGPT rewriting work when told not to and providing only generic feedback. We offer our exercise as a means to engage students in critical thought about ChatGPT's uses, limitations, and implications for academic integrity. We suggest ways to iterate on this tool and to incorporate it in future work but emphasize that students must be taught to use AI tools with considerable skepticism.</div></div>","PeriodicalId":35773,"journal":{"name":"Computers and Composition","volume":"76 ","pages":"Article 102923"},"PeriodicalIF":0.0000,"publicationDate":"2025-03-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Computers and Composition","FirstCategoryId":"1085","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S8755461525000106","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"Arts and Humanities","Score":null,"Total":0}

引用次数: 0

Abstract

AI based tools such as ChatGPT have presented many challenges to educators since they entered the scene in 2022. We present our effort to coexist with ChatGPT in the classroom, developing an exercise for first year writing students to use ChatGPT while revising papers. The effectiveness of this activity was determined using pretest/posttest surveys (n = 64 and 53) and one- page reflective essays. Survey results indicated that students had largely positive appraisals of the different elements of the exercise, describing them as useful without reducing their appreciation of writing as an essential skill for the future. Yet, student writing self-efficacy also did not improve after working with ChatGPT. Qualitative responses were often positive but students frequently reported frustrations with ChatGPT rewriting work when told not to and providing only generic feedback. We offer our exercise as a means to engage students in critical thought about ChatGPT's uses, limitations, and implications for academic integrity. We suggest ways to iterate on this tool and to incorporate it in future work but emphasize that students must be taught to use AI tools with considerable skepticism.

查看原文本刊更多论文

与ChatGPT共存：评估基于人工智能的论文修改工具

自2022年出现以来，ChatGPT等基于人工智能的工具给教育工作者带来了许多挑战。我们展示了在课堂上与ChatGPT共存的努力，为一年级写作学生开发了一个练习，让他们在修改论文时使用ChatGPT。这项活动的有效性是通过测试前/测试后调查（n = 64和53）和一页的反思性论文来确定的。调查结果表明，学生们对这个练习的不同元素大多有积极的评价，认为它们很有用，但并不会降低他们对写作作为未来一项基本技能的欣赏。然而，使用ChatGPT后，学生的写作自我效能也没有提高。定性反应通常是积极的，但学生们经常报告说，当ChatGPT被告知不要重写作业时，他们会感到沮丧，而且只提供一般性的反馈。我们提供我们的练习作为一种手段，让学生们批判性地思考ChatGPT的用途、局限性和对学术诚信的影响。我们建议迭代此工具并将其纳入未来的工作中，但强调必须教学生以相当大的怀疑态度使用人工智能工具。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Computers and Composition Arts and Humanities-Language and Linguistics

CiteScore

4.30

自引率

0.00%

发文量

审稿时长

25 days

期刊介绍： Computers and Composition: An International Journal is devoted to exploring the use of computers in writing classes, writing programs, and writing research. It provides a forum for discussing issues connected with writing and computer use. It also offers information about integrating computers into writing programs on the basis of sound theoretical and pedagogical decisions, and empirical evidence. It welcomes articles, reviews, and letters to the Editors that may be of interest to readers, including descriptions of computer-aided writing and/or reading instruction, discussions of topics related to computer use of software development; explorations of controversial ethical, legal, or social issues related to the use of computers in writing programs.