A word to the wise: Crafting impactful prompts for ChatGPT

IF 5.6 1区文学 Q1 EDUCATION & EDUCATIONAL RESEARCH

System Pub Date : 2025-06-21 DOI:10.1016/j.system.2025.103756

Myunghwan Hwang , Ki-Ho Lee , Hee-Kyung Lee

{"title":"A word to the wise: Crafting impactful prompts for ChatGPT","authors":"Myunghwan Hwang , Ki-Ho Lee , Hee-Kyung Lee","doi":"10.1016/j.system.2025.103756","DOIUrl":null,"url":null,"abstract":"<div><div>The integration of ChatGPT into higher education has grown rapidly, yet research on students’ prompting behaviors remains limited. This study examines learner-generated prompts, focusing on linguistic features, prompt engineering strategies, and content-related problems, as well as their impact on response quality, measured by learner satisfaction. A total of 238 prompts from ten Korean EFL graduate students in English education were collected through reflection logs and analyzed using one-way ANOVA, correlation analysis, network analysis, and K-means clustering. Results indicate that learners predominantly used concise prompts with minimal strategy, with English prompts yielding higher satisfaction than those in their first language. High-satisfaction prompts incorporated clear, interconnected strategies, while low-satisfaction prompts featured fragmented ones. Key strategies associated with high satisfaction included contextualization and specifying text style, tone, and length, while repetition correlated with dissatisfaction. Furthermore, a thematic analysis identified six main problems that hindered response quality: lack of context, unclear feedback, dual requests, grammatical errors, abstract vocabulary, and excessive constraints. These findings highlight the significance of clear, contextually rich, and strategically crafted prompts in optimizing interactions with ChatGPT, along with the need to equip EFL learners with distinct prompting techniques tailored for AI-assisted language learning, separate from traditional human communication.</div></div>","PeriodicalId":48185,"journal":{"name":"System","volume":"133 ","pages":"Article 103756"},"PeriodicalIF":5.6000,"publicationDate":"2025-06-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"System","FirstCategoryId":"98","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0346251X25001666","RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}

引用次数: 0

Abstract

The integration of ChatGPT into higher education has grown rapidly, yet research on students’ prompting behaviors remains limited. This study examines learner-generated prompts, focusing on linguistic features, prompt engineering strategies, and content-related problems, as well as their impact on response quality, measured by learner satisfaction. A total of 238 prompts from ten Korean EFL graduate students in English education were collected through reflection logs and analyzed using one-way ANOVA, correlation analysis, network analysis, and K-means clustering. Results indicate that learners predominantly used concise prompts with minimal strategy, with English prompts yielding higher satisfaction than those in their first language. High-satisfaction prompts incorporated clear, interconnected strategies, while low-satisfaction prompts featured fragmented ones. Key strategies associated with high satisfaction included contextualization and specifying text style, tone, and length, while repetition correlated with dissatisfaction. Furthermore, a thematic analysis identified six main problems that hindered response quality: lack of context, unclear feedback, dual requests, grammatical errors, abstract vocabulary, and excessive constraints. These findings highlight the significance of clear, contextually rich, and strategically crafted prompts in optimizing interactions with ChatGPT, along with the need to equip EFL learners with distinct prompting techniques tailored for AI-assisted language learning, separate from traditional human communication.

查看原文本刊更多论文

给聪明的人一句话：为ChatGPT制作有影响力的提示

ChatGPT与高等教育的结合发展迅速，但对学生提示行为的研究仍然有限。本研究考察了学习者生成的提示，重点关注语言特征、提示工程策略和内容相关问题，以及它们对响应质量的影响，以学习者满意度来衡量。通过反思日志收集10名韩国英语研究生的238条提示，并采用单因素方差分析、相关分析、网络分析和K-means聚类分析进行分析。结果表明，学习者主要使用简洁的提示，策略最少，英语提示的满意度高于母语提示。高满意度提示包含了清晰的、相互关联的策略，而低满意度提示以碎片化的策略为特征。与高满意度相关的关键策略包括语境化和指定文本风格、语气和长度，而重复与不满意相关。此外，一项专题分析确定了影响回应质量的六个主要问题：缺乏上下文、不明确的反馈、双重请求、语法错误、抽象词汇和过多的限制。这些发现强调了清晰、上下文丰富、精心设计的提示对于优化与ChatGPT的互动的重要性，以及为英语学习者提供独特的提示技术的必要性，这些提示技术是为人工智能辅助语言学习量身定制的，与传统的人类交流分开。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

System Multiple-

CiteScore

8.80

自引率

8.30%

发文量

202

审稿时长

64 days

期刊介绍： This international journal is devoted to the applications of educational technology and applied linguistics to problems of foreign language teaching and learning. Attention is paid to all languages and to problems associated with the study and teaching of English as a second or foreign language. The journal serves as a vehicle of expression for colleagues in developing countries. System prefers its contributors to provide articles which have a sound theoretical base with a visible practical application which can be generalized. The review section may take up works of a more theoretical nature to broaden the background.