{"title":"分析非传统学生的ChatGPT互动、参与、自我效能和表现：一种混合方法的方法","authors":"Mohan Yang, Shiyan Jiang, Belle Li, Kristin Herman, Tian Luo, Shanan Chappell Moots, Nolan Lovett","doi":"10.1111/bjet.13588","DOIUrl":null,"url":null,"abstract":"<div>\n \n <section>\n \n <p>Generative artificial intelligence brings opportunities and unique challenges to nontraditional higher education students, stemming, in part, from the experience of the digital divide. Providing access and practice is critical to bridge this divide and equip students with needed digital competencies. This mixed-methods study investigated how nontraditional higher education students interact with ChatGPT in multiple courses and examined relationships between ChatGPT interactions, engagement, self-efficacy and performance. Data were collected from 73 undergraduate and graduate students through chat logs, course reflections and artefacts, surveys and interviews. ChatGPT interactions were analysed using four metrics: prompt number, depth of knowledge (DoK), prompt relevance and originality. Results showed that ChatGPT prompt numbers (<i>β</i> = 0.256, <i>p</i> < 0.03) and engagement (<i>β</i> = 0.267, <i>p</i> < 0.05) significantly predicted performance, while self-efficacy did not. Students' DoK (<i>r</i> = 0.40, <i>p</i> < 0.01) and prompt relevance (<i>r</i> = 0.42, <i>p</i> < 0.01) were positively correlated with performance. Text mining analysis identified distinct interaction patterns, with ‘strategic inquirers’ demonstrating significantly higher performance than ‘exploratory inquirers’ through more sophisticated follow-up questioning. Qualitative findings revealed that while most students were first-time ChatGPT users who initially showed resistance, they developed growing acceptance. Still, students tended to use ChatGPT sparingly and, even then, as only a starting point for assignments. The study highlights the need for targeted guidance in prompt engineering and AI literacy training to help nontraditional higher education students leverage ChatGPT more effectively for higher-order thinking tasks.</p>\n </section>\n \n <section>\n \n <div>\n \n <div>\n \n <h3>Practitioner notes</h3>\n <p>What is already known about this topic\n\n </p><ul>\n \n <li>Nontraditional students face unique challenges in higher education, such as limited technological literacy and digital access.</li>\n \n <li>The emergence of generative AI tools presents both opportunities and challenges for addressing educational disparities.</li>\n \n <li>Existing studies on AI implementation predominantly focus on traditional students.</li>\n </ul>\n <p>What this paper adds\n\n </p><ul>\n \n <li>Empirical evidence of how nontraditional students interact with ChatGPT through multiple metrics (prompt number, DoK, relevance and originality).</li>\n \n <li>Distinct interaction patterns and their relationship to performance outcomes.</li>\n \n <li>The relationship among ChatGPT interactions, engagement, self-efficacy and performance.</li>\n </ul>\n <p>Implications for practice and/or policy\n\n </p><ul>\n \n <li>Need for explicit instruction in prompt engineering as a critical skill for higher-order thinking.</li>\n \n <li>Importance of providing targeted technology training and self-paced learning resources for nontraditional students.</li>\n \n <li>Value of developing comprehensive AI literacy training that addresses both tool capabilities and limitations.</li>\n </ul>\n </div>\n </div>\n </section>\n </div>","PeriodicalId":48315,"journal":{"name":"British Journal of Educational Technology","volume":"56 5","pages":"1973-2000"},"PeriodicalIF":8.1000,"publicationDate":"2025-04-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://bera-journals.onlinelibrary.wiley.com/doi/epdf/10.1111/bjet.13588","citationCount":"0","resultStr":"{\"title\":\"Analysing nontraditional students' ChatGPT interaction, engagement, self-efficacy and performance: A mixed-methods approach\",\"authors\":\"Mohan Yang, Shiyan Jiang, Belle Li, Kristin Herman, Tian Luo, Shanan Chappell Moots, Nolan Lovett\",\"doi\":\"10.1111/bjet.13588\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div>\\n \\n <section>\\n \\n <p>Generative artificial intelligence brings opportunities and unique challenges to nontraditional higher education students, stemming, in part, from the experience of the digital divide. Providing access and practice is critical to bridge this divide and equip students with needed digital competencies. This mixed-methods study investigated how nontraditional higher education students interact with ChatGPT in multiple courses and examined relationships between ChatGPT interactions, engagement, self-efficacy and performance. Data were collected from 73 undergraduate and graduate students through chat logs, course reflections and artefacts, surveys and interviews. ChatGPT interactions were analysed using four metrics: prompt number, depth of knowledge (DoK), prompt relevance and originality. Results showed that ChatGPT prompt numbers (<i>β</i> = 0.256, <i>p</i> < 0.03) and engagement (<i>β</i> = 0.267, <i>p</i> < 0.05) significantly predicted performance, while self-efficacy did not. Students' DoK (<i>r</i> = 0.40, <i>p</i> < 0.01) and prompt relevance (<i>r</i> = 0.42, <i>p</i> < 0.01) were positively correlated with performance. Text mining analysis identified distinct interaction patterns, with ‘strategic inquirers’ demonstrating significantly higher performance than ‘exploratory inquirers’ through more sophisticated follow-up questioning. Qualitative findings revealed that while most students were first-time ChatGPT users who initially showed resistance, they developed growing acceptance. Still, students tended to use ChatGPT sparingly and, even then, as only a starting point for assignments. The study highlights the need for targeted guidance in prompt engineering and AI literacy training to help nontraditional higher education students leverage ChatGPT more effectively for higher-order thinking tasks.</p>\\n </section>\\n \\n <section>\\n \\n <div>\\n \\n <div>\\n \\n <h3>Practitioner notes</h3>\\n <p>What is already known about this topic\\n\\n </p><ul>\\n \\n <li>Nontraditional students face unique challenges in higher education, such as limited technological literacy and digital access.</li>\\n \\n <li>The emergence of generative AI tools presents both opportunities and challenges for addressing educational disparities.</li>\\n \\n <li>Existing studies on AI implementation predominantly focus on traditional students.</li>\\n </ul>\\n <p>What this paper adds\\n\\n </p><ul>\\n \\n <li>Empirical evidence of how nontraditional students interact with ChatGPT through multiple metrics (prompt number, DoK, relevance and originality).</li>\\n \\n <li>Distinct interaction patterns and their relationship to performance outcomes.</li>\\n \\n <li>The relationship among ChatGPT interactions, engagement, self-efficacy and performance.</li>\\n </ul>\\n <p>Implications for practice and/or policy\\n\\n </p><ul>\\n \\n <li>Need for explicit instruction in prompt engineering as a critical skill for higher-order thinking.</li>\\n \\n <li>Importance of providing targeted technology training and self-paced learning resources for nontraditional students.</li>\\n \\n <li>Value of developing comprehensive AI literacy training that addresses both tool capabilities and limitations.</li>\\n </ul>\\n </div>\\n </div>\\n </section>\\n </div>\",\"PeriodicalId\":48315,\"journal\":{\"name\":\"British Journal of Educational Technology\",\"volume\":\"56 5\",\"pages\":\"1973-2000\"},\"PeriodicalIF\":8.1000,\"publicationDate\":\"2025-04-09\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://bera-journals.onlinelibrary.wiley.com/doi/epdf/10.1111/bjet.13588\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"British Journal of Educational Technology\",\"FirstCategoryId\":\"95\",\"ListUrlMain\":\"https://bera-journals.onlinelibrary.wiley.com/doi/10.1111/bjet.13588\",\"RegionNum\":1,\"RegionCategory\":\"教育学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"EDUCATION & EDUCATIONAL RESEARCH\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"British Journal of Educational Technology","FirstCategoryId":"95","ListUrlMain":"https://bera-journals.onlinelibrary.wiley.com/doi/10.1111/bjet.13588","RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}
引用次数: 0
摘要
生成式人工智能为非传统高等教育的学生带来了机遇和独特的挑战，部分源于数字鸿沟的经历。提供机会和实践对于弥合这一鸿沟并使学生具备所需的数字能力至关重要。这项混合方法研究调查了非传统高等教育学生如何在多门课程中与ChatGPT互动，并研究了ChatGPT互动、参与度、自我效能和绩效之间的关系。通过聊天记录、课程反思和人工制品、调查和访谈，从73名本科生和研究生中收集了数据。ChatGPT交互使用四个指标进行分析：提示数，知识深度（DoK），提示相关性和独创性。结果显示，ChatGPT提示数（β = 0.256, p < 0.03）和敬业度（β = 0.267, p < 0.05）显著预测绩效，而自我效能感无显著预测。学生的DoK （r = 0.40, p < 0.01）和提示相关性（r = 0.42, p < 0.01）与成绩呈正相关。文本挖掘分析确定了不同的交互模式，通过更复杂的后续问题，“战略询问者”的表现明显高于“探索性询问者”。定性调查结果显示，虽然大多数学生是第一次使用ChatGPT，最初表现出抵触情绪，但他们逐渐接受了。尽管如此，学生们还是倾向于谨慎地使用ChatGPT，即使这样，也只是作为作业的起点。该研究强调，在快速工程和人工智能素养培训中，需要有针对性的指导，以帮助非传统高等教育的学生更有效地利用ChatGPT完成高阶思维任务。非传统学生在高等教育中面临着独特的挑战，例如有限的技术素养和数字访问。生成式人工智能工具的出现为解决教育差距带来了机遇和挑战。现有的人工智能实施研究主要集中在传统学生身上。本文增加的经验证据表明，非传统学生如何通过多个指标（提示数、DoK、相关性和原创性）与ChatGPT互动。不同的交互模式及其与绩效结果的关系。ChatGPT互动、投入、自我效能和绩效的关系。对实践和/或政策的启示需要明确的指导，将快速工程作为高阶思维的关键技能。为非传统学生提供有针对性的技术培训和自主学习资源的重要性。开发全面的人工智能素养培训，解决工具能力和局限性的价值。
Analysing nontraditional students' ChatGPT interaction, engagement, self-efficacy and performance: A mixed-methods approach
Generative artificial intelligence brings opportunities and unique challenges to nontraditional higher education students, stemming, in part, from the experience of the digital divide. Providing access and practice is critical to bridge this divide and equip students with needed digital competencies. This mixed-methods study investigated how nontraditional higher education students interact with ChatGPT in multiple courses and examined relationships between ChatGPT interactions, engagement, self-efficacy and performance. Data were collected from 73 undergraduate and graduate students through chat logs, course reflections and artefacts, surveys and interviews. ChatGPT interactions were analysed using four metrics: prompt number, depth of knowledge (DoK), prompt relevance and originality. Results showed that ChatGPT prompt numbers (β = 0.256, p < 0.03) and engagement (β = 0.267, p < 0.05) significantly predicted performance, while self-efficacy did not. Students' DoK (r = 0.40, p < 0.01) and prompt relevance (r = 0.42, p < 0.01) were positively correlated with performance. Text mining analysis identified distinct interaction patterns, with ‘strategic inquirers’ demonstrating significantly higher performance than ‘exploratory inquirers’ through more sophisticated follow-up questioning. Qualitative findings revealed that while most students were first-time ChatGPT users who initially showed resistance, they developed growing acceptance. Still, students tended to use ChatGPT sparingly and, even then, as only a starting point for assignments. The study highlights the need for targeted guidance in prompt engineering and AI literacy training to help nontraditional higher education students leverage ChatGPT more effectively for higher-order thinking tasks.
Practitioner notes
What is already known about this topic
Nontraditional students face unique challenges in higher education, such as limited technological literacy and digital access.
The emergence of generative AI tools presents both opportunities and challenges for addressing educational disparities.
Existing studies on AI implementation predominantly focus on traditional students.
What this paper adds
Empirical evidence of how nontraditional students interact with ChatGPT through multiple metrics (prompt number, DoK, relevance and originality).
Distinct interaction patterns and their relationship to performance outcomes.
The relationship among ChatGPT interactions, engagement, self-efficacy and performance.
Implications for practice and/or policy
Need for explicit instruction in prompt engineering as a critical skill for higher-order thinking.
Importance of providing targeted technology training and self-paced learning resources for nontraditional students.
Value of developing comprehensive AI literacy training that addresses both tool capabilities and limitations.
期刊介绍：
BJET is a primary source for academics and professionals in the fields of digital educational and training technology throughout the world. The Journal is published by Wiley on behalf of The British Educational Research Association (BERA). It publishes theoretical perspectives, methodological developments and high quality empirical research that demonstrate whether and how applications of instructional/educational technology systems, networks, tools and resources lead to improvements in formal and non-formal education at all levels, from early years through to higher, technical and vocational education, professional development and corporate training.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
