Co-Coding Classroom Dialogue: A Single Researcher Case Study of ChatGPT-Assisted Analysis in Science Education

IF 5.1 2区教育学 Q1 EDUCATION & EDUCATIONAL RESEARCH

Journal of Computer Assisted Learning Pub Date : 2025-07-02 DOI:10.1111/jcal.70089

Eunhye Shin

{"title":"Co-Coding Classroom Dialogue: A Single Researcher Case Study of ChatGPT-Assisted Analysis in Science Education","authors":"Eunhye Shin","doi":"10.1111/jcal.70089","DOIUrl":null,"url":null,"abstract":"<div>\n \n \n <section>\n \n <h3> Background</h3>\n \n <p>Analysing classroom dialogue is a widely used approach for understanding students' learning, often requiring team-based collaborative research. This presents a challenge for single researchers due to the labour-intensive nature of the process. Emerging advancements in large language models (LLMs) such as ChatGPT, enhance qualitative research, particularly in inductive and deductive coding tasks.</p>\n </section>\n \n <section>\n \n <h3> Objectives</h3>\n \n <p>This study investigates the feasibility of a single researcher, the author of this study, collaborating with ChatGPT-4o for qualitative coding of classroom dialogue data. The goal is to develop effective human–ChatGPT co-coding methods and explore how such collaboration can enhance qualitative coding practices and provide insights into students' dialogue patterns.</p>\n </section>\n \n <section>\n \n <h3> Methods</h3>\n \n <p>The study analysed 1287 utterances from middle school science classes using a mixed-method approach. A new codebook was developed through an inductive process using ChatGPT, followed by deductive coding conducted by both the researcher and ChatGPT. Kappa values were compared between human–human and human–ChatGPT coding. Disagreements in code assignments were resolved by the researcher, with reference to ChatGPT's rationale. Coded utterances were analysed using ordered network analysis (ONA) to visualise dialogue patterns in classes.</p>\n </section>\n \n <section>\n \n <h3> Results and Conclusions</h3>\n \n <p>The coding conducted by the researcher and ChatGPT resulted in a Cohen's kappa of 0.56, with the highest level of disagreement observed in the category of Meta-cognition. The inductively co-developed codebook helped uncover students dialogue patterns during experimental activities. Although ChatGPT exhibited limitations in interpreting nuanced and context-dependent utterances, the findings highlight its potential as a valuable collaborator for solo researchers by supporting cognitive processes such as reflective interpretation and the development of new perspectives.</p>\n </section>\n </div>","PeriodicalId":48071,"journal":{"name":"Journal of Computer Assisted Learning","volume":"41 4","pages":""},"PeriodicalIF":5.1000,"publicationDate":"2025-07-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/jcal.70089","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Computer Assisted Learning","FirstCategoryId":"95","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1111/jcal.70089","RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}

引用次数: 0

Abstract

Background

Analysing classroom dialogue is a widely used approach for understanding students' learning, often requiring team-based collaborative research. This presents a challenge for single researchers due to the labour-intensive nature of the process. Emerging advancements in large language models (LLMs) such as ChatGPT, enhance qualitative research, particularly in inductive and deductive coding tasks.

Objectives

This study investigates the feasibility of a single researcher, the author of this study, collaborating with ChatGPT-4o for qualitative coding of classroom dialogue data. The goal is to develop effective human–ChatGPT co-coding methods and explore how such collaboration can enhance qualitative coding practices and provide insights into students' dialogue patterns.

Methods

The study analysed 1287 utterances from middle school science classes using a mixed-method approach. A new codebook was developed through an inductive process using ChatGPT, followed by deductive coding conducted by both the researcher and ChatGPT. Kappa values were compared between human–human and human–ChatGPT coding. Disagreements in code assignments were resolved by the researcher, with reference to ChatGPT's rationale. Coded utterances were analysed using ordered network analysis (ONA) to visualise dialogue patterns in classes.

Results and Conclusions

The coding conducted by the researcher and ChatGPT resulted in a Cohen's kappa of 0.56, with the highest level of disagreement observed in the category of Meta-cognition. The inductively co-developed codebook helped uncover students dialogue patterns during experimental activities. Although ChatGPT exhibited limitations in interpreting nuanced and context-dependent utterances, the findings highlight its potential as a valuable collaborator for solo researchers by supporting cognitive processes such as reflective interpretation and the development of new perspectives.

Abstract Image

查看原文本刊更多论文

共同编码课堂对话：科学教育中chatgpt辅助分析的单个研究者案例研究

分析课堂对话是一种广泛使用的理解学生学习的方法，通常需要基于团队的合作研究。由于该过程的劳动密集型性质，这对单个研究人员提出了挑战。大型语言模型（llm）的新进展，如ChatGPT，加强了定性研究，特别是在归纳和演绎编码任务方面。本研究探讨了一名研究者（本研究的作者）与chatgpt - 40合作对课堂对话数据进行定性编码的可行性。目标是开发有效的人类-聊天-语言共同编码方法，并探索这种合作如何增强定性编码实践，并提供对学生对话模式的见解。方法采用混合方法对中学科学课上的1287个话语进行分析。通过使用ChatGPT的归纳过程开发了一个新的密码本，然后由研究人员和ChatGPT进行演绎编码。比较了人-人编码和人- chatgpt编码的Kappa值。在代码分配上的分歧由研究人员根据ChatGPT的基本原理来解决。使用有序网络分析（ONA）对编码话语进行分析，以可视化课堂对话模式。结果与结论研究者与ChatGPT进行编码得到的Cohen’s kappa为0.56，其中元认知类别的不一致程度最高。归纳共同开发的密码本有助于揭示学生在实验活动中的对话模式。尽管ChatGPT在解释细微差别和上下文相关的话语方面表现出局限性，但研究结果强调了它作为独立研究人员有价值的合作者的潜力，它支持诸如反思性解释和新视角的发展等认知过程。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Journal of Computer Assisted Learning EDUCATION & EDUCATIONAL RESEARCH-

CiteScore

9.70

自引率

6.00%

发文量

116

期刊介绍： The Journal of Computer Assisted Learning is an international peer-reviewed journal which covers the whole range of uses of information and communication technology to support learning and knowledge exchange. It aims to provide a medium for communication among researchers as well as a channel linking researchers, practitioners, and policy makers. JCAL is also a rich source of material for master and PhD students in areas such as educational psychology, the learning sciences, instructional technology, instructional design, collaborative learning, intelligent learning systems, learning analytics, open, distance and networked learning, and educational evaluation and assessment. This is the case for formal (e.g., schools), non-formal (e.g., workplace learning) and informal learning (e.g., museums and libraries) situations and environments. Volumes often include one Special Issue which these provides readers with a broad and in-depth perspective on a specific topic. First published in 1985, JCAL continues to have the aim of making the outcomes of contemporary research and experience accessible. During this period there have been major technological advances offering new opportunities and approaches in the use of a wide range of technologies to support learning and knowledge transfer more generally. There is currently much emphasis on the use of network functionality and the challenges its appropriate uses pose to teachers/tutors working with students locally and at a distance. JCAL welcomes: -Empirical reports, single studies or programmatic series of studies on the use of computers and information technologies in learning and assessment -Critical and original meta-reviews of literature on the use of computers for learning -Empirical studies on the design and development of innovative technology-based systems for learning -Conceptual articles on issues relating to the Aims and Scope