When Generative AI Meets Socratic Method: Investigating Programming Learning Dynamics Through Behaviours, Interaction Qualities and Perceptions

IF 4.6 2区教育学 Q1 EDUCATION & EDUCATIONAL RESEARCH

Journal of Computer Assisted Learning Pub Date : 2026-02-17 DOI:10.1002/jcal.70210

Dan Sun, Yi Zheng, Jie Xu, Zhanshan Yang

{"title":"When Generative AI Meets Socratic Method: Investigating Programming Learning Dynamics Through Behaviours, Interaction Qualities and Perceptions","authors":"Dan Sun, Yi Zheng, Jie Xu, Zhanshan Yang","doi":"10.1002/jcal.70210","DOIUrl":null,"url":null,"abstract":"<div>\n \n \n <section>\n \n <h3> Background</h3>\n \n <p>The integration of generative artificial intelligence (GAI) tools like GPT into programming education offers transformative potential through personalised guidance and instant feedback, yet risks fostering overreliance and superficial learning due to their tendency to deliver direct, context-free answers.</p>\n </section>\n \n <section>\n \n <h3> Objectives</h3>\n \n <p>This quasi-experimental study addresses this gap by proposing a Socratic questioning framework to optimise GAI-facilitated programming instruction, emphasising critical thinking over passive solution retrieval.</p>\n </section>\n \n <section>\n \n <h3> Methods</h3>\n \n <p>We compared two pedagogical approaches: GAI-Scaffolded Learning (GSL), where GPT employs structured Socratic dialogue to guide problem-solving and GAI-Direct Learning (GDL), which provides immediate answers without guided inquiry. This research collected learners' programming behaviours, interactions data with GPT from screen recordings and platform log data and perceptions data. This research further utilised multiple learning analytics approaches (i.e., click stream analysis, lag-sequential analysis, epistemic network analysis [ENA] and statistics) to compare learners' programming behaviours, interaction patterns and perceptions under two approaches.</p>\n </section>\n \n <section>\n \n <h3> Results and Conclusions</h3>\n \n <p>Through an analysis of 80 college students' programming behaviours, interaction qualities and perceptions, we found some intriguing results. First, GSL engaged in cyclical, reflective practices (debugging, Socratic questioning, console use), while GDL prioritised rapid fixes via trial-and-error with GPT code, risking superficial mimicry and over-reliance on external resources. Second, ENA highlighted GSL's deeper engagement through interconnected feedback, emotional support and iterative inquiry, reducing frustration and sustaining persistence and GDL interactions focused on surface-level queries, lacking scaffolding for emotional/heuristic integration. Third, GSL maintained positive attitudes due to structured prompts aligning expectations and easing cognitive load. GDL attitudes declined from mismatched expectations and frustration.</p>\n </section>\n \n <section>\n \n <h3> Implications</h3>\n \n <p>Based on these findings, the study proposes pedagogical and developmental implications for future design and development of AI-augmented curricula, providing actionable insights for educators seeking to harness GAI's potential while nurturing critical thinking in programming education.</p>\n </section>\n </div>","PeriodicalId":48071,"journal":{"name":"Journal of Computer Assisted Learning","volume":"42 2","pages":""},"PeriodicalIF":4.6000,"publicationDate":"2026-02-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Computer Assisted Learning","FirstCategoryId":"95","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1002/jcal.70210","RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}

引用次数: 0

Abstract

Background

The integration of generative artificial intelligence (GAI) tools like GPT into programming education offers transformative potential through personalised guidance and instant feedback, yet risks fostering overreliance and superficial learning due to their tendency to deliver direct, context-free answers.

Objectives

This quasi-experimental study addresses this gap by proposing a Socratic questioning framework to optimise GAI-facilitated programming instruction, emphasising critical thinking over passive solution retrieval.

Methods

We compared two pedagogical approaches: GAI-Scaffolded Learning (GSL), where GPT employs structured Socratic dialogue to guide problem-solving and GAI-Direct Learning (GDL), which provides immediate answers without guided inquiry. This research collected learners' programming behaviours, interactions data with GPT from screen recordings and platform log data and perceptions data. This research further utilised multiple learning analytics approaches (i.e., click stream analysis, lag-sequential analysis, epistemic network analysis [ENA] and statistics) to compare learners' programming behaviours, interaction patterns and perceptions under two approaches.

Results and Conclusions

Through an analysis of 80 college students' programming behaviours, interaction qualities and perceptions, we found some intriguing results. First, GSL engaged in cyclical, reflective practices (debugging, Socratic questioning, console use), while GDL prioritised rapid fixes via trial-and-error with GPT code, risking superficial mimicry and over-reliance on external resources. Second, ENA highlighted GSL's deeper engagement through interconnected feedback, emotional support and iterative inquiry, reducing frustration and sustaining persistence and GDL interactions focused on surface-level queries, lacking scaffolding for emotional/heuristic integration. Third, GSL maintained positive attitudes due to structured prompts aligning expectations and easing cognitive load. GDL attitudes declined from mismatched expectations and frustration.

Implications

Based on these findings, the study proposes pedagogical and developmental implications for future design and development of AI-augmented curricula, providing actionable insights for educators seeking to harness GAI's potential while nurturing critical thinking in programming education.

查看原文本刊更多论文

当生成人工智能遇到苏格拉底方法：通过行为，交互质量和感知调查编程学习动态

将生成式人工智能（GAI）工具（如GPT）整合到编程教育中，通过个性化指导和即时反馈提供了变革的潜力，但由于它们倾向于提供直接的、与上下文无关的答案，因此有可能导致过度依赖和肤浅的学习。这项准实验研究通过提出一个苏格拉底式提问框架来优化人工智能促进的编程教学，强调批判性思维而不是被动的解决方案检索，从而解决了这一差距。我们比较了两种教学方法：人工智能框架学习（GSL）和人工智能直接学习（GDL），前者采用结构化的苏格拉底式对话来指导解决问题，后者提供即时答案，而无需引导性询问。本研究收集了学习者的编程行为、与GPT的交互数据（来自屏幕记录、平台日志数据和感知数据）。本研究进一步利用多种学习分析方法（即点击流分析、滞后序列分析、认知网络分析[ENA]和统计学）来比较两种方法下学习者的编程行为、交互模式和感知。结果与结论通过对80名大学生编程行为、交互质量和认知的分析，我们发现了一些有趣的结果。首先，GSL从事周期性的反思实践（调试，苏格拉底式提问，控制台使用），而GDL通过GPT代码的试错来优先考虑快速修复，冒着表面模仿和过度依赖外部资源的风险。其次，ENA强调了GSL通过相互关联的反馈、情感支持和迭代查询的更深层次的参与，减少了挫折感并保持了持久性，GDL的互动侧重于表面层面的查询，缺乏情感/启发式整合的框架。第三，GSL保持了积极的态度，因为结构化的提示调整了期望和减轻了认知负荷。GDL的态度从不匹配的期望和沮丧中下降。基于这些发现，该研究提出了未来设计和开发人工智能增强课程的教学和发展意义，为寻求利用人工智能潜力的教育工作者提供了可行的见解，同时在编程教育中培养批判性思维。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Journal of Computer Assisted Learning EDUCATION & EDUCATIONAL RESEARCH-

CiteScore

9.70

自引率

6.00%

发文量

116

期刊介绍： The Journal of Computer Assisted Learning is an international peer-reviewed journal which covers the whole range of uses of information and communication technology to support learning and knowledge exchange. It aims to provide a medium for communication among researchers as well as a channel linking researchers, practitioners, and policy makers. JCAL is also a rich source of material for master and PhD students in areas such as educational psychology, the learning sciences, instructional technology, instructional design, collaborative learning, intelligent learning systems, learning analytics, open, distance and networked learning, and educational evaluation and assessment. This is the case for formal (e.g., schools), non-formal (e.g., workplace learning) and informal learning (e.g., museums and libraries) situations and environments. Volumes often include one Special Issue which these provides readers with a broad and in-depth perspective on a specific topic. First published in 1985, JCAL continues to have the aim of making the outcomes of contemporary research and experience accessible. During this period there have been major technological advances offering new opportunities and approaches in the use of a wide range of technologies to support learning and knowledge transfer more generally. There is currently much emphasis on the use of network functionality and the challenges its appropriate uses pose to teachers/tutors working with students locally and at a distance. JCAL welcomes: -Empirical reports, single studies or programmatic series of studies on the use of computers and information technologies in learning and assessment -Critical and original meta-reviews of literature on the use of computers for learning -Empirical studies on the design and development of innovative technology-based systems for learning -Conceptual articles on issues relating to the Aims and Scope