IEEE Transactions on Learning Technologies最新文献

{"title":"Automated Program Repair for Introductory Programming Assignments","authors":"Han Wan;Hongzhen Luo;Mengying Li;Xiaoyan Luo","doi":"10.1109/TLT.2024.3403710","DOIUrl":"10.1109/TLT.2024.3403710","url":null,"abstract":"Automatic program repair (APR) tools are valuable for students to assist them with debugging tasks since program repair captures the code modification to make a buggy program pass the given test-suite. However, the process of manually generating catalogs of code modifications is intricate and time-consuming. This article proposes contextual error model repair (CEMR), an automated program repair tool for introductory programming assignments. CEMR is designed to learn program code modifications from incorrect–correct code pairs automatically. Then, it utilizes these code modifications along with CodeBERT, a generative AI, to repair students' new incorrect programs in the same programming assignment. CEMR builds on the observation that code edits performed by students in pairs of incorrect–correct code can be used as input–output examples for learning code modifications. The key idea of CEMR is to leverage the \u0000<italic>wisdom of the crowd</i>\u0000: it uses the existing code modifications of incorrect–correct student code pairs to repair the new incorrect student attempts. We chose three of the most related APR tools, Refazer, Refactory, and AlphaRepair, as the baselines to compare against CEMR. The experimental results demonstrate that, on public and real classroom datasets, CEMR achieves higher repair rates than the baselines. Through further analysis, CEMR has demonstrated promising effectiveness in addressing semantical and logical errors while its performance in fixing syntactical errors is limited. In terms of time for repairing buggy programs, CEMR costs approximately half as much as AlphaRepair requires. We opine that CEMR not only be seen as a program repair method that achieves good results with incorrect–correct code pairs but also be further utilized to generate hints to better assist students in learning programming.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1745-1760"},"PeriodicalIF":3.7,"publicationDate":"2024-03-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141146079","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Design and Evaluation of Trustworthy Knowledge Tracing Model for Intelligent Tutoring System","authors":"Yu Lu;Deliang Wang;Penghe Chen;Zhi Zhang","doi":"10.1109/TLT.2024.3403135","DOIUrl":"10.1109/TLT.2024.3403135","url":null,"abstract":"Amid the rapid evolution of artificial intelligence (AI), the intricate model structures and opaque decision-making processes of AI-based systems have raised the trustworthy issues in education. We, therefore, first propose a novel three-layer knowledge tracing model designed to address trustworthiness for an intelligent tutoring system. Each layer is crafted to tackle a specific challenge: transparency, explainability, and accountability. We have introduced an explainable AI (xAI) approach to offer technical interpreting information, validated by the established educational theories and principles. The validated interpreting information is subsequently transitioned from its technical context into educational insights, which are then incorporated into the newly designed user interface. Our evaluations indicate that an intelligent tutoring system, when equipped with the designed trustworthy knowledge tracing model, significantly enhances user trust and knowledge from the perspectives of both teachers and students. This study, thus, contributes a tangible solution that utilizes the xAI approach as the enabling technology to construct trustworthy systems or tools in education.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1701-1716"},"PeriodicalIF":3.7,"publicationDate":"2024-03-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141146168","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Write-Curate-Verify: A Case Study of Leveraging Generative AI for Scenario Writing in Scenario-Based Learning","authors":"Shurui Bai;Donn Emmanuel Gonda;Khe Foon Hew","doi":"10.1109/TLT.2024.3378306","DOIUrl":"10.1109/TLT.2024.3378306","url":null,"abstract":"This case study explored the use of generative artificial intelligence (GenAI), specifically chat generative pretraining transformer (ChatGPT), in writing scenarios for scenario-based learning (SBL). Our research addressed three key questions: 1) how do teachers leverage GenAI to write scenarios for SBL purposes? 2) what is the quality of GenAI-generated SBL scenarios and tasks? and 3) how does GenAI-supported SBL affect students’ motivation, learning performance, and learning perceptions? A three-step prompting engineering process (write the prompts, curate the output, and verify the output, WCV) was established during the teacher interaction with GenAI in the scenario writing. Findings revealed that by using the WCV approach, ChatGPT enabled the efficient creation of quality scenarios for SBL purposes in a short timeframe. Moreover, students exhibited increased intrinsic motivation, learning performance, and positive attitudes toward GenAI-supported scenarios. We also suggest guidelines for using the WCV prompt engineering process in scenario writing.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1313-1324"},"PeriodicalIF":3.7,"publicationDate":"2024-03-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140170034","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Automatic Generation of Multimedia Teaching Materials Based on Generative AI: Taking Tang Poetry as an Example","authors":"Xu Chen;Di Wu","doi":"10.1109/TLT.2024.3378279","DOIUrl":"10.1109/TLT.2024.3378279","url":null,"abstract":"Generative artificial intelligence (AI) is widely recognized as one of the most influential technologies for the future, having sparked a paradigm shift in scientific research. The field of education has also been greatly impacted by this transformative technology, with researchers exploring the applications of generative AI, particularly ChatGPT, in education. However, existing research primarily focuses on generating text from text, and there remains a relative scarcity of studies on leveraging multimodal generation capabilities to address key challenges in multimodal data supported instruction. In this article, we present a technical framework for generating Tang poetry situational videos, emphasizing the utilization of generative AI to address the need for multimedia teaching resources. Our framework comprises three main modules: textual situational comprehension, image creation, and video generation. Moreover, we have developed a situational video generation system that incorporates various technologies, including text-to-text generation models, text-to-image generation models, image interpolation, text-to-speech synthesis, and video synthesis. To ascertain the efficacy of the modules within the Tang poetry situational video generation system, we undertook a comparative analysis utilizing the prevalent text-to-image and text-to-video generation models. The empirical findings indicate that our approach is capable of generating images that exhibit greater semantic similarity with the poems, thereby enabling a better comprehension of the poem's connotations and its key components. Concurrently, the Tang poetry videos generated can significantly contribute to the reduction of cognitive load and the enhancement of understanding during the learning process. Our research showcases the potential of generative AI in the education field, specifically in the domain of multimodal teaching resources.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1353-1366"},"PeriodicalIF":3.7,"publicationDate":"2024-03-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140170039","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Using ChatGPT for Science Learning: A Study on Pre-service Teachers' Lesson Planning","authors":"Gyeong-Geon Lee;Xiaoming Zhai","doi":"10.1109/TLT.2024.3401457","DOIUrl":"10.1109/TLT.2024.3401457","url":null,"abstract":"While ongoing efforts have continuously emphasized the integration of ChatGPT with science teaching and learning, there are limited empirical studies exploring its actual utility in the classroom. This study aims to fill this gap by analyzing the lesson plans developed by 29 pre-service elementary teachers and assessing how they integrated ChatGPT into science learning activities. We first examined how ChatGPT was integrated with the subject domains, teaching methods/strategies, and then evaluated the lesson plans using a generative artificial intelligence (AI)-technological pedagogical and content knowledge (TPACK)-based rubric. We further examined pre-service teachers' perceptions and concerns about integrating ChatGPT into science learning. Results show a diverse number of ChatGPT applications in different science domains—e.g., Biology (9/29), Chemistry (7/29), and Earth Science (7/29). A total of 14 types of teaching methods/strategies were identified in the lesson plans. On average, the pre-service teachers' lesson plans scored high on the modified TPACK-based rubric (M = 3.29; SD = 0.91; on a 1–4 scale), indicating a reasonable envisage of integrating ChatGPT into science learning, particularly in “instructional strategies and ChatGPT” (M = 3.48; SD = 0.99). However, they scored relatively lower on exploiting ChatGPT's functions toward its full potential (M = 3.00; SD = 0.93), compared to other aspects. We also identified several inappropriate use cases of ChatGPT in lesson planning (e.g., as a source of hallucinated Internet material and technically unsupported visual guidance). Pre-service teachers anticipated ChatGPT to afford high-quality questioning, self-directed learning, individualized learning support, and formative assessment. Meanwhile, they also expressed concerns about its accuracy and the risks that students may be overly dependent on ChatGPT. They further suggested solutions to systemizing classroom dynamics between teachers and students. The study underscores the need for more research on the roles of generative AI in actual classroom settings and provides insights for future AI-integrated science learning.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1683-1700"},"PeriodicalIF":3.7,"publicationDate":"2024-03-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141063211","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Automated Essay Scoring and Revising Based on Open-Source Large Language Models","authors":"Yishen Song;Qianta Zhu;Huaibo Wang;Qinhua Zheng","doi":"10.1109/TLT.2024.3396873","DOIUrl":"10.1109/TLT.2024.3396873","url":null,"abstract":"Manually scoring and revising student essays has long been a time-consuming task for educators. With the rise of natural language processing techniques, automated essay scoring (AES) and automated essay revising (AER) have emerged to alleviate this burden. However, current AES and AER models require large amounts of training data and lack generalizability, which makes them hard to implement in daily teaching activities. Moreover, online sites offering AES and AER services charge high fees and have security issues uploading student content. In light of these challenges and recognizing the advancements in large language models (LLMs), we aim to fill these research gaps by analyzing the performance of open-source LLMs when accomplishing AES and AER tasks. Using a human-scored essay dataset (\u0000<italic>n</i>\u0000 = 600) collected in an online assessment, we implemented zero-shot, few-shot, and p-tuning AES methods based on the LLMs and conducted a human–machine consistency check. We conducted a similarity test and a score difference test for the results of AER with LLMs support. The human–machine consistency check result shows that the performance of open-source LLMs with a 10 B parameter size in the AES task is close to that of some deep-learning baseline models, and it can be improved by integrating the comment with the score into the shot or training continuous prompts. The similarity test and score difference test results show that open-source LLMs can effectively accomplish the AER task, improving the quality of the essays while ensuring that the revision results are similar to the original essays. This study reveals a practical path to cost-effectively, time-efficiently, and content-safely assisting teachers with student essay scoring and revising using open-source LLMs.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1920-1930"},"PeriodicalIF":2.9,"publicationDate":"2024-03-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140885774","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Experimenting With Soft Robotics in Education: A Systematic Literature Review From 2006 to 2022","authors":"Israel Ulises Cayetano-Jiménez;Erick Axel Martinez-Ríos;Rogelio Bustamante-Bello;Ricardo A. Ramírez-Mendoza;María Soledad Ramírez-Montoya","doi":"10.1109/TLT.2024.3372894","DOIUrl":"10.1109/TLT.2024.3372894","url":null,"abstract":"Educational robotics (ER) is a discipline of applied robotics focused on teaching robot design, analysis, application, and operation. Traditionally, ER has favored rigid robots, overlooking the potential of soft robots (SRs). While rigid robots offer insights into dynamics, kinematics, and control, they have limitations in exploring the depths of mechanical design and material properties. In this regard, SRs present an opportunity to expand educational topics and activities in robotics through their unique bioinspired properties and accessibility. Despite their promise, there is a notable lack of research on SRs as educational tools, limiting the identification of research avenues that could promote their adoption in educational settings. This study conducts a systematic literature review to elucidate the impact of SRs across academic levels, pedagogical strategies, prevalent artificial muscles, educational activities, and assessment methods. The findings indicate a significant focus on K-12 workshops utilizing soft pneumatic actuators. Furthermore, SRs have fostered the development of fabrication and mechanical design skills beyond mere programming tasks. However, there is a shortage of studies analyzing their use in higher education or their impact on learning outcomes, suggesting a critical need for comprehensive evaluations to determine their effectiveness, rather than solely relying on surveys for student feedback. Thus, there is an opportunity to explore and evaluate the use of SRs in more advanced settings and multidisciplinary activities, urging for rigorous assessments of their influence on learning outcomes. By undertaking this, we aim to provide a foundation for integrating SRs into the ER curriculum, potentially transforming teaching methodologies and enriching students' learning experiences.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1261-1278"},"PeriodicalIF":3.7,"publicationDate":"2024-03-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10460415","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140045730","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Enhancing Medical Training Through Learning From Mistakes by Interacting With an Ill-Trained Reinforcement Learning Agent","authors":"Yasar C. Kakdas;Sinan Kockara;Tansel Halic;Doga Demirel","doi":"10.1109/TLT.2024.3372508","DOIUrl":"10.1109/TLT.2024.3372508","url":null,"abstract":"This article presents a 3-D medical simulation that employs reinforcement learning (RL) and interactive RL (IRL) to teach and assess the procedure of donning and doffing personal protective equipment (PPE). The simulation is motivated by the need for effective, safe, and remote training techniques in medicine, particularly in light of the COVID-19 pandemic. The simulation has two modes: a tutorial mode and an assessment mode. In the tutorial mode, a computer-based, ill-trained RL agent utilizes RL to learn the correct sequence of donning the PPE by trial and error. This allows students to experience many outlier cases they might not encounter in an in-class educational model. In the assessment mode, an IRL-based method is used to evaluate how effective the participant is at correcting the mistakes performed by the RL agent. Each time the RL agent interacts with the environment and performs an action, the participants provide positive or negative feedback regarding the action taken. Following the assessment, participants receive a score based on the accuracy of their feedback and the time taken for the RL agent to learn the correct sequence. An experiment was conducted using two groups, each consisting of ten participants. The first group received RL-assisted training for donning PPE, followed by an IRL-based assessment. Meanwhile, the second group observed a video featuring the RL agent demonstrating only the correct donning order without outlier cases, replicating traditional training, before undergoing the same assessment as the first group. Results showed that RL-assisted training with many outlier cases was more effective than traditional training with only regular cases. Moreover, combining RL with IRL significantly enhanced the participants' performance. Notably, 90% of the participants finished the assessment with perfect scores within three iterations. In contrast, only 10% of those who did not engage in RL-assisted training finished the assessment with a perfect score, highlighting the substantial impact of RL and IRL integration on participants’ overall achievement.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1248-1260"},"PeriodicalIF":3.7,"publicationDate":"2024-03-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140037300","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Toward an AI Knowledge Assistant for Context-Aware Learning Experiences in Software Capstone Project Development","authors":"Andrés Neyem;Luis A. González;Marcelo Mendoza;Juan Pablo Sandoval Alcocer;Leonardo Centellas;Carlos Paredes","doi":"10.1109/TLT.2024.3396735","DOIUrl":"10.1109/TLT.2024.3396735","url":null,"abstract":"Software assistants have significantly impacted software development for both practitioners and students, particularly in capstone projects. The effectiveness of these tools varies based on their knowledge sources; assistants with localized domain-specific knowledge may have limitations, while tools, such as ChatGPT, using broad datasets, might offer recommendations that do not always match the specific objectives of a capstone course. Addressing a gap in current educational technology, this article introduces an AI Knowledge Assistant specifically designed to overcome the limitations of the existing tools by enhancing the quality and relevance of large language models (LLMs). It achieves this through the innovative integration of contextual knowledge from a local “lessons learned” database tailored to the capstone course. We conducted a study with 150 students using the assistant during their capstone course. Integrated into the Kanban project tracking system, the assistant offered recommendations using different strategies: direct searches in the lessons learned database, direct queries to a generative pretrained transformers (GPT) model, query enrichment with lessons learned before submission to GPT and large language model meta AI (LLaMa) models, and query enhancement with Stack Overflow data before GPT processing. Survey results underscored a strong preference among students for direct LLM queries and those enriched with local repository insights, highlighting the assistant's practical value. Furthermore, our linguistic analysis conclusively demonstrated that texts generated by the LLM closely mirrored the linguistic standards and topical relevance of university course requirements. This alignment not only fosters a deeper understanding of course content but also significantly enhances the material's applicability to real-world scenarios.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1639-1654"},"PeriodicalIF":3.7,"publicationDate":"2024-03-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140826943","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Automated Multimode Teaching Behavior Analysis: A Pipeline-Based Event Segmentation and Description","authors":"Qiuyu Zheng;Zengzhao Chen;Mengke Wang;Yawen Shi;Shaohui Chen;Zhi Liu","doi":"10.1109/TLT.2024.3396159","DOIUrl":"10.1109/TLT.2024.3396159","url":null,"abstract":"The rationality and the effectiveness of classroom teaching behavior directly influence the quality of classroom instruction. Analyzing teaching behavior intelligently can provide robust data support for teacher development and teaching supervision. By observing verbal and nonverbal behaviors of teachers in the classroom, valuable data on teacher–student interaction, classroom atmosphere, and teacher–student rapport can be obtained. However, traditional approaches of teaching behavior analysis primarily focus on student groups in the classroom, neglecting intelligent analysis and intervention of teacher behavior. Moreover, these traditional methods often rely on manual annotation and decision making, which are time consuming and labor intensive, and cannot efficiently facilitate analysis. To address these limitations, this article proposes an innovative automated multimode teaching behavior analysis framework, known as AMTBA. First, a model for segmenting classroom events is introduced, which separates teacher behavior sequences logically. Next, this article utilizes deep learning strategies with optimal performance to conduct multimode analysis and identification of split classroom events, enabling the fine-grained measurement of teacher's behavior in terms of verbal interaction, emotion, gaze, and position. Overall, we establish a uniform description framework. The AMTBA framework is utilized to analyze eight classrooms, and the obtained teacher behavior data are used to analyze differences. The empirical results reveal the differences of teacher behavior in different types of teachers, different teaching modes, and different classes. These findings provide an efficient solution for large-scale and multidisciplinary educational analysis and demonstrate the practical value of AMTBA in educational analytics.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1717-1733"},"PeriodicalIF":3.7,"publicationDate":"2024-03-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140827092","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}