IEEE Transactions on Learning Technologies最新文献

{"title":"ChatPRCS: A Personalized Support System for English Reading Comprehension Based on ChatGPT","authors":"Xizhe Wang;Yihua Zhong;Changqin Huang;Xiaodi Huang","doi":"10.1109/TLT.2024.3405747","DOIUrl":"10.1109/TLT.2024.3405747","url":null,"abstract":"Reading comprehension is a widely adopted method for learning English, involving reading articles and answering related questions. However, the reading comprehension training typically focuses on the skill level required for a standardized learning stage, without considering the impact of individual differences in linguistic competence. This article presents a personalized support system for reading comprehension, named chat generative pretrained transformer (ChatGPT)-based personalized reading comprehension support (ChatPRCS), based on the zone of proximal development (ZPD) theory. It leverages the advanced capabilities of large language models, exemplified by ChatGPT. ChatPRCS employs methods, including skill prediction, question generation and automatic evaluation, to enhance reading comprehension instruction. First, a ZPD-based algorithm is developed to predict students' reading comprehension skills. This algorithm analyzes historical data to generate questions with appropriate difficulty. Second, a series of ChatGPT prompt patterns is proposed to address two key aspects of reading comprehension objectives: question generation, and automated evaluation. These patterns further improve the quality of generated questions. Finally, by integrating personalized skill prediction and reading comprehension prompt patterns, ChatPRCS is validated through a series of experiments. Empirical results demonstrate that it provides learners with high-quality reading comprehension questions that are broadly aligned with expert-crafted questions at a statistical level. Furthermore, this study investigates the effect of the system on learning achievement, learning motivation, and cognitive load, providing further evidence of its effectiveness in instructing English reading comprehension.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1762-1776"},"PeriodicalIF":3.7,"publicationDate":"2024-03-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141171260","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Personalized Early Warning of Learning Performance for College Students: A Multilevel Approach via Cognitive Ability and Learning State Modeling","authors":"Hua Ma;Wen Zhao;Yuqi Tang;Peiji Huang;Haibin Zhu;Wensheng Tang;Keqin Li","doi":"10.1109/TLT.2024.3382217","DOIUrl":"10.1109/TLT.2024.3382217","url":null,"abstract":"To prevent students from learning risks and improve teachers' teaching quality, it is of great significance to provide accurate early warning of learning performance to students by analyzing their interactions through an e-learning system. In existing research, the correlations between learning risks and students' changing cognitive abilities or learning states are still underexplored, and the personalized early warning is unavailable for students at different levels. To accurately identify the possible learning risks faced by students at different levels, this article proposes a personalized early warning approach to learning performance for college students via cognitive ability and learning state modeling. In this approach, students' learning process data and historical performance data are analyzed to track students' cognitive abilities in the whole learning process, and model their learning states from four dimensions, i.e., learning quality, learning engagement, latent learning state, and historical learning state. Then, the Adaboost algorithm is used to predict students' learning performance, and an evaluation rule with five levels is designed to dynamically provide multilevel personalized early warning to students. Finally, the comparative experiments based on real-world datasets demonstrate that the proposed approach could effectively predict all students' learning performance, and provide accurate early warning services to them.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1440-1453"},"PeriodicalIF":3.7,"publicationDate":"2024-03-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140316238","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Modeling Student Performance Using Feature Crosses Information for Knowledge Tracing","authors":"Lixiang Xu;Zhanlong Wang;Suojuan Zhang;Xin Yuan;Minjuan Wang;Enhong Chen","doi":"10.1109/TLT.2024.3381045","DOIUrl":"10.1109/TLT.2024.3381045","url":null,"abstract":"Knowledge tracing (KT) is an intelligent educational technology used to model students' learning progress and mastery in adaptive learning environments for personalized education. Despite utilizing deep learning models in KT, current approaches often oversimplify students' exercise records into knowledge sequences, which fail to explore the rich information within individual questions. In addition, existing KT models tend to neglect the complex, higher order relationships between questions and latent concepts. Therefore, we introduce a novel model called feature crosses information-based KT (FCIKT) to explore the intricate interplay between questions, latent concepts, and question difficulties. FCIKT utilizes a fusion module to perform feature crosses operations on questions, integrating information from our constructed multirelational heterogeneous graph using graph convolutional networks. We deployed a multihead attention mechanism, which enriches the static embedding representations of questions and concepts with dynamic semantic information to simulate real-world scenarios of problem-solving. We also used gated recurrent units to dynamically capture and update the students' knowledge state for final prediction. Extensive experiments demonstrated the validity and interpretability of our proposed model.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1390-1403"},"PeriodicalIF":2.9,"publicationDate":"2024-03-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140198823","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"How Well Can Tutoring Audio Be Autoclassified and Machine Explained With XAI: A Comparison of Three Types of Methods","authors":"Lishan Zhang;Linyu Deng;Sixv Zhang;Ling Chen","doi":"10.1109/TLT.2024.3381028","DOIUrl":"10.1109/TLT.2024.3381028","url":null,"abstract":"With the popularity of online one-to-one tutoring, there are emerging concerns about the quality and effectiveness of this kind of tutoring. Although there are some evaluation methods available, they are heavily relied on manual coding by experts, which is too costly. Therefore, using machine learning to predict instruction quality automatically is an effective way to reduce human costs. Three classification methods are analyzed in this article: 1) random forest algorithm with human-engineered descriptive features; 2) long and short-term memory algorithm with acoustic features generated by open speech and music interpretation by large space extraction toolkit; and 3) convolutional neural network algorithm with Mel spectrogram of the audio. The results show that the three approaches can complete the prediction task well, with the second approach exhibiting the best accuracy. The importance of the features in these classification models is analyzed according to eXplainable Artificial Intelligence techniques (i.e., XAI) and statistical feature analysis methods. In this way, key indicators of high-quality tutoring are identified. This study demonstrated the usefulness of XAI techniques in understanding why some tutoring sessions are of good quality and others are not. The results can be potentially used to guide the improvement of online one-to-one tutoring in the future.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1302-1312"},"PeriodicalIF":3.7,"publicationDate":"2024-03-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140198862","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Automated Program Repair for Introductory Programming Assignments","authors":"Han Wan;Hongzhen Luo;Mengying Li;Xiaoyan Luo","doi":"10.1109/TLT.2024.3403710","DOIUrl":"10.1109/TLT.2024.3403710","url":null,"abstract":"Automatic program repair (APR) tools are valuable for students to assist them with debugging tasks since program repair captures the code modification to make a buggy program pass the given test-suite. However, the process of manually generating catalogs of code modifications is intricate and time-consuming. This article proposes contextual error model repair (CEMR), an automated program repair tool for introductory programming assignments. CEMR is designed to learn program code modifications from incorrect–correct code pairs automatically. Then, it utilizes these code modifications along with CodeBERT, a generative AI, to repair students' new incorrect programs in the same programming assignment. CEMR builds on the observation that code edits performed by students in pairs of incorrect–correct code can be used as input–output examples for learning code modifications. The key idea of CEMR is to leverage the \u0000<italic>wisdom of the crowd</i>\u0000: it uses the existing code modifications of incorrect–correct student code pairs to repair the new incorrect student attempts. We chose three of the most related APR tools, Refazer, Refactory, and AlphaRepair, as the baselines to compare against CEMR. The experimental results demonstrate that, on public and real classroom datasets, CEMR achieves higher repair rates than the baselines. Through further analysis, CEMR has demonstrated promising effectiveness in addressing semantical and logical errors while its performance in fixing syntactical errors is limited. In terms of time for repairing buggy programs, CEMR costs approximately half as much as AlphaRepair requires. We opine that CEMR not only be seen as a program repair method that achieves good results with incorrect–correct code pairs but also be further utilized to generate hints to better assist students in learning programming.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1745-1760"},"PeriodicalIF":3.7,"publicationDate":"2024-03-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141146079","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Design and Evaluation of Trustworthy Knowledge Tracing Model for Intelligent Tutoring System","authors":"Yu Lu;Deliang Wang;Penghe Chen;Zhi Zhang","doi":"10.1109/TLT.2024.3403135","DOIUrl":"10.1109/TLT.2024.3403135","url":null,"abstract":"Amid the rapid evolution of artificial intelligence (AI), the intricate model structures and opaque decision-making processes of AI-based systems have raised the trustworthy issues in education. We, therefore, first propose a novel three-layer knowledge tracing model designed to address trustworthiness for an intelligent tutoring system. Each layer is crafted to tackle a specific challenge: transparency, explainability, and accountability. We have introduced an explainable AI (xAI) approach to offer technical interpreting information, validated by the established educational theories and principles. The validated interpreting information is subsequently transitioned from its technical context into educational insights, which are then incorporated into the newly designed user interface. Our evaluations indicate that an intelligent tutoring system, when equipped with the designed trustworthy knowledge tracing model, significantly enhances user trust and knowledge from the perspectives of both teachers and students. This study, thus, contributes a tangible solution that utilizes the xAI approach as the enabling technology to construct trustworthy systems or tools in education.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1701-1716"},"PeriodicalIF":3.7,"publicationDate":"2024-03-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141146168","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Write-Curate-Verify: A Case Study of Leveraging Generative AI for Scenario Writing in Scenario-Based Learning","authors":"Shurui Bai;Donn Emmanuel Gonda;Khe Foon Hew","doi":"10.1109/TLT.2024.3378306","DOIUrl":"10.1109/TLT.2024.3378306","url":null,"abstract":"This case study explored the use of generative artificial intelligence (GenAI), specifically chat generative pretraining transformer (ChatGPT), in writing scenarios for scenario-based learning (SBL). Our research addressed three key questions: 1) how do teachers leverage GenAI to write scenarios for SBL purposes? 2) what is the quality of GenAI-generated SBL scenarios and tasks? and 3) how does GenAI-supported SBL affect students’ motivation, learning performance, and learning perceptions? A three-step prompting engineering process (write the prompts, curate the output, and verify the output, WCV) was established during the teacher interaction with GenAI in the scenario writing. Findings revealed that by using the WCV approach, ChatGPT enabled the efficient creation of quality scenarios for SBL purposes in a short timeframe. Moreover, students exhibited increased intrinsic motivation, learning performance, and positive attitudes toward GenAI-supported scenarios. We also suggest guidelines for using the WCV prompt engineering process in scenario writing.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1313-1324"},"PeriodicalIF":3.7,"publicationDate":"2024-03-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140170034","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Automatic Generation of Multimedia Teaching Materials Based on Generative AI: Taking Tang Poetry as an Example","authors":"Xu Chen;Di Wu","doi":"10.1109/TLT.2024.3378279","DOIUrl":"10.1109/TLT.2024.3378279","url":null,"abstract":"Generative artificial intelligence (AI) is widely recognized as one of the most influential technologies for the future, having sparked a paradigm shift in scientific research. The field of education has also been greatly impacted by this transformative technology, with researchers exploring the applications of generative AI, particularly ChatGPT, in education. However, existing research primarily focuses on generating text from text, and there remains a relative scarcity of studies on leveraging multimodal generation capabilities to address key challenges in multimodal data supported instruction. In this article, we present a technical framework for generating Tang poetry situational videos, emphasizing the utilization of generative AI to address the need for multimedia teaching resources. Our framework comprises three main modules: textual situational comprehension, image creation, and video generation. Moreover, we have developed a situational video generation system that incorporates various technologies, including text-to-text generation models, text-to-image generation models, image interpolation, text-to-speech synthesis, and video synthesis. To ascertain the efficacy of the modules within the Tang poetry situational video generation system, we undertook a comparative analysis utilizing the prevalent text-to-image and text-to-video generation models. The empirical findings indicate that our approach is capable of generating images that exhibit greater semantic similarity with the poems, thereby enabling a better comprehension of the poem's connotations and its key components. Concurrently, the Tang poetry videos generated can significantly contribute to the reduction of cognitive load and the enhancement of understanding during the learning process. Our research showcases the potential of generative AI in the education field, specifically in the domain of multimodal teaching resources.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1353-1366"},"PeriodicalIF":3.7,"publicationDate":"2024-03-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140170039","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Leveraging Multicriteria Integer Programming Optimization for Effective Team Formation","authors":"Pallavi Singh;Phat K. Huynh;Dang Nguyen;Trung Q. Le;Wilfrido Moreno","doi":"10.1109/TLT.2024.3401734","DOIUrl":"10.1109/TLT.2024.3401734","url":null,"abstract":"In organizational and academic settings, the strategic formation of teams is paramount, necessitating an approach that transcends conventional methodologies. This study introduces a novel application of multicriteria integer programming (MCIP), which simultaneously accommodates multiple criteria, thereby innovatively addressing the complex task of team formation. Unlike traditional single-objective optimization methods, our research designs a comprehensive framework capable of modeling a wide array of factors, including skill levels, backgrounds, and personality traits. The objective function of this framework is optimized to maximize within-team diversity while minimizing both conflict levels and variance in diversity between teams. Central to our approach is a two-stage optimization process. Initially, it segments the population into subgroups using a weighted heterogeneous multivariate <italic>K</i>-means algorithm, allowing for a targeted and nuanced team assembly. This is followed by the application of a surrogate optimization technique within these subgroups, efficiently navigating the complexities of MCIP for large-scale applications. Our approach is further enhanced by the inclusion of explicit constraints such as potential interpersonal conflicts, a factor often overlooked in previous studies. The results from our study demonstrate the optimality and robustness of our model across simulation scenarios with different data heterogeneity levels. The contributions of this study are manifold, addressing critical gaps in the existing literature with a theory-backed, empirically validated framework for advanced team formation. Beyond theoretical implications, our work provides a practical guide for implementing conflict-aware, sophisticated team formation strategies in real-world scenarios. This advancement paves the way for future research to explore and enhance this model, providing more sophisticated and efficient team formation strategies.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"18 ","pages":"72-84"},"PeriodicalIF":2.9,"publicationDate":"2024-03-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141063023","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Using ChatGPT for Science Learning: A Study on Pre-service Teachers' Lesson Planning","authors":"Gyeong-Geon Lee;Xiaoming Zhai","doi":"10.1109/TLT.2024.3401457","DOIUrl":"10.1109/TLT.2024.3401457","url":null,"abstract":"While ongoing efforts have continuously emphasized the integration of ChatGPT with science teaching and learning, there are limited empirical studies exploring its actual utility in the classroom. This study aims to fill this gap by analyzing the lesson plans developed by 29 pre-service elementary teachers and assessing how they integrated ChatGPT into science learning activities. We first examined how ChatGPT was integrated with the subject domains, teaching methods/strategies, and then evaluated the lesson plans using a generative artificial intelligence (AI)-technological pedagogical and content knowledge (TPACK)-based rubric. We further examined pre-service teachers' perceptions and concerns about integrating ChatGPT into science learning. Results show a diverse number of ChatGPT applications in different science domains—e.g., Biology (9/29), Chemistry (7/29), and Earth Science (7/29). A total of 14 types of teaching methods/strategies were identified in the lesson plans. On average, the pre-service teachers' lesson plans scored high on the modified TPACK-based rubric (M = 3.29; SD = 0.91; on a 1–4 scale), indicating a reasonable envisage of integrating ChatGPT into science learning, particularly in “instructional strategies and ChatGPT” (M = 3.48; SD = 0.99). However, they scored relatively lower on exploiting ChatGPT's functions toward its full potential (M = 3.00; SD = 0.93), compared to other aspects. We also identified several inappropriate use cases of ChatGPT in lesson planning (e.g., as a source of hallucinated Internet material and technically unsupported visual guidance). Pre-service teachers anticipated ChatGPT to afford high-quality questioning, self-directed learning, individualized learning support, and formative assessment. Meanwhile, they also expressed concerns about its accuracy and the risks that students may be overly dependent on ChatGPT. They further suggested solutions to systemizing classroom dynamics between teachers and students. The study underscores the need for more research on the roles of generative AI in actual classroom settings and provides insights for future AI-integrated science learning.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1683-1700"},"PeriodicalIF":3.7,"publicationDate":"2024-03-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141063211","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}