IEEE Transactions on Learning Technologies最新文献

{"title":"Enhancing Medical Training Through Learning From Mistakes by Interacting With an Ill-Trained Reinforcement Learning Agent","authors":"Yasar C. Kakdas;Sinan Kockara;Tansel Halic;Doga Demirel","doi":"10.1109/TLT.2024.3372508","DOIUrl":"10.1109/TLT.2024.3372508","url":null,"abstract":"This article presents a 3-D medical simulation that employs reinforcement learning (RL) and interactive RL (IRL) to teach and assess the procedure of donning and doffing personal protective equipment (PPE). The simulation is motivated by the need for effective, safe, and remote training techniques in medicine, particularly in light of the COVID-19 pandemic. The simulation has two modes: a tutorial mode and an assessment mode. In the tutorial mode, a computer-based, ill-trained RL agent utilizes RL to learn the correct sequence of donning the PPE by trial and error. This allows students to experience many outlier cases they might not encounter in an in-class educational model. In the assessment mode, an IRL-based method is used to evaluate how effective the participant is at correcting the mistakes performed by the RL agent. Each time the RL agent interacts with the environment and performs an action, the participants provide positive or negative feedback regarding the action taken. Following the assessment, participants receive a score based on the accuracy of their feedback and the time taken for the RL agent to learn the correct sequence. An experiment was conducted using two groups, each consisting of ten participants. The first group received RL-assisted training for donning PPE, followed by an IRL-based assessment. Meanwhile, the second group observed a video featuring the RL agent demonstrating only the correct donning order without outlier cases, replicating traditional training, before undergoing the same assessment as the first group. Results showed that RL-assisted training with many outlier cases was more effective than traditional training with only regular cases. Moreover, combining RL with IRL significantly enhanced the participants' performance. Notably, 90% of the participants finished the assessment with perfect scores within three iterations. In contrast, only 10% of those who did not engage in RL-assisted training finished the assessment with a perfect score, highlighting the substantial impact of RL and IRL integration on participants’ overall achievement.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1248-1260"},"PeriodicalIF":3.7,"publicationDate":"2024-03-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140037300","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Toward an AI Knowledge Assistant for Context-Aware Learning Experiences in Software Capstone Project Development","authors":"Andrés Neyem;Luis A. González;Marcelo Mendoza;Juan Pablo Sandoval Alcocer;Leonardo Centellas;Carlos Paredes","doi":"10.1109/TLT.2024.3396735","DOIUrl":"10.1109/TLT.2024.3396735","url":null,"abstract":"Software assistants have significantly impacted software development for both practitioners and students, particularly in capstone projects. The effectiveness of these tools varies based on their knowledge sources; assistants with localized domain-specific knowledge may have limitations, while tools, such as ChatGPT, using broad datasets, might offer recommendations that do not always match the specific objectives of a capstone course. Addressing a gap in current educational technology, this article introduces an AI Knowledge Assistant specifically designed to overcome the limitations of the existing tools by enhancing the quality and relevance of large language models (LLMs). It achieves this through the innovative integration of contextual knowledge from a local “lessons learned” database tailored to the capstone course. We conducted a study with 150 students using the assistant during their capstone course. Integrated into the Kanban project tracking system, the assistant offered recommendations using different strategies: direct searches in the lessons learned database, direct queries to a generative pretrained transformers (GPT) model, query enrichment with lessons learned before submission to GPT and large language model meta AI (LLaMa) models, and query enhancement with Stack Overflow data before GPT processing. Survey results underscored a strong preference among students for direct LLM queries and those enriched with local repository insights, highlighting the assistant's practical value. Furthermore, our linguistic analysis conclusively demonstrated that texts generated by the LLM closely mirrored the linguistic standards and topical relevance of university course requirements. This alignment not only fosters a deeper understanding of course content but also significantly enhances the material's applicability to real-world scenarios.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1639-1654"},"PeriodicalIF":3.7,"publicationDate":"2024-03-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140826943","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Automated Multimode Teaching Behavior Analysis: A Pipeline-Based Event Segmentation and Description","authors":"Qiuyu Zheng;Zengzhao Chen;Mengke Wang;Yawen Shi;Shaohui Chen;Zhi Liu","doi":"10.1109/TLT.2024.3396159","DOIUrl":"10.1109/TLT.2024.3396159","url":null,"abstract":"The rationality and the effectiveness of classroom teaching behavior directly influence the quality of classroom instruction. Analyzing teaching behavior intelligently can provide robust data support for teacher development and teaching supervision. By observing verbal and nonverbal behaviors of teachers in the classroom, valuable data on teacher–student interaction, classroom atmosphere, and teacher–student rapport can be obtained. However, traditional approaches of teaching behavior analysis primarily focus on student groups in the classroom, neglecting intelligent analysis and intervention of teacher behavior. Moreover, these traditional methods often rely on manual annotation and decision making, which are time consuming and labor intensive, and cannot efficiently facilitate analysis. To address these limitations, this article proposes an innovative automated multimode teaching behavior analysis framework, known as AMTBA. First, a model for segmenting classroom events is introduced, which separates teacher behavior sequences logically. Next, this article utilizes deep learning strategies with optimal performance to conduct multimode analysis and identification of split classroom events, enabling the fine-grained measurement of teacher's behavior in terms of verbal interaction, emotion, gaze, and position. Overall, we establish a uniform description framework. The AMTBA framework is utilized to analyze eight classrooms, and the obtained teacher behavior data are used to analyze differences. The empirical results reveal the differences of teacher behavior in different types of teachers, different teaching modes, and different classes. These findings provide an efficient solution for large-scale and multidisciplinary educational analysis and demonstrate the practical value of AMTBA in educational analytics.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1717-1733"},"PeriodicalIF":3.7,"publicationDate":"2024-03-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140827092","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Development of an Intelligent Tutoring System That Assesses Internal Visualization Skills in Engineering Using Multimodal Triangulation","authors":"Hanall Sung;Martina A. Rau;Barry D. Van Veen","doi":"10.1109/TLT.2024.3396393","DOIUrl":"10.1109/TLT.2024.3396393","url":null,"abstract":"In many science, technology, engineering, and mathematics (STEM) domains, instruction on foundational concepts heavily relies on visuals. Instructors often assume that students can mentally visualize concepts but students often struggle with internal visualization skills—the ability to mentally visualize information. In order to address this issue, we developed a formal, as well as an informal assessment of students’ internal visualization skills in the context of engineering instruction. To validate the assessments, we used data triangulation methods. We drew on data from two separate studies conducted in a small-scale lab experiment and in a larger-scale classroom context. Our studies demonstrate that an intelligent tutoring system with interactive visual representations can serve as an informal assessment of students’ internal visualization skills, predicting their performance on a formal assessment of these skills. Our study enriches methodological and theoretical underpinnings in educational research and practices in multiple ways: it contributes to research methodologies by illustrating how multimodal triangulation can be used for test development, theories of learning by offering pathways to assessing internal visualization skills that are not directly observable, and instructional practices in STEM education by enabling instructors to determine when and where they should provide additional scaffoldings.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1625-1638"},"PeriodicalIF":3.7,"publicationDate":"2024-03-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140826860","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Supporting Teachers’ Professional Development With Generative AI: The Effects on Higher Order Thinking and Self-Efficacy","authors":"Jijian Lu;Ruxin Zheng;Zikun Gong;Huifen Xu","doi":"10.1109/TLT.2024.3369690","DOIUrl":"10.1109/TLT.2024.3369690","url":null,"abstract":"Generative artificial intelligence (AI) has emerged as a noteworthy milestone and a consequential advancement in the annals of major disciplines within the domains of human science and technology. This study aims to explore the effects of generative AI-assisted preservice teaching skills training on preservice teachers’ self-efficacy and higher order thinking. The participants of this study were 215 preservice mathematics, science, and computer teachers from a university in China. First, a pretest–post-test quasi-experimental design was implemented for an experimental group (teaching skills training by generative AI) and a control group (teaching skills training by traditional methods) by investigating the teacher self-efficacy and higher order thinking of the two groups before and after the experiment. Finally, a semistructured interview comprising open-ended questions was administered to 25 preservice teachers within the experimental group to present their views on generative AI-assisted teaching. The results showed that the scores of preservice teachers in the experimental group, who used generative AI for teachers’ professional development, were considerably higher than those of the control group, both in teacher self-efficacy (\u0000<italic>F</i>\u0000 = 8.589, \u0000<italic>p</i>\u0000 = 0.0084 < 0.05) and higher order thinking (\u0000<italic>F</i>\u0000 = 7.217, \u0000<italic>p</i>\u0000 = 0.008 < 0.05). It revealed that generative AI can be effective in supporting teachers’ professional development. This study produced a practical teachers’ professional development method for preservice teachers with generative AI.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1279-1289"},"PeriodicalIF":3.7,"publicationDate":"2024-02-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139978999","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Facilitating the Learning Engineering Process for Educational Conversational Modules Using Transformer-Based Language Models","authors":"Behzad Mirzababaei;Viktoria Pammer-Schindler","doi":"10.1109/TLT.2024.3367738","DOIUrl":"10.1109/TLT.2024.3367738","url":null,"abstract":"In this article, we investigate a systematic workflow that supports the learning engineering process of formulating the starting question for a conversational module based on existing learning materials, specifying the input that transformer-based language models need to function as classifiers, and specifying the adaptive dialogue structure, i.e., the turns the classifiers can choose between. Our primary purpose is to evaluate the effectiveness of conversational modules if a learning engineer follows our workflow. Notably, our workflow is technically lightweight, in the sense that no further training of the models is expected. To evaluate the workflow, we created three different conversational modules. For each, we assessed classifier quality and how coherent the follow-up question asked by the agent was based on the classification results of the user response. The classifiers reached F1-macro scores between 0.66 and 0.86, and the percentage of coherent follow-up questions asked by the agent was between 79% and 84%. These results highlight, first, the potential of transformer-based models to support learning engineers in developing dedicated conversational agents. Second, it highlights the necessity to consider the quality of the adaptation mechanism together with the adaptive dialogue. As such models continue to be improved, their benefits for learning engineering will rise. Future work would be valuable to investigate the usability of this workflow by learning engineers with different backgrounds and prior knowledge on the technical and pedagogical aspects of learning engineering.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1222-1235"},"PeriodicalIF":3.7,"publicationDate":"2024-02-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10440567","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139956841","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Exploring the Possibilities of Edu-Metaverse: A New 3-D Ecosystem Model for Innovative Learning","authors":"Tracy Bobko;Mikiko Corsette;Minjuan Wang;Erin Springer","doi":"10.1109/TLT.2024.3364908","DOIUrl":"10.1109/TLT.2024.3364908","url":null,"abstract":"This article discusses the transformative impact of technology on knowledge acquisition and sharing, focusing on the emergence of the metaverse as a virtual community with vast potential for virtual learning. Learning in the metaverse is found to enhance engagement, motivation, and retention, while fostering 21st-century skills. It also offers personalized and quality education, benefiting students in remote areas. This article explores the Edu-Metaverse ecosystem, which illustrates the interconnectedness of various metaverse components supporting sustainable and equitable learning. The study aims to investigate the alignment of this ecosystem model with teaching and learning activities in exemplary metaverse platforms, its role in fostering inclusive and sustainable learning environments, and how to enhance and rebuild it through 3-D modeling and real metaverse teaching settings experimentation. Throughout this article, the terms “metaverse in education” and “Edu-Metaverse” are used interchangeably. The metaverse is defined as a virtual shared space, ranging from fully virtual worlds, such as virtual reality to partially virtual ones, such as augmented reality. The Edu-Metaverse ecosystem encompasses technologies, platforms, and stakeholders responsible for virtual learning environments. Sustainability, in this context, entails designing systems that withstand environmental, economic, and social pressures while providing equitable and inclusive learning opportunities. Continuous engagement through missions and quests ensures sustainable learning experiences for students. This article highlights the potential of the metaverse to revolutionize education and emphasizes the importance of research before widespread implementation in educational institutions and talent development fields. The Edu-Metaverse ecosystem is presented as a promising framework for advancing virtual learning and fostering inclusive and sustainable education.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1290-1301"},"PeriodicalIF":3.7,"publicationDate":"2024-02-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139945618","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Intelligent Retrieval and Comprehension of Entrepreneurship Education Resources Based on Semantic Summarization of Knowledge Graphs","authors":"Haiyang Yu;Entai Wang;Qi Lang;Jianan Wang","doi":"10.1109/TLT.2024.3364155","DOIUrl":"10.1109/TLT.2024.3364155","url":null,"abstract":"The latest technologies in natural language processing provide creative, knowledge retrieval, and question-answering technologies in the design of intelligent education, which can provide learners with personalized feedback and expert guidance. Entrepreneurship education aims to cultivate and develop the innovative thinking and entrepreneurial skills of students, making it a practical form of education. However, a knowledge retrieval and question-answering teaching assistant system for entrepreneurship education has not been proposed. This observation motivated us to develop a reading comprehension framework to address the challenges of domain-specific knowledge gaps and the weak comprehension of complex texts encountered by intelligent education models in practical applications. The proposed framework mainly includes: question understanding, relevant knowledge retrieval, mathematical calculation, and answer prediction. The techniques involved in the aforementioned modules mainly include text embedding, similarity retrieval, graph convolution, and long short-term memory network. By integrating this model into entrepreneurship courses, learners can participate in real-time discussions and receive immediate feedback, creating a more dynamic and interactive learning environment. To assess the effectiveness of the proposed model, this article conducts answer prediction on single-choice exercises related to entrepreneurship education courses. This study employs the potential of using a question-and-answer format to enhance intelligent entrepreneurship education, paving the way for a more effective and engaging online learning experience.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1210-1221"},"PeriodicalIF":3.7,"publicationDate":"2024-02-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139945748","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Using a Chatbot to Provide Formative Feedback: A Longitudinal Study of Intrinsic Motivation, Cognitive Load, and Learning Performance","authors":"Jiaqi Yin;Tiong-Thye Goh;Yi Hu","doi":"10.1109/TLT.2024.3364015","DOIUrl":"10.1109/TLT.2024.3364015","url":null,"abstract":"This study aimed to examine sustainable effects of chatbot-based formative feedback on intrinsic motivation, cognitive load, and learning performance. A longitudinal quasi-experimental design with 173 undergraduate students was conducted. The experiment is a between-subject design. Students either received formative feedback from a chatbot or a teacher. Utilizing linear mixed model and t-test for data analysis, results showed the following. First, chatbot-based feedback resulted in increased learning interest, perceived choice, and value while decreasing perceived pressure over time. Second, chatbot-based feedback was effective in reducing cognitive load, particularly when learning contents involved conceptual or difficult knowledge. Finally, chatbot-based feedback was found to be more efficient and effective in supporting the mastery of application-based knowledge compared with teacher-based feedback. This study has practical implications for the design of chatbots, and it also enriches the methods of providing ongoing formative feedback in large-scale classrooms.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1404-1415"},"PeriodicalIF":3.7,"publicationDate":"2024-02-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139945547","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Serious Video Games for Agricultural Learning: Scoping Review","authors":"Ismael E. Espinosa-Curiel;Carlos A. García de Alba-Chávez","doi":"10.1109/TLT.2024.3364086","DOIUrl":"10.1109/TLT.2024.3364086","url":null,"abstract":"Serious video games provide a immersive learning environment for agriculture by simulating real-life challenges scenarios. However, empirical evidence of their effectiveness is sparse. This scoping review follows PRISMA-ScR guidelines to summarize literature on serious video games for agricultural learning, highlighting research trends and identifying gaps. We systematically searched nine prominent research databases for papers on serious video games for agriculture learning published between January 2000 and July 2022. Two independent reviewers conducted screening, data extraction, and synthesized the collected data using a narrative approach. The initial search identified 3,297 articles, of which 0.58% (\u0000<italic>n</i>\u0000 = 19) were included in the review. Most reviewed games were released in the last five years, with a predominant presence in the mobile platform. They commonly employed a simulation-based approach, featuring 2-D graphics and designed for single-player experiences. These games mainly target students, focusing on crop production and sustainable agriculture. Educational theories were often unspecified in the studies. Evaluation protocols primarily consisted of pilot studies, emphasizing user experience and knowledge enhancement. Positive outcomes, such as improved user experiences, knowledge, and attitude and behavior changes, were commonly observed in these studies. This study highlights advancements in using serious video games for agricultural learning over 20 years. However, it stresses the need for deeper exploration of game elements' impact on user experience and effectiveness. Creating games for underrepresented players and specific agricultural challenges is essential, as is enhancing theoretical foundations and learning approaches. Rigorous research designs are vital for assessing game effectiveness across short, medium, and long terms.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1155-1169"},"PeriodicalIF":3.7,"publicationDate":"2024-02-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139956845","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}