{"title":"Development of an Intelligent Tutoring System That Assesses Internal Visualization Skills in Engineering Using Multimodal Triangulation","authors":"Hanall Sung;Martina A. Rau;Barry D. Van Veen","doi":"10.1109/TLT.2024.3396393","DOIUrl":"10.1109/TLT.2024.3396393","url":null,"abstract":"In many science, technology, engineering, and mathematics (STEM) domains, instruction on foundational concepts heavily relies on visuals. Instructors often assume that students can mentally visualize concepts but students often struggle with internal visualization skills—the ability to mentally visualize information. In order to address this issue, we developed a formal, as well as an informal assessment of students’ internal visualization skills in the context of engineering instruction. To validate the assessments, we used data triangulation methods. We drew on data from two separate studies conducted in a small-scale lab experiment and in a larger-scale classroom context. Our studies demonstrate that an intelligent tutoring system with interactive visual representations can serve as an informal assessment of students’ internal visualization skills, predicting their performance on a formal assessment of these skills. Our study enriches methodological and theoretical underpinnings in educational research and practices in multiple ways: it contributes to research methodologies by illustrating how multimodal triangulation can be used for test development, theories of learning by offering pathways to assessing internal visualization skills that are not directly observable, and instructional practices in STEM education by enabling instructors to determine when and where they should provide additional scaffoldings.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1625-1638"},"PeriodicalIF":3.7,"publicationDate":"2024-03-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140826860","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Supporting Teachers’ Professional Development With Generative AI: The Effects on Higher Order Thinking and Self-Efficacy","authors":"Jijian Lu;Ruxin Zheng;Zikun Gong;Huifen Xu","doi":"10.1109/TLT.2024.3369690","DOIUrl":"10.1109/TLT.2024.3369690","url":null,"abstract":"Generative artificial intelligence (AI) has emerged as a noteworthy milestone and a consequential advancement in the annals of major disciplines within the domains of human science and technology. This study aims to explore the effects of generative AI-assisted preservice teaching skills training on preservice teachers’ self-efficacy and higher order thinking. The participants of this study were 215 preservice mathematics, science, and computer teachers from a university in China. First, a pretest–post-test quasi-experimental design was implemented for an experimental group (teaching skills training by generative AI) and a control group (teaching skills training by traditional methods) by investigating the teacher self-efficacy and higher order thinking of the two groups before and after the experiment. Finally, a semistructured interview comprising open-ended questions was administered to 25 preservice teachers within the experimental group to present their views on generative AI-assisted teaching. The results showed that the scores of preservice teachers in the experimental group, who used generative AI for teachers’ professional development, were considerably higher than those of the control group, both in teacher self-efficacy (\u0000<italic>F</i>\u0000 = 8.589, \u0000<italic>p</i>\u0000 = 0.0084 < 0.05) and higher order thinking (\u0000<italic>F</i>\u0000 = 7.217, \u0000<italic>p</i>\u0000 = 0.008 < 0.05). It revealed that generative AI can be effective in supporting teachers’ professional development. This study produced a practical teachers’ professional development method for preservice teachers with generative AI.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1279-1289"},"PeriodicalIF":3.7,"publicationDate":"2024-02-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139978999","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Facilitating the Learning Engineering Process for Educational Conversational Modules Using Transformer-Based Language Models","authors":"Behzad Mirzababaei;Viktoria Pammer-Schindler","doi":"10.1109/TLT.2024.3367738","DOIUrl":"10.1109/TLT.2024.3367738","url":null,"abstract":"In this article, we investigate a systematic workflow that supports the learning engineering process of formulating the starting question for a conversational module based on existing learning materials, specifying the input that transformer-based language models need to function as classifiers, and specifying the adaptive dialogue structure, i.e., the turns the classifiers can choose between. Our primary purpose is to evaluate the effectiveness of conversational modules if a learning engineer follows our workflow. Notably, our workflow is technically lightweight, in the sense that no further training of the models is expected. To evaluate the workflow, we created three different conversational modules. For each, we assessed classifier quality and how coherent the follow-up question asked by the agent was based on the classification results of the user response. The classifiers reached F1-macro scores between 0.66 and 0.86, and the percentage of coherent follow-up questions asked by the agent was between 79% and 84%. These results highlight, first, the potential of transformer-based models to support learning engineers in developing dedicated conversational agents. Second, it highlights the necessity to consider the quality of the adaptation mechanism together with the adaptive dialogue. As such models continue to be improved, their benefits for learning engineering will rise. Future work would be valuable to investigate the usability of this workflow by learning engineers with different backgrounds and prior knowledge on the technical and pedagogical aspects of learning engineering.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1222-1235"},"PeriodicalIF":3.7,"publicationDate":"2024-02-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10440567","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139956841","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Exploring the Possibilities of Edu-Metaverse: A New 3-D Ecosystem Model for Innovative Learning","authors":"Tracy Bobko;Mikiko Corsette;Minjuan Wang;Erin Springer","doi":"10.1109/TLT.2024.3364908","DOIUrl":"10.1109/TLT.2024.3364908","url":null,"abstract":"This article discusses the transformative impact of technology on knowledge acquisition and sharing, focusing on the emergence of the metaverse as a virtual community with vast potential for virtual learning. Learning in the metaverse is found to enhance engagement, motivation, and retention, while fostering 21st-century skills. It also offers personalized and quality education, benefiting students in remote areas. This article explores the Edu-Metaverse ecosystem, which illustrates the interconnectedness of various metaverse components supporting sustainable and equitable learning. The study aims to investigate the alignment of this ecosystem model with teaching and learning activities in exemplary metaverse platforms, its role in fostering inclusive and sustainable learning environments, and how to enhance and rebuild it through 3-D modeling and real metaverse teaching settings experimentation. Throughout this article, the terms “metaverse in education” and “Edu-Metaverse” are used interchangeably. The metaverse is defined as a virtual shared space, ranging from fully virtual worlds, such as virtual reality to partially virtual ones, such as augmented reality. The Edu-Metaverse ecosystem encompasses technologies, platforms, and stakeholders responsible for virtual learning environments. Sustainability, in this context, entails designing systems that withstand environmental, economic, and social pressures while providing equitable and inclusive learning opportunities. Continuous engagement through missions and quests ensures sustainable learning experiences for students. This article highlights the potential of the metaverse to revolutionize education and emphasizes the importance of research before widespread implementation in educational institutions and talent development fields. The Edu-Metaverse ecosystem is presented as a promising framework for advancing virtual learning and fostering inclusive and sustainable education.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1290-1301"},"PeriodicalIF":3.7,"publicationDate":"2024-02-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139945618","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Intelligent Retrieval and Comprehension of Entrepreneurship Education Resources Based on Semantic Summarization of Knowledge Graphs","authors":"Haiyang Yu;Entai Wang;Qi Lang;Jianan Wang","doi":"10.1109/TLT.2024.3364155","DOIUrl":"10.1109/TLT.2024.3364155","url":null,"abstract":"The latest technologies in natural language processing provide creative, knowledge retrieval, and question-answering technologies in the design of intelligent education, which can provide learners with personalized feedback and expert guidance. Entrepreneurship education aims to cultivate and develop the innovative thinking and entrepreneurial skills of students, making it a practical form of education. However, a knowledge retrieval and question-answering teaching assistant system for entrepreneurship education has not been proposed. This observation motivated us to develop a reading comprehension framework to address the challenges of domain-specific knowledge gaps and the weak comprehension of complex texts encountered by intelligent education models in practical applications. The proposed framework mainly includes: question understanding, relevant knowledge retrieval, mathematical calculation, and answer prediction. The techniques involved in the aforementioned modules mainly include text embedding, similarity retrieval, graph convolution, and long short-term memory network. By integrating this model into entrepreneurship courses, learners can participate in real-time discussions and receive immediate feedback, creating a more dynamic and interactive learning environment. To assess the effectiveness of the proposed model, this article conducts answer prediction on single-choice exercises related to entrepreneurship education courses. This study employs the potential of using a question-and-answer format to enhance intelligent entrepreneurship education, paving the way for a more effective and engaging online learning experience.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1210-1221"},"PeriodicalIF":3.7,"publicationDate":"2024-02-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139945748","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Using a Chatbot to Provide Formative Feedback: A Longitudinal Study of Intrinsic Motivation, Cognitive Load, and Learning Performance","authors":"Jiaqi Yin;Tiong-Thye Goh;Yi Hu","doi":"10.1109/TLT.2024.3364015","DOIUrl":"10.1109/TLT.2024.3364015","url":null,"abstract":"This study aimed to examine sustainable effects of chatbot-based formative feedback on intrinsic motivation, cognitive load, and learning performance. A longitudinal quasi-experimental design with 173 undergraduate students was conducted. The experiment is a between-subject design. Students either received formative feedback from a chatbot or a teacher. Utilizing linear mixed model and t-test for data analysis, results showed the following. First, chatbot-based feedback resulted in increased learning interest, perceived choice, and value while decreasing perceived pressure over time. Second, chatbot-based feedback was effective in reducing cognitive load, particularly when learning contents involved conceptual or difficult knowledge. Finally, chatbot-based feedback was found to be more efficient and effective in supporting the mastery of application-based knowledge compared with teacher-based feedback. This study has practical implications for the design of chatbots, and it also enriches the methods of providing ongoing formative feedback in large-scale classrooms.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1404-1415"},"PeriodicalIF":3.7,"publicationDate":"2024-02-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139945547","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ismael E. Espinosa-Curiel;Carlos A. García de Alba-Chávez
{"title":"Serious Video Games for Agricultural Learning: Scoping Review","authors":"Ismael E. Espinosa-Curiel;Carlos A. García de Alba-Chávez","doi":"10.1109/TLT.2024.3364086","DOIUrl":"10.1109/TLT.2024.3364086","url":null,"abstract":"Serious video games provide a immersive learning environment for agriculture by simulating real-life challenges scenarios. However, empirical evidence of their effectiveness is sparse. This scoping review follows PRISMA-ScR guidelines to summarize literature on serious video games for agricultural learning, highlighting research trends and identifying gaps. We systematically searched nine prominent research databases for papers on serious video games for agriculture learning published between January 2000 and July 2022. Two independent reviewers conducted screening, data extraction, and synthesized the collected data using a narrative approach. The initial search identified 3,297 articles, of which 0.58% (\u0000<italic>n</i>\u0000 = 19) were included in the review. Most reviewed games were released in the last five years, with a predominant presence in the mobile platform. They commonly employed a simulation-based approach, featuring 2-D graphics and designed for single-player experiences. These games mainly target students, focusing on crop production and sustainable agriculture. Educational theories were often unspecified in the studies. Evaluation protocols primarily consisted of pilot studies, emphasizing user experience and knowledge enhancement. Positive outcomes, such as improved user experiences, knowledge, and attitude and behavior changes, were commonly observed in these studies. This study highlights advancements in using serious video games for agricultural learning over 20 years. However, it stresses the need for deeper exploration of game elements' impact on user experience and effectiveness. Creating games for underrepresented players and specific agricultural challenges is essential, as is enhancing theoretical foundations and learning approaches. Rigorous research designs are vital for assessing game effectiveness across short, medium, and long terms.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1155-1169"},"PeriodicalIF":3.7,"publicationDate":"2024-02-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139956845","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Blended Laboratory Design Using Raspberry Pi Pico for Digital Circuits and Systems","authors":"Zoe C. M. Davidson;Shuping Dang;Xenofon Vasilakos","doi":"10.1109/TLT.2024.3363230","DOIUrl":"10.1109/TLT.2024.3363230","url":null,"abstract":"Raspberry Pi Pico, based on chip RP2040, is an easy-to-use development microcontroller board that can provide flexible input/output functions and meets the teaching needs of basic electronics to first-year university undergraduates. This article presents our blended laboratory design using Raspberry Pi Pico for the course unit Digital Circuits and Systems. Considering the impacts of Coronavirus Disease 2019 (COVID-19) and the reduced number of students attending the in-person laboratory, we provide an alternative approach using an online Raspberry Pi Pico simulator produced by Wokwi for those students who cannot attend the physical laboratory. The entire laboratory is designed by design-based learning pedagogical methodology and consists of three dependent sessions. Throughout the three laboratory sessions, first-year undergraduates are expected to understand the basic digital logic and electronic circuits by building a simplified interactive traffic light controller system using Raspberry Pi Pico and Python programming. The intended learning outcomes, full details of the blended laboratory design, and the laboratory design evaluation results are given and discussed in this article to verify the effectiveness of the blended laboratory design using Raspberry Pi Pico. By analyzing the empirical data collected from laboratory participants, the effectiveness of the proposed blended laboratory design can be well supported, and all intended learning outcomes are successfully achieved subject to the impacts of COVID-19.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1170-1183"},"PeriodicalIF":3.7,"publicationDate":"2024-02-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139945555","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Thomas Sergent;Morgane Daniel;François Bouchet;Thibault Carron
{"title":"How Can Self-Evaluation and Self-Efficacy Skills of Young Learners be Scaffolded in a Web Application?","authors":"Thomas Sergent;Morgane Daniel;François Bouchet;Thibault Carron","doi":"10.1109/TLT.2024.3360121","DOIUrl":"10.1109/TLT.2024.3360121","url":null,"abstract":"Self-regulated learning (SRL) skills are critical for students of all ages to maximize their learning. Two key processes of SRL are being aware of one's performance (self-evaluation) and believing in one's capabilities to produce given attainments (self-efficacy). To assess and improve these capabilities in young children (5–8), we use a literacy web application, where we introduced two randomly triggered prompts to evaluate perceived difficulty and desired difficulty. Comparing students' actual performance with their responses to self-regulatory prompts provides information about their ability to self-regulate their learning, in particular their self-evaluation and self-efficacy. The novelty of this work resides in studying the SRL of young children (5–8) in digital learning environments while learning another task (reading in our case), measuring and improving some SRL abilities themselves and not only measuring and improving academic results in other tasks, and the large number of students on which the studies were carried (over 400 000). Using 15 982 994 responses from 467 116 students, we first measured two types of SRL deficits, and then, we assessed how a scaffolding and remediation strategy can reduce these deficits. In Study 1, we compare a group receiving remediation feedback to a control group, whereas in Study 2, we determine the impact of age and level on the remediation efficiency. Our contribution is twofold: a method to address on the long term a deficit in self-evaluation or in self-efficacy in a digital learning environment, and a corroboration of the fact that students who are academically at risk lack self-efficacy and avoid tackling challenging exercises compared with their level. We, therefore, recommend that digital learning environments integrate an overlay of SRL, such as self-evaluation and self-efficacy remediation loops, especially for younger students and students who are struggling academically. We included notes for educational practitioners in this article for this purpose.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1184-1197"},"PeriodicalIF":3.7,"publicationDate":"2024-01-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139945561","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Advancing Education Through Extended Reality and Internet of Everything Enabled Metaverses: Applications, Challenges, and Open Issues","authors":"Senthil Kumar Jagatheesaperumal;Kashif Ahmad;Ala Al-Fuqaha;Junaid Qadir","doi":"10.1109/TLT.2024.3358859","DOIUrl":"10.1109/TLT.2024.3358859","url":null,"abstract":"Metaverse has evolved as one of the popular research agenda that let users learn, socialize, and collaborate in a networked 3-D immersive virtual world. Due to the rich multimedia streaming capability and immersive user experience with high-speed communication, the metaverse is an ideal model for education, training, and skill development tasks. To facilitate research in this area, we provide a comprehensive review of the various educational use cases and explore how enabling technologies, such as extended reality and the Internet of Everything will play a major role in educational services in future metaverses. Then, we provide an overview of metaverse-based educational applications focusing on education, training, and skill development and analyze the technologies they are built upon. We identify common research problems and future research directions in the domain. This article also identifies core ethical considerations of metaverse for education and potential pitfalls. We believe this survey can fully demonstrate the versatility of metaverse-driven education, which could serve as a potential guideline for the researchers.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1120-1139"},"PeriodicalIF":3.7,"publicationDate":"2024-01-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10415252","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139945439","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}