IEEE Transactions on Learning Technologies最新文献

{"title":"Implementing Artificial Intelligence in Physiotherapy Education: A Case Study on the Use of Large Language Models (LLM) to Enhance Feedback","authors":"Ignacio Villagrán;Rocío Hernández;Gregory Schuit;Andrés Neyem;Javiera Fuentes-Cimma;Constanza Miranda;Isabel Hilliger;Valentina Durán;Gabriel Escalona;Julián Varas","doi":"10.1109/TLT.2024.3450210","DOIUrl":"10.1109/TLT.2024.3450210","url":null,"abstract":"This article presents a controlled case study focused on implementing and using generative artificial intelligence, specifically large language models (LLMs), in physiotherapy education to assist instructors with formulating effective technology-mediated feedback for students. It outlines how these advanced technologies have been integrated into an existing feedback-oriented platform to guide instructors in providing feedback inputs and establish a reference framework for future innovations in practical skills training for health professions education. Specifically, the proposed solution uses LLMs to automatically evaluate feedback inputs made by instructors based on predefined and literature-based quality criteria and generates actionable textual explanations for reformulation. In addition, if the instructor requires, the tool supports summary generation for large sets of text inputs to achieve better student reception and understanding. The case study describes how these features were integrated into the feedback-oriented platform, how their effectiveness was evaluated in a controlled setting with documented feedback inputs, and the results of its implementation with real users through cognitive walkthroughs. Initial results indicate that this innovative implementation holds great potential to enhance learning and performance in physiotherapy education and has the potential to expand to other health disciplines where the development of procedural skills is critical, offering a valuable tool to assess and improve feedback based on quality standards for effective feedback processes. The cognitive walkthroughs allowed us to determine participants' usability decisions in the face of these new features and to evaluate the perceived usefulness, how this would integrate into their workload, and their opinion regarding the potential for the future within this teaching strategy. This article concludes with a discussion of the implications of these findings for practice and future research directions in this developing field.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"2079-2090"},"PeriodicalIF":2.9,"publicationDate":"2024-08-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142225554","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Boundary Conditions of Generalizing Predictive Models for Academic Performance: Within Cohort Versus Within Course","authors":"Sonja Kleter;Uwe Matzat;Rianne Conijn","doi":"10.1109/TLT.2024.3443079","DOIUrl":"10.1109/TLT.2024.3443079","url":null,"abstract":"Much of learning analytics research has focused on factors influencing model generalizability of predictive models for academic performance. The degree of model generalizability across courses may depend on aspects, such as the similarity of the course setup, course material, the student cohort, or the teacher. Which of these contextual factors affect generalizability and to what extent is yet unclear. The current study explicitly compares model generalizability within course versus within cohort of predictive models. This study considered 66 behavioral indicators, which are commonly used in the literature. Indicators regarding frequency and duration of online study time, accessing study material, time management, assignments and quizzes, and weekly measures, were extracted from the university's learning management system. Numerical and binary predictive models were generated via recursive feature selection. Model generalizability was evaluated in terms of both model stability and model performance. The results showed that model stability was better for numerical models generalized within course compared to models generalized within cohort or across course and across cohort. Nevertheless, model stability was low for the binary models and only moderate for numerical models under all the conditions. Concerning model performance, the increase in estimation error after model generalizability depends on the initial model performance for models generalized within course and within cohort. Contrary to previous research, with respect to performance, we found no difference between model generalizability within cohort and within course. We suspect that performance reduction after any form of model generalizability depends on initial performance.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"2183-2194"},"PeriodicalIF":2.9,"publicationDate":"2024-08-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142225552","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Improving Ray Tracing Understanding With Immersive Environments","authors":"Nuno Verdelho Trindade;Lídia Custódio;Alfredo Ferreira;João Madeiras Pereira","doi":"10.1109/TLT.2024.3436656","DOIUrl":"https://doi.org/10.1109/TLT.2024.3436656","url":null,"abstract":"Ray tracing is a computer graphics technique used to produce realistic visuals by physically simulating the behavior of light. Although this technique can be described straightforwardly, fully comprehending it might be challenging. It is typically taught in the classroom using the 2-D formats, such as paper or a blackboard. We propose using immersive environments for incrementing the understanding of ray tracing. We focus on improving the knowledge of the technique in experienced users, particularly Master of Computer Science students minoring in a computer-graphics-related area. We argue that exploring the ray tracing process in an immersive visualization environment can further improve the understanding of ray tracing acquired using conventional means. With that objective, this study starts by presenting \u0000<italic>RayTracerVR</i>\u0000, a virtual reality prototype tool for learning the mechanisms of ray tracing. This tool can be used to visually explore and interact with the different aspects of the technique. It allows users to observe the progression of the rays throughout the sequential stages of the ray tracing process and analyze its corresponding computer pseudocode. The study includes user evaluation where \u0000<italic>RayTracerVR</i>\u0000 is employed to assess improvements in ray tracing understanding. The prototype's usability is also assessed. The findings indicate that using the ray tracing immersive learning environment results in a supplemental increase in understanding in users who have previously learned ray tracing using conventional means.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1975-1988"},"PeriodicalIF":2.9,"publicationDate":"2024-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10634035","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141998662","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"The Three-Stage Hierarchical Logistic Model Controlling Personalized Playback of Audio Information for Intelligent Tutoring Systems","authors":"A. N. Varnavsky","doi":"10.1109/TLT.2024.3439470","DOIUrl":"10.1109/TLT.2024.3439470","url":null,"abstract":"The most critical parameter of audio and video information output is the playback speed, which affects many viewing or listening metrics, including when learning using tutoring systems. However, the availability of quantitative models for personalized playback speed control considering the learner's personal traits is still an open question. The work aims to develop a model to control the personalized playback speed of audio information for beginners and experienced learners for intelligent tutoring systems. Analysis of the data from the experimental study using traditional machine learning methods did not allow us to classify the preferred playback rate with accuracy higher than 60%. Therefore, we developed the three-level hierarchical logistic model that predicts the preferred playback speed of audio material on the scale from “very low speed” to “high speed” for beginners and experienced learners with 80% accuracy. The model uses a combination of cognitive and psychomotor traits of individual learners and aims to maximize audio listening convenience and satisfaction. We explained the influence of the learners' selected personal traits on the preferred speed of audio playback. We calculated the convenience of listening to the audio materials without and with the model. By using the model, the convenience of listening to audio materials increased by an average of 13% at a low speech speed and 37% at a high speech speed. The model extends the control theory of multimedia information in e-learning systems by describing the influence of selected psychophysiological traits of learners on the preferred playback speed of audio materials.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"2005-2019"},"PeriodicalIF":2.9,"publicationDate":"2024-08-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141941041","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"SkillsLab+—A New Way to Teach Practical Medical Skills in an Augmented Reality Application With Haptic Feedback","authors":"Christian Gießer;Johannes Schmitt;Emma Löwenstein;Christian Weber;Veit Braun;Rainer Brück","doi":"10.1109/TLT.2024.3435979","DOIUrl":"10.1109/TLT.2024.3435979","url":null,"abstract":"Digital technologies have transformed medical care and education by providing rapid access to knowledge and advanced methods, such as augmented reality and haptic feedback. These technologies are improving the efficiency of healthcare professionals and the quality of medical education. Particularly in Germany, where a shortage of skilled workers and an aging population are increasing pressure on the healthcare system, digital methods can help to optimize workflows and improve training. The integration of haptic feedback in this context makes it possible to make virtual objects tangible, increasing immersion and ultimately learning success. The application presented, SkillsLab+, uses augmented reality and haptic feedback via a data glove to provide a digitized version of the analog SkillsLab medical training programme. SkillsLab+ was evaluated using standardized questionnaires (Igroup Presence Questionnaire, presence, and haptic questionnaires). In order to determine the learning outcomes of the students, an AB test was carried out comparing the final grades. At the same time, a subjective questionnaire was used to assess whether students felt better prepared for the exam. In this context, this article aims to evaluate the learning success and compare the results with the previous proof of concept study of 2022. The results of the comparison show an improvement in the responses to the SkillsLab+ questionnaire in 2023. The result of the examination also improved compared to the group without AR experience. This shows the improvement in application and learning with the help of augmented reality and haptic feedback. They were more confident, had better results, and felt better prepared for the exams.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"2034-2047"},"PeriodicalIF":2.9,"publicationDate":"2024-07-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141865662","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Balancing Performance and Explainability in Academic Dropout Prediction","authors":"Andrea Zanellati;Stefano Pio Zingaro;Maurizio Gabbrielli","doi":"10.1109/TLT.2024.3425959","DOIUrl":"10.1109/TLT.2024.3425959","url":null,"abstract":"Academic dropout remains a significant challenge for education systems, necessitating rigorous analysis and targeted interventions. This study employs machine learning techniques, specifically random forest (RF) and feature tokenizer transformer (FTT), to predict academic attrition. Utilizing a comprehensive dataset of over 40 000 students from an Italian university, the research incorporates a range of variables, including demographic information, prior educational metrics, and real-time academic performance indicators. We present a nuanced comparative evaluation of the RF and FTT models, highlighting their predictive accuracy and interpretative capabilities. Our empirical results demonstrate the effectiveness of machine learning in managing student attrition, with FTT models outperforming RF models in terms of predictive accuracy and achieving a sensitivity rate of 81%. Significantly, the inclusion of historical academic data enhances the models' ability to identify students at increased risk of dropping out. Furthermore, we apply advanced explanatory techniques, such as shapley additive explanations, to investigate the discriminative power of these models across different student profiles. This provides valuable insights into the key variables influencing dropout risk, contributing to a more holistic understanding of the issue. In addition, we conduct a fairness analysis to ensure the ethical robustness of our predictive models, making them not only effective but also equitable tools.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"2140-2153"},"PeriodicalIF":2.9,"publicationDate":"2024-07-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10612222","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141774818","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Predicting Student Performance in a Programming Tutoring System Using AI and Filtering Techniques","authors":"Miloš Ilić;Goran Keković;Vladimir Mikić;Katerina Mangaroska;Lazar Kopanja;Boban Vesin","doi":"10.1109/TLT.2024.3431473","DOIUrl":"10.1109/TLT.2024.3431473","url":null,"abstract":"In recent years, there has been an increasing trend of utilizing artificial intelligence (AI) methodologies over traditional statistical methods for predicting student performance in e-learning contexts. Notably, many researchers have adopted AI techniques without conducting a comprehensive investigation into the most appropriate and accurate approach to employ. Additionally, determining the optimal input parameters for each AI technique remains a pertinent question in this domain. This study employs machine learning (ML) and artificial neural networks (ANN) to predict student grades within a programming tutoring system. The experiment involved university students whose interaction data with the e-learning system were analyzed and used for predictions. By identifying the structural relationships between the properties of the input data, this research aims to determine the most efficient AI method for accurately predicting student performance in e-learning systems. The structure of the input data in these systems is described by variables related to individual student activities, so correlations between variables were a natural starting point for further theoretical considerations. In this manner, by applying a filtering technique based on the minimum redundancy–maximum relevance (mrMR) criterion, it was shown that correlations among predictors and between predictors and the target variable play a significant role in defining the appropriate model for predicting student grades. The results showed that ANN (the Levenberg–Marquardt algorithm with Bayesian regularization) outperformed ML methods, achieving the highest prediction accuracy. The results obtained from this study can be of great importance for learning technologies engineering and AI in general.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1931-1945"},"PeriodicalIF":2.9,"publicationDate":"2024-07-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141774820","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"EQGG: Automatic Question Group Generation","authors":"Po-Chun Huang;Ying-Hong Chan;Ching-Yu Yang;Hung-Yuan Chen;Yao-Chung Fan","doi":"10.1109/TLT.2024.3430225","DOIUrl":"10.1109/TLT.2024.3430225","url":null,"abstract":"Question generation (QG) task plays a crucial role in adaptive learning. While significant QG performance advancements are reported, the existing QG studies are still far from practical usage. One point that needs strengthening is to consider the generation of \u0000<italic>question group</i>\u0000, which remains untouched. For forming a question group, intrafactors among generated questions should be considered. This article proposes a two-stage framework by combining neural language models and genetic algorithms for addressing the issue of question group generation. Furthermore, experimental evaluation based on benchmark datasets is conducted, and the results show that the proposed framework significantly outperforms the compared baselines. Human evaluations are also conducted to validate the design and understand the limitations.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"2048-2061"},"PeriodicalIF":2.9,"publicationDate":"2024-07-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141774821","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Online Self-Service Learning Platform for Application-Inspired Cloud Development and Operations (DevOps) Curriculum","authors":"Roshan Lal Neupane;Prasad Calyam;Songjie Wang;Kiran Neupane;Ashish Pandey;Xiyao Cheng;Durbek Gafurov;Hemanth Sai Yeddulapalli;Noah Glaser;Kanu Priya Singh;Yuanyuan Gu;Shangman Li;Sharan Srinivas","doi":"10.1109/TLT.2024.3428842","DOIUrl":"10.1109/TLT.2024.3428842","url":null,"abstract":"Cloud-hosted services are being increasingly used in hosting business and scientific applications due to cost-effectiveness, scalability, and ease of deployment. To facilitate rapid development, change and release process of cloud-hosted applications, the area of development and operations (DevOps) is fast evolving. It is necessary to train the future generation of scientific application development professionals such that they are knowledgeable in the DevOps-enabled continuous integration/delivery automation. In this article, we present the design and development of our “Mizzou Cloud DevOps platform,” an online self-service platform to learn cutting-edge Cloud DevOps tools/technologies using open/public cloud infrastructures for wide adoption amongst instructors/students. Our learning platform features scalability, flexibility, and extendability in providing Cloud DevOps concepts knowledge and hands-on skills. We detail our “application-inspired learning” methodology that is based on integration of real-world application use cases in eight learning modules that include laboratory exercises and self-study activities. The learning modules allow students to gain skills in using latest technologies (e.g., containerization, cluster and edge computing, data pipeline automation) to implement relevant security, monitoring, and adaptation mechanisms. The evaluation of our platform features a knowledge growth study to assess student learning, followed by a usability study to assess the online learning platform, as well as the curriculum content as perceived by instructors and students across multiple hands-on workshops.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1946-1960"},"PeriodicalIF":2.9,"publicationDate":"2024-07-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141774819","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Digital Twins of Pet Robots to Prolong Interdependent Relationships and Effects on Student Learning Performance","authors":"Vando Gusti Al Hakim;Su-Hang Yang;Jen-Hang Wang;Hung-Hsuan Lin;Gwo-Dong Chen","doi":"10.1109/TLT.2024.3416209","DOIUrl":"10.1109/TLT.2024.3416209","url":null,"abstract":"The use of robots in education has the potential to engage students in learning activities and aims to form lasting relationships with them. To encourage sustainable, long-term human–robot interactions, a promising approach is to cultivate a pet-like, interdependent relationship. However, the potential of such relationships in education remains unclear, and the limited availability of robots in classrooms necessitates flexible and scalable designs. To address these challenges, this study leverages digital twin technology to facilitate ubiquitous engagement with pet robots, thereby prolonging interdependent relationships through a SeamlessPet robot learning approach. Here, students engaged with both virtual and physical pet robots, enabling realistic and continuous interactions akin to communicating directly with a physical robot. This integration ensured consistent availability and authentic interactions, enhancing educational outcomes demonstrated in situational presentations. An experiment with 70 university students in a Japanese Hospitality Management Program in Taiwan demonstrated that this approach resulted in better learning achievements and fostered a positive learning experience. The pet-like features embedded within the digital twin robots played a vital role in fostering prolonged learning participation, empowering students to take ownership of their learning, stay motivated, and feel supported at any time and from anywhere in the learning process. Educators and curriculum developers are encouraged to consider this approach, particularly in courses with a final project presentation that uses a robot to demonstrate study results.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1883-1897"},"PeriodicalIF":2.9,"publicationDate":"2024-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141509525","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}