{"title":"Implementing Artificial Intelligence in Physiotherapy Education: A Case Study on the Use of Large Language Models (LLM) to Enhance Feedback","authors":"Ignacio Villagrán;Rocío Hernández;Gregory Schuit;Andrés Neyem;Javiera Fuentes-Cimma;Constanza Miranda;Isabel Hilliger;Valentina Durán;Gabriel Escalona;Julián Varas","doi":"10.1109/TLT.2024.3450210","DOIUrl":null,"url":null,"abstract":"This article presents a controlled case study focused on implementing and using generative artificial intelligence, specifically large language models (LLMs), in physiotherapy education to assist instructors with formulating effective technology-mediated feedback for students. It outlines how these advanced technologies have been integrated into an existing feedback-oriented platform to guide instructors in providing feedback inputs and establish a reference framework for future innovations in practical skills training for health professions education. Specifically, the proposed solution uses LLMs to automatically evaluate feedback inputs made by instructors based on predefined and literature-based quality criteria and generates actionable textual explanations for reformulation. In addition, if the instructor requires, the tool supports summary generation for large sets of text inputs to achieve better student reception and understanding. The case study describes how these features were integrated into the feedback-oriented platform, how their effectiveness was evaluated in a controlled setting with documented feedback inputs, and the results of its implementation with real users through cognitive walkthroughs. Initial results indicate that this innovative implementation holds great potential to enhance learning and performance in physiotherapy education and has the potential to expand to other health disciplines where the development of procedural skills is critical, offering a valuable tool to assess and improve feedback based on quality standards for effective feedback processes. The cognitive walkthroughs allowed us to determine participants' usability decisions in the face of these new features and to evaluate the perceived usefulness, how this would integrate into their workload, and their opinion regarding the potential for the future within this teaching strategy. This article concludes with a discussion of the implications of these findings for practice and future research directions in this developing field.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":null,"pages":null},"PeriodicalIF":2.9000,"publicationDate":"2024-08-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Learning Technologies","FirstCategoryId":"95","ListUrlMain":"https://ieeexplore.ieee.org/document/10648793/","RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS","Score":null,"Total":0}
引用次数: 0
Abstract
This article presents a controlled case study focused on implementing and using generative artificial intelligence, specifically large language models (LLMs), in physiotherapy education to assist instructors with formulating effective technology-mediated feedback for students. It outlines how these advanced technologies have been integrated into an existing feedback-oriented platform to guide instructors in providing feedback inputs and establish a reference framework for future innovations in practical skills training for health professions education. Specifically, the proposed solution uses LLMs to automatically evaluate feedback inputs made by instructors based on predefined and literature-based quality criteria and generates actionable textual explanations for reformulation. In addition, if the instructor requires, the tool supports summary generation for large sets of text inputs to achieve better student reception and understanding. The case study describes how these features were integrated into the feedback-oriented platform, how their effectiveness was evaluated in a controlled setting with documented feedback inputs, and the results of its implementation with real users through cognitive walkthroughs. Initial results indicate that this innovative implementation holds great potential to enhance learning and performance in physiotherapy education and has the potential to expand to other health disciplines where the development of procedural skills is critical, offering a valuable tool to assess and improve feedback based on quality standards for effective feedback processes. The cognitive walkthroughs allowed us to determine participants' usability decisions in the face of these new features and to evaluate the perceived usefulness, how this would integrate into their workload, and their opinion regarding the potential for the future within this teaching strategy. This article concludes with a discussion of the implications of these findings for practice and future research directions in this developing field.
期刊介绍:
The IEEE Transactions on Learning Technologies covers all advances in learning technologies and their applications, including but not limited to the following topics: innovative online learning systems; intelligent tutors; educational games; simulation systems for education and training; collaborative learning tools; learning with mobile devices; wearable devices and interfaces for learning; personalized and adaptive learning systems; tools for formative and summative assessment; tools for learning analytics and educational data mining; ontologies for learning systems; standards and web services that support learning; authoring tools for learning materials; computer support for peer tutoring; learning via computer-mediated inquiry, field, and lab work; social learning techniques; social networks and infrastructures for learning and knowledge sharing; and creation and management of learning objects.