British Journal of Educational Technology最新文献

{"title":"Co-creating an equality diversity and inclusion learning analytics dashboard for addressing awarding gaps in higher education","authors":"Vaclav Bayer,&nbsp;Paul Mulholland,&nbsp;Martin Hlosta,&nbsp;Tracie Farrell,&nbsp;Christothea Herodotou,&nbsp;Miriam Fernandez","doi":"10.1111/bjet.13509","DOIUrl":"10.1111/bjet.13509","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <p>Educational outcomes from traditionally underrepresented groups are generally worse than for their more advantaged peers. This problem is typically known as the awarding gap (we use the term awarding gap over ‘attainment gap’ as attainment places the responsibility on students to attain at equal levels) and continues to pose a challenge for educational systems across the world. While Learning Analytics (LA) dashboards help identify patterns contributing to the awarding gap, they often lack stakeholder involvement, offering very little support to institutional Equality, Diversity and Inclusion (EDI) leads or educators to pinpoint and address these gaps. This paper introduces an innovative EDI LA dashboard, co-created with diverse stakeholders. Rigorously evaluated, the dashboard provides fine-grained insights and course-level analysis, empowering institutions to effectively address awarding gaps and contribute to a diverse and inclusive higher education landscape.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <div>\u0000 \u0000 <div>\u0000 \u0000 <h3>Practitioners notes</h3>\u0000 <p>What is already known about this topic\u0000\u0000 </p><ul>\u0000 \u0000 <li>Traditionally underrepresented groups face educational disparities, commonly known as the awarding gap.</li>\u0000 \u0000 <li>Underachievement is a complex multi-dimensional problem and cannot be solely attributable to individual student deficiencies.</li>\u0000 \u0000 <li>LA dashboards targeting this specific problem are often not public, there is little research about them, and are frequently designed with little involvement of educational stakeholders.</li>\u0000 </ul>\u0000 <p>What this paper adds\u0000\u0000 </p><ul>\u0000 \u0000 <li>Pioneers the introduction of a dashboard specifically designed to address the awarding gap problem.</li>\u0000 \u0000 <li>Emphasises the significant data needs of educational stakeholders in tackling awarding gaps.</li>\u0000 \u0000 <li>Expands the design dimensions of Learning Analytics (LA) by introducing a specific design approach rooted in established user experience (UX) design methods.</li>\u0000 </ul>\u0000 <p>Implications for practice and/or policy\u0000\u0000 </p><ul>\u0000 \u0000 <li>Insights from this study will guide practitioners, designers, and developers in creating AI-based educational systems to effectively target the awarding gap problem.</li>\u0000 </ul>\u0000 </","PeriodicalId":48315,"journal":{"name":"British Journal of Educational Technology","volume":null,"pages":null},"PeriodicalIF":6.7,"publicationDate":"2024-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/bjet.13509","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141614031","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"The life cycle of large language models in education: A framework for understanding sources of bias","authors":"Jinsook Lee,&nbsp;Yann Hicke,&nbsp;Renzhe Yu,&nbsp;Christopher Brooks,&nbsp;René F. Kizilcec","doi":"10.1111/bjet.13505","DOIUrl":"10.1111/bjet.13505","url":null,"abstract":"<div>\u0000 \u0000 <section>\u0000 \u0000 \u0000 <p>Large language models (LLMs) are increasingly adopted in educational contexts to provide personalized support to students and teachers. The unprecedented capacity of LLM-based applications to understand and generate natural language can potentially improve instructional effectiveness and learning outcomes, but the integration of LLMs in education technology has renewed concerns over algorithmic bias, which may exacerbate educational inequalities. Building on prior work that mapped the traditional machine learning life cycle, we provide a framework of the LLM life cycle from the initial development of LLMs to customizing pre-trained models for various applications in educational settings. We explain each step in the LLM life cycle and identify potential sources of bias that may arise in the context of education. We discuss why current measures of bias from traditional machine learning fail to transfer to LLM-generated text (eg, tutoring conversations) because text encodings are high-dimensional, there can be multiple correct responses, and tailoring responses may be pedagogically desirable rather than unfair. The proposed framework clarifies the complex nature of bias in LLM applications and provides practical guidance for their evaluation to promote educational equity.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <div>\u0000 \u0000 <div>\u0000 \u0000 <h3>Practitioner notes</h3>\u0000 <p>What is already known about this topic\u0000\u0000 </p><ul>\u0000 \u0000 <li>The life cycle of traditional machine learning (ML) applications which focus on predicting labels is well understood.</li>\u0000 \u0000 <li>Biases are known to enter in traditional ML applications at various points in the life cycle, and methods to measure and mitigate these biases have been developed and tested.</li>\u0000 \u0000 <li>Large language models (LLMs) and other forms of generative artificial intelligence (GenAI) are increasingly adopted in education technologies (EdTech), but current evaluation approaches are not specific to the domain of education.</li>\u0000 </ul>\u0000 <p>What this paper adds\u0000\u0000 </p><ul>\u0000 \u0000 <li>A holistic perspective of the LLM life cycle with domain-specific examples in education to highlight opportunities and challenges for incorporating natural language understanding (NLU) and natural language generation (NLG) into EdTech.</li>\u0000 \u0000 <li>Potential sources of bias are identified in each step of the LLM life cycle and discussed in the context of education.</li>\u0000 \u0000 <li>A framework for understanding where to expect potential harms of LLMs for ","PeriodicalId":48315,"journal":{"name":"British Journal of Educational Technology","volume":null,"pages":null},"PeriodicalIF":6.7,"publicationDate":"2024-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141614158","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Advancing equity and inclusion in educational practices with AI-powered educational decision support systems (AI-EDSS)","authors":"Olga Viberg,&nbsp;René F. Kizilcec,&nbsp;Alyssa Friend Wise,&nbsp;Ioana Jivet,&nbsp;Nia Nixon","doi":"10.1111/bjet.13507","DOIUrl":"10.1111/bjet.13507","url":null,"abstract":"<p>A key goal of educational institutions around the world is to provide inclusive, equitable quality education and lifelong learning opportunities for all learners. Achieving this requires contextualized approaches to accommodate diverse global values and promote learning opportunities that best meet the needs and goals of all learners as individuals and members of different communities. Advances in learning analytics (LA), natural language processes (NLP), and artificial intelligence (AI), especially generative AI technologies, offer potential to aid educational decision making by supporting analytic insights and personalized recommendations. However, these technologies also raise serious risks for reinforcing or exacerbating existing inequalities; these dangers arise from multiple factors including biases represented in training datasets, the technologies' abilities to take autonomous decisions, and processes for tool development that do not centre the needs and concerns of historically marginalized groups. To ensure that Educational Decision Support Systems (EDSS), particularly AI-powered ones, are equipped to promote equity, they must be created and evaluated holistically, considering their potential for both targeted and systemic impacts on all learners, especially members of historically marginalized groups. Adopting a socio-technical and cultural perspective is crucial for designing, deploying, and evaluating AI-EDSS that truly advance educational equity and inclusion. This editorial introduces the contributions of five papers for the special section on advancing equity and inclusion in educational practices with AI-EDSS. These papers focus on (i) a review of biases in large language models (LLMs) applications offers practical guidelines for their evaluation to promote educational equity, (ii) techniques to mitigate disparities across countries and languages in LLMs representation of educationally relevant knowledge, (iii) implementing equitable and intersectionality-aware machine learning applications in education, (iv) introducing a LA dashboard that aims to promote institutional equality, diversity, and inclusion, and (v) vulnerable student digital well-being in AI-EDSS. Together, these contributions underscore the importance of an interdisciplinary approach in developing and utilizing AI-EDSS to not only foster a more inclusive and equitable educational landscape worldwide but also reveal a critical need for a broader contextualization of equity that incorporates the socio-technical questions of what kinds of decisions AI is being used to support, for what purposes, and whose goals are prioritized in this process.</p>","PeriodicalId":48315,"journal":{"name":"British Journal of Educational Technology","volume":null,"pages":null},"PeriodicalIF":6.7,"publicationDate":"2024-07-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/bjet.13507","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141614159","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Evidence-based learning analytics: Reusing and reapplying successful methods and techniques in real learning settings","authors":"Cristian Cechinel,&nbsp;Jorge Maldonado-Mahauad,&nbsp;Roberto Munoz,&nbsp;Xavier Ochoa","doi":"10.1111/bjet.13506","DOIUrl":"10.1111/bjet.13506","url":null,"abstract":"","PeriodicalId":48315,"journal":{"name":"British Journal of Educational Technology","volume":null,"pages":null},"PeriodicalIF":6.7,"publicationDate":"2024-07-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/bjet.13506","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141614029","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Examining cognitive processes of spatial thinking in university students: Insights from a web‐based geographic information systems study","authors":"Xi Xiang, Di Xi","doi":"10.1111/bjet.13502","DOIUrl":"https://doi.org/10.1111/bjet.13502","url":null,"abstract":"Spatial thinking is essential for nurturing spatially literate graduates in tertiary education. However, there is limited research on individual differences in cognitive processes and their impact on spatial problem solving in disciplinary contexts. This study aimed to investigate cognitive processes involved in spatial thinking in geography majors using a web‐based geographic information systems (GIS) mapping tool. The results revealed three clusters characterised by distinctive cognitive processes: <jats:italic>spatial analytic</jats:italic>, <jats:italic>spatial diagrammatic</jats:italic> and <jats:italic>alternative</jats:italic>. Each cluster adopted unique spatial strategies to solve problems with web‐based GIS. Notably, <jats:italic>spatial analytic</jats:italic> learners demonstrated the most optimal profile, resulting in high spatial task performance. These findings have implications for maximising students' learning potential in spatial thinking in the tertiary classroom, optimising performance outcomes in spatial problem solving and building intelligent tutoring systems for adaptive learning.<jats:label/><jats:boxed-text content-type=\"box\" position=\"anchor\"><jats:caption>Practitioner notes</jats:caption>What is already known about this topic <jats:list list-type=\"bullet\"> <jats:list-item>There are individual differences in spatial reasoning.</jats:list-item> <jats:list-item>The processes of spatial thinking may have an impact on learners' spatial performance outcomes.</jats:list-item> </jats:list>What this paper adds <jats:list list-type=\"bullet\"> <jats:list-item>Three clusters characterised by distinctive processes of spatial thinking were identified: <jats:italic>spatial analytic</jats:italic>, <jats:italic>spatial diagrammatic</jats:italic> and <jats:italic>alternative</jats:italic>.</jats:list-item> <jats:list-item>Each cluster adopted unique spatial strategies to solve problems with web‐based GIS.</jats:list-item> <jats:list-item><jats:italic>Spatial analytic</jats:italic> learners demonstrated the optimal profile, resulting in high‐level spatial performance, whereas <jats:italic>alternative</jats:italic> learners exhibited the maladaptive profile, which was associated with low task outcomes.</jats:list-item> </jats:list>Implications for practice and/or policy <jats:list list-type=\"bullet\"> <jats:list-item>Web‐based GIS mapping tools make it possible to track the processes of spatial thinking that have remained largely unexplored.</jats:list-item> <jats:list-item>Cluster analysis and lag sequential analysis reveal differences in spatial reasoning, aiding educators in maximising the potential for university students to learn spatial thinking and optimising performance outcomes in spatial problem solving.</jats:list-item> <jats:list-item>Our findings could inform learning technology designers to build adaptive learning applications in which students receive automatic feedback and tailored support while completing spatial tasks at th","PeriodicalId":48315,"journal":{"name":"British Journal of Educational Technology","volume":null,"pages":null},"PeriodicalIF":6.6,"publicationDate":"2024-07-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141574770","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Exploring the impact of VoiceBots on multimedia programming education among Ghanaian university students","authors":"Harry Barton Essel, Dimitrios Vlachopoulos, Henry Nunoo‐Mensah, John Opuni Amankwa","doi":"10.1111/bjet.13504","DOIUrl":"https://doi.org/10.1111/bjet.13504","url":null,"abstract":"<jats:label/>Conversational user interfaces (CUI), including voice interfaces, which allow users to converse with computers via voice, are gaining wide popularity. VoiceBots allow users to receive a response in real‐time, regardless of the communication device. VoiceBots have been explored in fields such as customer service to automate repetitive queries and help reduce redundant tasks; however, they have not been widely applied in the classroom. This study aimed to explore the effects of VoiceBot implementation on student learning. A pre‐test–post‐test design was implemented with 65 participating undergraduate students in multimedia programming who were randomly allocated to scenarios representing a 2 × 2 design (experimental and control cohorts). Data were collected using an academic achievement test and semi‐structured interviews, which allowed for a more in‐depth analysis of the students' experiences with the VoiceBot. The results showed that how the VoiceBot was applied positively influenced student learning in the experimental cohort. Moreover, the focus group data demonstrated that the VoiceBot can be a valuable assistant for students and could be easily replicated in other courses. To the best of our knowledge, this study was the first to use VoiceBot to engage undergraduate students in Ghana, thus contributing to the growing literature stream on the development of VoiceBots to improve student learning experiences. This study elucidates the design process using a zero‐coding technique, which is considered a suitable approach for educational institutions with limited resources.<jats:label/><jats:boxed-text content-type=\"box\" position=\"anchor\"><jats:caption>Practitioner notes</jats:caption>What is already known about this topic <jats:list list-type=\"bullet\"> <jats:list-item>Conversational user interfaces (CUIs), including voice interfaces, have gained popularity and are used to interact with computers through natural language.</jats:list-item> <jats:list-item>VoiceBots have been utilised in various fields such as customer service to automate tasks and reduce redundancy. Instant messaging systems such as WhatsApp and Telegram have been used for communication in educational contexts.</jats:list-item> <jats:list-item>Advances in artificial intelligence (AI) and natural language processing (NLP) have led to significant improvements in voice‐enabled CUIs (VoiceBots).</jats:list-item> <jats:list-item>Existing studies indicate that chatbots affect students' motivation, learning experiences, and achievements; however, research on using VoiceBots for learning improvement is limited.</jats:list-item> </jats:list>What this paper adds <jats:list list-type=\"bullet\"> <jats:list-item>A VoiceBot was introduced as an assistant to facilitate learning in a multimedia programming course.</jats:list-item> <jats:list-item>The study used an experimental design with an experimental cohort using a WhatsApp group platform equipped with a zero‐coding VoiceBot and a c","PeriodicalId":48315,"journal":{"name":"British Journal of Educational Technology","volume":null,"pages":null},"PeriodicalIF":6.6,"publicationDate":"2024-07-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141547672","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Robot NAO integrated lesson vs. traditional lesson: Measuring learning outcomes on the topic of “societal change” and the mediating effect of students' attitudes","authors":"Violeta Rosanda, Ivan Bratko, Mateja Gačnik, Vid Podpečan, Andreja Istenič","doi":"10.1111/bjet.13501","DOIUrl":"https://doi.org/10.1111/bjet.13501","url":null,"abstract":"Our research aims to examine the effectiveness of introducing social robots as educational technology within authentic classroom activities without modifying them to be designed for a robot. We chose as test subject the fifth‐grade curricular topic “<jats:italic>The role of technology and its impact on society</jats:italic>”, meeting the critical stage of moral development students aged of 11–12. The study, with both experimental (EG) and control groups (CG), will be conducted over 6 weeks. This study will examine the impact of robot‐supported lessons with post‐participation testing on learning outcomes and examine students' perception of the robot in the classroom as a potential correlation with academic performance. The form of the study will be a between‐group non‐randomised controlled experiment. Control and experimental groups will be matched concerning gender, mastery of technology and previous knowledge and understanding of the curricular topic in focus. The instructional design of process‐outcome strategies will incorporate all of Bloom's taxonomic levels. In the review of related studies, we identified gaps in social robot‐supported lessons within the regular curriculum between‐group experiment. Based on a review of related research showing more focus on robot performance in the classroom from technical‐interaction aspects we want to convey from pedagogical starting point. The robot's placement in the pedagogical process will be considered an integral part of the teacher's technical environment. We will use the pre‐participation test to establish whether there is the initial equivalence between EG and CG in terms of gender, mastery of technology, and previous knowledge and understanding of the curricular topic under examination.","PeriodicalId":48315,"journal":{"name":"British Journal of Educational Technology","volume":null,"pages":null},"PeriodicalIF":6.6,"publicationDate":"2024-07-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141547640","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Investigating teacher orchestration load in scripted CSCL: A multimodal data analysis perspective","authors":"Lubna Hakami,&nbsp;Davinia Hernández-Leo,&nbsp;Ishari Amarasinghe,&nbsp;Batuhan Sayis","doi":"10.1111/bjet.13500","DOIUrl":"10.1111/bjet.13500","url":null,"abstract":"<p>Despite the growing interest in using multimodal data to analyse students' actions in Computers-Supported Collaborative Learning (CSCL) settings, studying teacher's orchestration load in such settings remains overlooked. The notion of classroom orchestration, and orchestration load, offer a lens to study the implications of increasingly complex technology-supported learning environments on teacher performance. A combination of multimodal data may aid in understanding teachers' orchestration actions and, as a result, gain insights regarding the orchestration load teachers perceive in scripted CSCL situations. Studying teacher orchestration load in CSCL helps understand the workload teachers experience while facilitating student collaboration and assists in informing design decisions for teacher supporting tools. In this paper, we collect and analyse data from different modalities (i.e. electrodermal activity, observation notes, log data, dashboard screen recordings and responses to self-reported questionnaires) to study teachers' orchestration load in scripted CSCL. A tool called PyramidApp was used to deploy CSCL activities and a teacher-facing dashboard was used to facilitate teachers in managing collaboration in real time. The findings of the study show the potential of multimodal data analysis in investigating and estimating the orchestration load experienced by teachers in scripted CSCL activities. Study findings further demonstrate factors emerging from multimodal data such as task type, activity duration, and number of students influenced teachers' orchestration load.</p>","PeriodicalId":48315,"journal":{"name":"British Journal of Educational Technology","volume":null,"pages":null},"PeriodicalIF":6.7,"publicationDate":"2024-06-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/bjet.13500","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141528964","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"What factors influence scientific concept learning? A study based on the fuzzy‐set qualitative comparative analysis","authors":"Jingjing Ma, Qingtang Liu, Shufan Yu, Jindian Liu, Xiaojuan Li, Chunhua Wang","doi":"10.1111/bjet.13499","DOIUrl":"https://doi.org/10.1111/bjet.13499","url":null,"abstract":"This research employs the fuzzy‐set qualitative comparative analysis (fsQCA) method to investigate the configurations of multiple factors influencing scientific concept learning, including augmented reality (AR) technology, the concept map (CM) strategy and individual differences (eg, prior knowledge, experience and attitudes). A quasi‐experiment was conducted with 194 seventh‐grade students divided into four groups: AR and CM (<jats:italic>N</jats:italic> = 52), AR and non‐CM (<jats:italic>N</jats:italic> = 51), non‐AR and CM (<jats:italic>N</jats:italic> = 40), non‐AR and non‐CM (<jats:italic>N</jats:italic> = 51). These students participated in a science lesson on ‘The structure of peach blossom’. This study represents students' science learning outcomes by measuring their academic performance and cognitive load. The fsQCA results reveal that: (1) factors influencing students' academic performance and cognitive load are interdependent, and a single factor cannot constitute a necessary condition for learning outcomes; (2) multiple pathways can lead to the same learning outcome, challenging the notion of a singular best path derived from traditional analysis methods; (3) the configurations of good and poor learning outcomes exhibit asymmetry. For example, high prior knowledge exists in both configurations leading to good and poor learning outcomes, depending on how other conditions are combined.<jats:label/><jats:boxed-text content-type=\"box\" position=\"anchor\"><jats:caption>Practitioner notes</jats:caption>What is already known about this topic <jats:list list-type=\"bullet\"> <jats:list-item>Augmented reality proves to be a useful technological tool for improving science learning.</jats:list-item> <jats:list-item>The concept map can guide students to describe the relationships between concepts and make a connection between new knowledge and existing knowledge structures.</jats:list-item> <jats:list-item>Individual differences have been emphasized as essential external factors in controlling the effectiveness of learning.</jats:list-item> </jats:list>What this paper adds <jats:list list-type=\"bullet\"> <jats:list-item>This study innovatively employed the fsQCA analysis method to reveal the complex phenomenon of the scientific concept learning process at a fine‐grained level.</jats:list-item> <jats:list-item>This study discussed how individual differences interact with AR and concept map strategy to influence scientific concept learning.</jats:list-item> </jats:list>Implications for practice and/or policy <jats:list list-type=\"bullet\"> <jats:list-item>No single factor present or absent is necessary for learning outcomes, but the combinations of AR and concept map strategy always obtain satisfactory learning outcomes.</jats:list-item> <jats:list-item>There are multiple pathways to achieving good learning outcomes rather than a single optimal solution.</jats:list-item> <jats:list-item>The implementation of educational interventions should fully consid","PeriodicalId":48315,"journal":{"name":"British Journal of Educational Technology","volume":null,"pages":null},"PeriodicalIF":6.6,"publicationDate":"2024-06-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141528967","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Evidence-based multimodal learning analytics for feedback and reflection in collaborative learning","authors":"Lixiang Yan,&nbsp;Vanessa Echeverria,&nbsp;Yueqiao Jin,&nbsp;Gloria Fernandez-Nieto,&nbsp;Linxuan Zhao,&nbsp;Xinyu Li,&nbsp;Riordan Alfredo,&nbsp;Zachari Swiecki,&nbsp;Dragan Gašević,&nbsp;Roberto Martinez-Maldonado","doi":"10.1111/bjet.13498","DOIUrl":"10.1111/bjet.13498","url":null,"abstract":"<div>\u0000 \u0000 <section>\u0000 \u0000 <p>Multimodal learning analytics (MMLA) offers the potential to provide evidence-based insights into complex learning phenomena such as collaborative learning. Yet, few MMLA applications have closed the learning analytics loop by being evaluated in real-world educational settings. This study evaluates the effectiveness of an MMLA solution in enhancing feedback and reflection within a complex and highly dynamic collaborative learning environment. A two-year longitudinal study was conducted with 399 students and 17 teachers, utilising an MMLA system in reflective debriefings in the context of healthcare education. We analysed the survey data of 74 students and 11 teachers regarding their perceptions of the MMLA system. We applied the Evaluation Framework for Learning Analytics, augmented by complexity, accuracy and trust measures, to assess both teachers' and students' perspectives. The findings illustrated that teachers and students both had generally positive perceptions of the MMLA solution. Teachers found the MMLA solution helpful in facilitating feedback provision and reflection during debriefing sessions. Similarly, students found the MMLA solution effective in providing clarity on the data collected, stimulating reflection on their learning behaviours, and prompting considerations for adaptation in their learning behaviours. However, the complexity of the MMLA solution and the need for qualitative measures of communication emerged as areas for improvement. Additionally, the study highlighted the importance of data accuracy, transparency, and privacy protection to maintain user trust. The findings provide valuable contributions to advancing our understanding of the use of MMLA in supporting feedback and reflection practices in intricate collaborative learning while identifying avenues for further research and improvement. We also provided several insights and practical recommendations for successful MMLA implementation in authentic learning contexts.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <div>\u0000 \u0000 <div>\u0000 \u0000 <h3>Practitioner notes</h3>\u0000 <p>What is currently known about this topic\u0000\u0000 </p><ul>\u0000 \u0000 <li>Multimodal learning analytics (MMLA) seeks to generate data-informed insights about learners' metacognitive and emotional states as well as their learning behaviours, by utilising intricate physical and physiological signals.</li>\u0000 \u0000 <li>MMLA has not only pioneered novel data analytic methods but also aspired to complete the learning analytics loop by crafting innovative, tangible solutions that relay these insights to the concerned stakeholders.</li>\u0000 \u0000 <li>A prominent directi","PeriodicalId":48315,"journal":{"name":"British Journal of Educational Technology","volume":null,"pages":null},"PeriodicalIF":6.7,"publicationDate":"2024-06-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/bjet.13498","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141504759","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}