‘Instructor in action’: Co-design and evaluation of human-centred LA-informed feedback in MOOCs

IF 5.1 2区教育学 Q1 EDUCATION & EDUCATIONAL RESEARCH

Journal of Computer Assisted Learning Pub Date : 2024-09-04 DOI:10.1111/jcal.13057

Paraskevi Topali, Ruth Cobos, Unai Agirre-Uribarren, Alejandra Martínez-Monés, Sara Villagrá-Sobrino

{"title":"‘Instructor in action’: Co-design and evaluation of human-centred LA-informed feedback in MOOCs","authors":"Paraskevi Topali, Ruth Cobos, Unai Agirre-Uribarren, Alejandra Martínez-Monés, Sara Villagrá-Sobrino","doi":"10.1111/jcal.13057","DOIUrl":null,"url":null,"abstract":"<div>\n \n \n <section>\n \n <h3> Background</h3>\n \n <p>Personalised and timely feedback in massive open online courses (MOOCs) is hindered due to the large scale and diverse needs of learners. Learning analytics (LA) can support scalable interventions, however they often lack pedagogical and contextual grounding. Previous research claimed that a human-centred approach in the design of LA solutions can be beneficial. Yet, there is a scarcity of empirical studies discussing participatory approaches addressing LA for feedback in MOOCs.</p>\n </section>\n \n <section>\n \n <h3> Paper Objectives</h3>\n \n <p>We report a human-centred design, where an instructor and a tool developer employed a conceptual framework, named FeeD4Mi, to shape personalised LA-informed feedback in a MOOC.</p>\n </section>\n \n <section>\n \n <h3> Methods</h3>\n \n <p>The current study follows a qualitative interpretative approach to understand how the proposed conceptual framework FeeD4Mi served the MOOC instructor and the tool developer to design feedback interventions and re-design an existing LA tool, respectively.</p>\n </section>\n \n <section>\n \n <h3> Results and Conclusions</h3>\n \n <p>The results showed that FeeD4Mi helped the tool developer to create a contextualised LA tool, named edX-LIMS+. edX-LIMS+ allowed the instructor to monitor and assist struggling MOOC learners in a timely manner. Additionally, FeeD4Mi successfully guided the course instructor in designing and delivering personalised interventions. Yet, the feedback design process was perceived as time-demanding, an issue we plan to address in our future work.</p>\n </section>\n \n <section>\n \n <h3> Takeaways</h3>\n \n <p>The current study extends the existing empirical research about employing human-centred approaches for the design of LA-driven interventions. Moreover, this study advances the theory of scalable feedback tactics, through a conceptual framework that aims to guide MOOC instructors during the definition of feedback interventions. We envision that this research contributes to promoting participatory approaches for designing, delivering, and evaluating LA-informed feedback interventions in authentic contexts.</p>\n </section>\n </div>","PeriodicalId":48071,"journal":{"name":"Journal of Computer Assisted Learning","volume":"40 6","pages":"3149-3166"},"PeriodicalIF":5.1000,"publicationDate":"2024-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/jcal.13057","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Computer Assisted Learning","FirstCategoryId":"95","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1111/jcal.13057","RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}

引用次数: 0

Abstract

Background

Personalised and timely feedback in massive open online courses (MOOCs) is hindered due to the large scale and diverse needs of learners. Learning analytics (LA) can support scalable interventions, however they often lack pedagogical and contextual grounding. Previous research claimed that a human-centred approach in the design of LA solutions can be beneficial. Yet, there is a scarcity of empirical studies discussing participatory approaches addressing LA for feedback in MOOCs.

Paper Objectives

We report a human-centred design, where an instructor and a tool developer employed a conceptual framework, named FeeD4Mi, to shape personalised LA-informed feedback in a MOOC.

Methods

The current study follows a qualitative interpretative approach to understand how the proposed conceptual framework FeeD4Mi served the MOOC instructor and the tool developer to design feedback interventions and re-design an existing LA tool, respectively.

Results and Conclusions

The results showed that FeeD4Mi helped the tool developer to create a contextualised LA tool, named edX-LIMS+. edX-LIMS+ allowed the instructor to monitor and assist struggling MOOC learners in a timely manner. Additionally, FeeD4Mi successfully guided the course instructor in designing and delivering personalised interventions. Yet, the feedback design process was perceived as time-demanding, an issue we plan to address in our future work.

Takeaways

The current study extends the existing empirical research about employing human-centred approaches for the design of LA-driven interventions. Moreover, this study advances the theory of scalable feedback tactics, through a conceptual framework that aims to guide MOOC instructors during the definition of feedback interventions. We envision that this research contributes to promoting participatory approaches for designing, delivering, and evaluating LA-informed feedback interventions in authentic contexts.

Abstract Image

查看原文本刊更多论文

教师在行动"：在 MOOC 中共同设计和评估以人为本的 LA 信息反馈

背景由于大规模开放式在线课程（MOOC）的规模庞大，学习者的需求各不相同，因此，大规模开放式在线课程（MOOC）中个性化的及时反馈受到了阻碍。学习分析（LA）可以支持可扩展的干预措施，但它们往往缺乏教学和情境基础。以往的研究表明，在设计学习分析解决方案时采用以人为本的方法是有益的。论文目的我们报告了一个以人为本的设计，在这个设计中，一名教师和一名工具开发人员采用了一个名为 FeeD4Mi 的概念框架，在 MOOC 中形成了个性化的 LA 信息反馈。结果与结论结果表明，FeeD4Mi 帮助工具开发人员创建了一个名为 edX-LIMS+ 的情境化 LA 工具。此外，FeeD4Mi 还成功地指导了课程教师设计和提供个性化的干预措施。然而，我们认为反馈设计过程耗时较长，我们计划在今后的工作中解决这一问题。本研究扩展了现有的实证研究，即采用以人为本的方法设计洛杉矶驱动的干预措施。此外，本研究通过旨在指导 MOOC 讲师定义反馈干预措施的概念框架，推进了可扩展反馈策略理论的发展。我们设想，这项研究将有助于推广参与式方法，以便在真实情境中设计、实施和评估以 LA 为依据的反馈干预措施。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Journal of Computer Assisted Learning EDUCATION & EDUCATIONAL RESEARCH-

CiteScore

9.70

自引率

6.00%

发文量

116

期刊介绍： The Journal of Computer Assisted Learning is an international peer-reviewed journal which covers the whole range of uses of information and communication technology to support learning and knowledge exchange. It aims to provide a medium for communication among researchers as well as a channel linking researchers, practitioners, and policy makers. JCAL is also a rich source of material for master and PhD students in areas such as educational psychology, the learning sciences, instructional technology, instructional design, collaborative learning, intelligent learning systems, learning analytics, open, distance and networked learning, and educational evaluation and assessment. This is the case for formal (e.g., schools), non-formal (e.g., workplace learning) and informal learning (e.g., museums and libraries) situations and environments. Volumes often include one Special Issue which these provides readers with a broad and in-depth perspective on a specific topic. First published in 1985, JCAL continues to have the aim of making the outcomes of contemporary research and experience accessible. During this period there have been major technological advances offering new opportunities and approaches in the use of a wide range of technologies to support learning and knowledge transfer more generally. There is currently much emphasis on the use of network functionality and the challenges its appropriate uses pose to teachers/tutors working with students locally and at a distance. JCAL welcomes: -Empirical reports, single studies or programmatic series of studies on the use of computers and information technologies in learning and assessment -Critical and original meta-reviews of literature on the use of computers for learning -Empirical studies on the design and development of innovative technology-based systems for learning -Conceptual articles on issues relating to the Aims and Scope