Problematizing feedback loops: ‘on’, ‘with’, and ‘beyond’ analytics dashboards in MOOCs

ABSTRACTThis paper examines the enactment of feedback in Massive Open Online Courses (MOOCs), focusing on analytics dashboards. Building on scholarship that recognizes data practices as entangled and ‘messy’, the paper problematizes the model of the feedback loop that assumes that analytics dashboards ‘feed back’ data to instructors and/or learners through a singular flow of data processing. By setting out an empirical study that focuses on four MOOCs, in two universities and on two platforms, the paper maps where, when, and how design teams, instructors, and learners are involved in the enactment of feedback through and beyond analytics dashboards. The findings draw on visualizations that highlight complex relations among people and technologies, which include multiple ‘loops’. The paper concludes with questioning the need to capture feedback in a singular loop and suggests prioritizing continued attention to the roles and responsibilities of people – educational designers, instructors, and learners – in MOOCs.KEYWORDS: Analytics dashboardsfeedbackMOOCstopologydata infrastructures AcknowledgmentsWe want to thank Lizzy Garner-Foy, Rachael Mfoafo, and other members of the Online Course Production Service at the University of Edinburgh, and Lien Castelein, Elke Van der Stappen, Jeroen Buntinx, and Kenny Verbeke of the MOOC Team at the KU Leuven for their contributions. We would also like to thank the instructors of ‘Beer: The Science of Brewing’ and ‘Sustainable Business Models’, including Casper Van Cleemput, and the instructors of ‘Climate Solutions’: Dave Reay and Erika Warnatzsch. Furthermore, we would like to thank the learners, including Carol Lewis, Richard Nyoni, Wei Wenyang, Ines Kadangwe, and Alina Liapota. Last but not least, a big thanks to the anonymous contributions of learners, instructors, and staff members.Disclosure statementNo potential conflict of interest was reported by the author(s).Additional informationFundingThis work was supported by the Research Foundation Flanders (FWO) under Grant K208722N.