A literature review on the use of retrospective LMS data to investigate online Teaching and Learning practices

Abstract

Access to high quality education is a cornerstone of social, cultural, and economic recovery after any crisis. This is also true of the global COVID-19 pandemic which has disrupted the pedagogical practices of higher education institutions around the world (Butler-Henderson, Crawford, Rudolph, Lalani, & Sabu, 2020). Digital learning has become the new-norm, and tertiary education institutions have been propelled to innovate their teaching methods by integrating digital learning through the adoption of cost-effective (Al-Maroof & Al-Emran, 2018) and adaptable (Bhat et al., 2018) Learning Management Systems (LMS) (Bervell & Umar, 2017; Cigdem & Ozturk, 2016). In these systems, pedagogical technologies are embedded in an infrastructure that enables administration and management of learning contents, communication, assessment, and collaboration (Washington, 2019). Due to the fact that the fundamental task of LMS is to support digital teaching and learning (El Bahsh & Daoud, 2016), they are some of the most extensively used learning technologies in higher education (Abazi-Bexheti, Jajaga, & Abazi-Alili, 2018). LMS technology is now widely adopted to support face-to-face, blended, and online pedagogical practices. In this context, LMS data provides large-scale capture, processing, and analysis of students’ interactions with the system, with each other, and with their teaching support within the system (Chung, 2014).  LMS data provides a rich resource through which to investigate online learning technologies and behaviours (see, for example, Chung, 2014). Such work is increasingly being done across multiple domains, however no systematic review has yet been conducted which surveys such work - specifically examining retrospective LMS data. This is the goal of the present paper.   A literature review was conducted to examine what data analysis methods have been used to better understand online pedagogy. The review focused specifically on the use of tertiary level retrospective LMS data and was not limited to a specific academic domain. The review was conducted from January - December 2021, with searching being conducted in January 2021. A total of 97 full text articles were included in the literature review. The literature review aimed to identify the kinds of research questions retrospective LMS data is being used to answer, the analytical techniques used to analyse this data, and the types of study designs used in this field of research. The number of students represented in the data and the academic domains were also considered. Quality of data and analytical reporting was assessed in order to interrogate the opportunities and challenges of reproducible research in studies using retrospective LMS data. Finally, the review considered the degree to which the analysis of retrospective LMS data met the needs of the research question.   Understanding how retrospective LMS data has been used to examine pedagogical practice in previous research equips us to reorientate Teaching and Learning in the immediate aftermath of COVID-19. This will become increasingly important as we move towards a future characterised by an escalation of remote and online learning opportunities. Through surveying previous research in this area, this paper provides an important foundation for future work utilising retrospective LMS data to understand online Teaching and Learning in the peri-COVID era.   References   Abazi-Bexheti, L., Kadriu, A., Apostolova-Trpkovska, M., Jajaga, E., & Abazi-Alili, H. (2018). LMS solution: Evidence of Google classroom usage inhigher education. Business Systems Research, 9(1), 31–43. https://doi.org/10.2478/bsrj-2018-0003 Al-Maroof, R. A. S., & Al-Emran, M. (2018). Students acceptance of Google classroom: An exploratory study using PLS-SEM approach. International Journal of Emerging Technologies in Learning (IJET), 13(06), 112. https://doi.org/10.3991/ijet.v13i06.8275 Bahsh, R. El, & Daoud, M. I. (2016). Evaluating the use of Moodle to achieve effective and interactive learning : A case study at the German Jordanian University. In Proceedings of the 2nd international conference on open source software computing (OSSCOM 2016) (pp. 16–20). Beirut, Lebanon: IEEE Bervell, B., & Umar, I. N. (2017). A decade of LMS acceptance and adoption research in sub-Sahara African higher education: A systematic review of models, methodologies, milestones and main challenges. Eurasia Journal of Mathematics, Science and Technology Education, 13(11), 7269–7286. https://doi. org/10.12973/ejmste/79444 Bhat, S., Raju, R., Bikramjit, A., & Souza, R. D. (2018). Leveraging e-learning through Google classroom: A usability study. Journal of Engineering Education Transformations, 31(3), 1–7 Butler-Henderson, K., Crawford, J., Rudolph, J., Lalani, K., & Sabu, K.M. (2020). COVID-19 in Higher Education Literature Database (CHELD V1): An open access systematic literature review database with coding rules. Journal of Applied Learning and Teaching, 3(3), DOI:https://doi.org/10.37074/jalt.2020.3.2.11 Chung, G. K. W. K. (2014). Toward the Relational Management of Educational Measurement Data. Teachers College Record, 116(11), p. 1-16 Cigdem, H., & Ozturk, M. (2016). Factors affecting students’ behavioral intention to use LMS at a Turkish post-secondary vocational school. International Review of Research in Open and Distributed Learning, 17(3). https://doi.org/10.19173/irrodl.v17i3.2253 Kumar, J. A., Bervell, B., & Osman, S. (2020). Google classroom: insights from Malaysian higher education students’ and instructors’ experiences. Education and Information Technologies, 25(5), pp. 4175-4195. DOI: 10.1007/s10639-020-10163-x Washington, G. Y. (2019). The learning management system matters in face-to-face higher education courses. Journal of Educational Technology Systems, 1–21. https://doi.org/10.1177/0047239519874037