J. Learn. Anal.Pub Date : 2021-10-25DOI: 10.35542/osf.io/qfh7z
M. Cukurova, Madiha Khan-Galaria, E. Millán, R. Luckin
{"title":"A Learning Analytics Approach to Monitoring the Quality of Online One-to-One Tutoring","authors":"M. Cukurova, Madiha Khan-Galaria, E. Millán, R. Luckin","doi":"10.35542/osf.io/qfh7z","DOIUrl":"https://doi.org/10.35542/osf.io/qfh7z","url":null,"abstract":"One-to-one online tutoring provided by human tutors can improve students’ learning outcomes. However, monitoring the quality of such tutoring is a significant challenge. In this paper, we propose a learning analytics approach for monitoring online one-to-one tutoring quality. The approach analyses teacher behaviours and classifies tutoring sessions into those that are effective and those that are not effective. More specifically, we use sequential behaviour pattern mining to analyse tutoring sessions using the CM-SPAM algorithm and classify tutoring sessions into effective and less effective using the J-48 and JRIP decision tree classifiers. To show the feasibility of the approach, we analysed data from 2250 minutes of online one-to-one primary Maths tutoring sessions with 44 tutors from 8 schools. The results showed that the approach can classify tutors’ effectiveness with high accuracy (F measures of 0.89 and 0.98 were achieved). The results also showed that effective tutors present significantly more frequent hint provision and proactive planning behaviours than their less effective colleagues in these online one-to-one sessions. Furthermore, effective tutors sequence their monitoring actions with appropriate pauses and initiations of students’ self-correction behaviours. We conclude that the proposed approach is feasible to monitor the quality of online one-to-one primary Maths tutoring sessions.","PeriodicalId":145357,"journal":{"name":"J. Learn. Anal.","volume":"119 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116892088","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
J. Learn. Anal.Pub Date : 2021-10-13DOI: 10.18608/jla.2021.7379
Yi-Shan Tsai, A. Whitelock-Wainwright, D. Gašević
{"title":"More Than Figures on Your Laptop: (Dis)trustful Implementation of Learning Analytics","authors":"Yi-Shan Tsai, A. Whitelock-Wainwright, D. Gašević","doi":"10.18608/jla.2021.7379","DOIUrl":"https://doi.org/10.18608/jla.2021.7379","url":null,"abstract":"The adoption of learning analytics (LA) in complex educational systems is woven into sociocultural and technical challenges that have induced distrust in data and difficulties in scaling LA. This paper presents a study that investigated areas of distrust and threats to trustworthy LA through a series of consultations with teaching staff and students at a large UK university. Surveys and focus groups were conducted to explore participant expectations of LA. The observed distrust is broadly attributed to three areas: the subjective nature of numbers, the fear of power diminution, and approaches to design and implementation of LA. The paper highlights areas to maintain existing trust with policy procedures and areas to cultivate trust by engaging with tensions arising from the social process of LA.","PeriodicalId":145357,"journal":{"name":"J. Learn. Anal.","volume":"26 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133481121","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
J. Learn. Anal.Pub Date : 2021-10-01DOI: 10.18608/jla.2021.7261
Tom Olney, S. Walker, Carlton Wood, Anactoria Clarke
{"title":"Are We Living In LA (P)LA Land? Reporting on the Practice of 30 STEM Tutors in their Use of a Learning Analytics Implementation at the Open University","authors":"Tom Olney, S. Walker, Carlton Wood, Anactoria Clarke","doi":"10.18608/jla.2021.7261","DOIUrl":"https://doi.org/10.18608/jla.2021.7261","url":null,"abstract":"Most higher education institutions view their increasing use of learning analytics as having significant potential to improve student academic achievement, retention outcomes, and learning and teaching practice but the realization of this potential remains stubbornly elusive. While there is an abundance of published research on the creation of visualizations, dashboards, and predictive models, there has been little work done to explore the impact of learning analytics on the actual practice of teachers. Through the lens of social informatics (an approach that views the users of technologies as active social actors whose technological practices constitute a wider socio-technical system) this qualitative study reports on an investigation into the practice of 30 tutors in the STEM faculty at Europe’s largest distance learning organization, The Open University UK (OU). When asked to incorporate learning analytics (including predictive learning analytics) contained in the Early Alert Indicator (EAI) dashboard during the 2017–2018 academic year into their practice, we found that tutors interacted with this dashboard in certain unanticipated ways and developed three identifiable “shadow practices”.","PeriodicalId":145357,"journal":{"name":"J. Learn. Anal.","volume":"14 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125338671","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
J. Learn. Anal.Pub Date : 2021-10-01DOI: 10.18608/jla.2021.7161
Yingbin Zhang, L. Paquette, R. Baker, Jaclyn L. Ocumpaugh, Nigel Bosch, Gautam, Biswas, Anabil Munshi
{"title":"Can Strategic Behaviour Facilitate Confusion Resolution? The Interplay Between Confusion and Metacognitive Strategies in Betty's Brain","authors":"Yingbin Zhang, L. Paquette, R. Baker, Jaclyn L. Ocumpaugh, Nigel Bosch, Gautam, Biswas, Anabil Munshi","doi":"10.18608/jla.2021.7161","DOIUrl":"https://doi.org/10.18608/jla.2021.7161","url":null,"abstract":"Confusion may benefit learning when it is resolved or partially resolved. Metacognitive strategies (MS) may help learners to resolve confusion when it occurs during learning and problem solving. This study examined the relationship between confusion and MS that students evoked in Betty’s Brain, a computer-based learning-by-modelling environment where elementary and middle school students learn science by building causal maps. Participants were sixth graders. Emotion data were collected from real-time observations by trained researchers. MS and task performance information were determined by analyzing the action logs. Pre- and post-tests were used to assess learning gains. The results revealed that the use of MS was a function of the state of student confusion. However, confusion resolution was not related to MS behaviour, and MS did not moderate the effect of confusion on student task performance in Betty’s Brain or on learning gains.","PeriodicalId":145357,"journal":{"name":"J. Learn. Anal.","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129242920","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
J. Learn. Anal.Pub Date : 2021-09-03DOI: 10.18608/jla.2021.7345
Radek Pelánek
{"title":"Analyzing and Visualizing Learning Data: A System Designer's Perspective","authors":"Radek Pelánek","doi":"10.18608/jla.2021.7345","DOIUrl":"https://doi.org/10.18608/jla.2021.7345","url":null,"abstract":"In this work, we consider learning analytics for primary and secondary schools from the perspective of the designer of a learning system. We provide an overview of practically useful analytics techniques with descriptions of their applications and specific illustrations. We highlight data biases and caveats that complicate the analysis and its interpretation. Although we intentionally focus on techniques for internal use by designers, many of these techniques may inspire the development of dashboards for teachers or students. We also identify the consequences and challenges for research.","PeriodicalId":145357,"journal":{"name":"J. Learn. Anal.","volume":"33 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133022610","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
J. Learn. Anal.Pub Date : 2021-09-03DOI: 10.18608/jla.2021.7353
Elad Yacobson, Orly Fuhrman, S. Hershkovitz, Giora Alexandron
{"title":"De-identification is Insufficient to Protect Student Privacy, or - What Can a Field Trip Reveal?","authors":"Elad Yacobson, Orly Fuhrman, S. Hershkovitz, Giora Alexandron","doi":"10.18608/jla.2021.7353","DOIUrl":"https://doi.org/10.18608/jla.2021.7353","url":null,"abstract":"Learning analytics have the potential to improve teaching and learning in K–12 education, but as student data is increasingly being collected and transferred for the purpose of analysis, it is important to take measures that will protect student privacy. A common approach to achieve this goal is the de-identification of the data, meaning the removal of personal details that can reveal student identity. However, as we demonstrate, de-identification alone is not a complete solution. We show how we can discover sensitive information about students by linking de-identified datasets with publicly available school data, using unsupervised machine learning techniques. This underlines that de-identification alone is insufficient if we wish to further learning analytics in K–12 without compromising student privacy.","PeriodicalId":145357,"journal":{"name":"J. Learn. Anal.","volume":"32 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128951844","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
J. Learn. Anal.Pub Date : 2021-09-03DOI: 10.18608/jla.2021.7357
M. Rodríguez-Triana, L. Prieto, Y. Dimitriadis, T. Jong, D. Gillet
{"title":"ADA for IBL: Lessons Learned in Aligning Learning Design and Analytics for Inquiry-Based Learning Orchestration","authors":"M. Rodríguez-Triana, L. Prieto, Y. Dimitriadis, T. Jong, D. Gillet","doi":"10.18608/jla.2021.7357","DOIUrl":"https://doi.org/10.18608/jla.2021.7357","url":null,"abstract":"Orchestrating technology-enhanced learning is a difficult task, especially in demanding pedagogical approaches like inquiry-based learning (IBL). To foster effective teacher adoption, both the complexity of designing IBL activities and the uncertainty about the student learning path during enactment need to be addressed. Previous research suggests that aligning learning design and learning analytics can be an effective way to provide such orchestration support. This paper reports on a design-based research (DBR) project to explore teachers’ orchestration needs in Go-Lab (a technological ecosystem for IBL used by thousands of primary and secondary school teachers around the world), and on how solutions that align learning design and analytics can fulfill such needs. The analysis of data from multiple events (including surveys, case studies, workshops with teachers, and platform usage analyses) led to a catalogue of IBL orchestration needs that can be tackled by aligning learning design and analytics, as well as a list of guidelines for technology development aiming to support IBL orchestration. These two contributions can support the creation of future learning analytics–enhanced IBL environments that are both pedagogically grounded and usable by teachers in authentic settings.","PeriodicalId":145357,"journal":{"name":"J. Learn. Anal.","volume":"36 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124511912","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
J. Learn. Anal.Pub Date : 2021-09-03DOI: 10.18608/jla.2021.7325
A. V. Leeuwen, C. K. Campen, I. Molenaar, N. Rummel
{"title":"How Teacher Characteristics Relate to How Teachers Use Dashboards: Results From Two Case Studies in K-12","authors":"A. V. Leeuwen, C. K. Campen, I. Molenaar, N. Rummel","doi":"10.18608/jla.2021.7325","DOIUrl":"https://doi.org/10.18608/jla.2021.7325","url":null,"abstract":"Teacher dashboards are a specific form of analytics in which visual displays provide teachers with information about their students; for example, concerning student progress and performance on tasks during lessons or lectures. In the present paper, we focus on the role of teacher dashboards in the context of teacher decision-making in K–12 education. There is large variation in teacher dashboard use in the classroom, which could be explained by teacher characteristics. Therefore, we investigate the role of teacher characteristics — such as experience, age, gender, and self-efficacy — in how teachers use dashboards. More specifically, we present two case studies to understand how diversity in teacher dashboard use is related to teacher characteristics. Surprisingly, in both case studies, teacher characteristics were not associated with dashboard use. Based on our findings, we propose an initial framework to understand what contributes to diversity of dashboard use. This framework might support future research to attribute diversity in dashboard use. This paper should be seen as a first step in examining the role of teacher characteristics in dashboard use in K–12 education.","PeriodicalId":145357,"journal":{"name":"J. Learn. Anal.","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117102684","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
J. Learn. Anal.Pub Date : 2021-09-03DOI: 10.18608/jla.2021.7543
Vitomir Kovanovíc, Claudia Mazziotti, J. Lodge
{"title":"Learning Analytics for Primary and Secondary Schools","authors":"Vitomir Kovanovíc, Claudia Mazziotti, J. Lodge","doi":"10.18608/jla.2021.7543","DOIUrl":"https://doi.org/10.18608/jla.2021.7543","url":null,"abstract":"Over the past decade, the increasing use of learning analytics opened the possibility of making data-driven decisions for improving student learning. Driven by the strong university adoption of learning analytics, most early learning analytics research focused on issues specific to tertiary education. With the broader adoption of educational technologies in primary and secondary education and the emergence of new classroom-focused technologies, there has been a growing awareness of the potentials of learning analytics for supporting students and diagnosing their learning progress in pre-university contexts. This special section focused on investigating, developing, and evaluating state-of-the-art learning analytics approaches within primary and secondary school settings. In this editorial, we summarize the papers of the special section and discuss the challenges and opportunities for learning analytics within the school context. We conclude with the discussion around the opportunities for future work and the implications of this special section for the field of learning analytics.","PeriodicalId":145357,"journal":{"name":"J. Learn. Anal.","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114328446","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
J. Learn. Anal.Pub Date : 2021-09-03DOI: 10.18608/jla.2021.7371
Elizabeth B. Cloude, Daniel Carpenter, Daryn A. Dever, R. Azevedo, James C. Lester
{"title":"Game-Based Learning Analytics for Supporting Adolescents' Reflection","authors":"Elizabeth B. Cloude, Daniel Carpenter, Daryn A. Dever, R. Azevedo, James C. Lester","doi":"10.18608/jla.2021.7371","DOIUrl":"https://doi.org/10.18608/jla.2021.7371","url":null,"abstract":"Reflection is critical for adolescents’ problem solving and learning in game-based learning environments (GBLEs). Yet challenges exist in the literature because most studies lack a theoretical perspective and clear operational definition to inform how and when reflection should be scaffolded during game-based learning. In this paper, we address these issues by studying the quantity and quality of 120 adolescents’ written reflections and their relation to their learning and problem solving with Crystal Island, a GBLE. Specifically, we (1) define reflection and how it relates to skill and knowledge acquisition; (2) review studies examining reflection and its relation to problem solving and learning with emerging technologies; and (3) provide direction for building reflection prompts into GBLEs that are aligned with the learning goals built into the learning session (e.g., learn about microbiology versus successfully solve a problem) to maximize adolescents’ reflection, learning, and performance. Overall, our findings emphasize how important it is to examine not only the quantity of reflection but also the depth of written reflection as it relates to specific learning goals. We discuss the implications of using game-learning analytics to guide instructional decision making in the classroom.","PeriodicalId":145357,"journal":{"name":"J. Learn. Anal.","volume":"13 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129305966","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}