Hassan Khosravi, Olga Viberg, Vitomir Kovanović, Rebecca Ferguson
{"title":"Generative AI and Learning Analytics","authors":"Hassan Khosravi, Olga Viberg, Vitomir Kovanović, Rebecca Ferguson","doi":"10.18608/jla.2023.8333","DOIUrl":"https://doi.org/10.18608/jla.2023.8333","url":null,"abstract":"This editorial looks back at the Journal of Learning Analytics (JLA) in 2023 and forward to 2024. Considering the recent proliferation of large language models such as GPT4 and Bard, the first section of this editorial points to the need for robust Generative AI (GenAI) analytics, calling for consideration of how GenAI may impact learning analytics research and practice. The second section looks back over the past year, providing statistics on submissions and considering the cost of publication in an open-access journal.","PeriodicalId":36754,"journal":{"name":"Journal of Learning Analytics","volume":"49 21","pages":""},"PeriodicalIF":3.9,"publicationDate":"2023-12-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138949963","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Mary Francis, M. Avoseh, Karen Card, Lisa Newland, Kevin Streff
{"title":"Student Privacy and Learning Analytics","authors":"Mary Francis, M. Avoseh, Karen Card, Lisa Newland, Kevin Streff","doi":"10.18608/jla.2023.7975","DOIUrl":"https://doi.org/10.18608/jla.2023.7975","url":null,"abstract":"This single-site case study will seek to answer the following question: how is the concept of privacy addressed in relation to a student success information system within a small, public institution of higher education? Three themes were found within the inductive coding process, which used interviews, documentation, and videos as data resources. Overall, the case study shows an institution in the early stages of implementing a commercial learning analytics system and provides suggestions for how it can be more proactive in implementing privacy considerations in developing policies and procedures.","PeriodicalId":36754,"journal":{"name":"Journal of Learning Analytics","volume":"59 10","pages":""},"PeriodicalIF":3.9,"publicationDate":"2023-12-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139009599","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
D. Rotelli, Aleksandra Maslennikova, Anna Monreale
{"title":"Session-Based Time-Window Identification in Virtual Learning Environments","authors":"D. Rotelli, Aleksandra Maslennikova, Anna Monreale","doi":"10.18608/jla.2023.7911","DOIUrl":"https://doi.org/10.18608/jla.2023.7911","url":null,"abstract":"Students organize and manage their own learning time, choosing when, what, and how to study due to the flexibility of online learning. Each person has unique learning habits that define their behaviours and distinguish them from others. To investigate the temporal behaviour of students in online learning environments, we seek to identify suitable time-windows that could be used to investigate their temporal behaviour. First, we present a novel perspective for identifying different types of sessions based on individual needs. The majority of previous works address this issue by establishing an arbitrary session timeout threshold. In this paper, we propose an algorithm for determining the optimal threshold for a given session. Second, we use data-driven methods to support investigators in determining time-windows based on the identified sessions. To this end, we developed a visual tool that assists data scientists and researchers to determine the optimal settings for session identification and locating suitable time-windows.","PeriodicalId":36754,"journal":{"name":"Journal of Learning Analytics","volume":"43 4","pages":""},"PeriodicalIF":3.9,"publicationDate":"2023-12-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139008288","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Bayesian Generative Modelling of Student Results in Course Networks","authors":"Marcel Haas, Colin Caprani, Benji Van Beurden","doi":"10.18608/jla.2023.7957","DOIUrl":"https://doi.org/10.18608/jla.2023.7957","url":null,"abstract":"We present an innovative modelling technique that simultaneously constrains student performance, course difficulty, and the sensitivity with which a course can differentiate between students by means of grades. Grade lists are the only necessary ingredient. Networks of courses will be constructed where the edges are populations of students that took both connected course nodes. Using idealized experiments and two real-world data sets, we show that the model, even though simple in its set-up, can constrain the properties of courses very well, as long as some basic requirements in the data set are met: (1) significant overlap in student populations, and thus information exchange through the network; (2) non-zero variance in the grades for a given course; and (3) some correlation between grades for different courses. The model can then be used to evaluate a curriculum, a course, or even subsets of students for a very wide variety of applications, ranging from program accreditation to exam fraud detection. We publicly release the code with examples that fully recreate the results presented here.","PeriodicalId":36754,"journal":{"name":"Journal of Learning Analytics","volume":"7 11","pages":""},"PeriodicalIF":3.9,"publicationDate":"2023-12-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139009144","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Valentina Albano, D. Firmani, Luigi Laura, Jerin George Mathew, Anna Lucia Paoletti, Irene Torrente
{"title":"NLP-Based Management of Large Multiple-Choice Test Item Repositories","authors":"Valentina Albano, D. Firmani, Luigi Laura, Jerin George Mathew, Anna Lucia Paoletti, Irene Torrente","doi":"10.18608/jla.2023.7897","DOIUrl":"https://doi.org/10.18608/jla.2023.7897","url":null,"abstract":"Multiple-choice questions (MCQs) are widely used in educational assessments and professional certification exams. Managing large repositories of MCQs, however, poses several challenges due to the high volume of questions and the need to maintain their quality and relevance over time. One of these challenges is the presence of questions that duplicate concepts but are formulated differently. Such questions can indeed elude syntactic controls but provide no added value to the repository.In this paper, we focus on this specific challenge and propose a workflow for the discovery and management of potential duplicate questions in large MCQ repositories. Overall, the workflow comprises three main steps: MCQ preprocessing, similarity computation, and finally a graph-based exploration and analysis of the obtained similarity values. For the preprocessing phase, we consider three main strategies: (i) removing the list of candidate answers from each question, (ii) augmenting each question with the correct answer, or (iii) augmenting each question with all candidate answers. Then, we use deep learning–based natural language processing (NLP) techniques, based on the Transformers architecture, to compute similarities between MCQs based on semantics. Finally, we propose a new approach to graph exploration based on graph communities to analyze the similarities and relationships between MCQs in the graph. We illustrate the approach with a case study of the Competenze Digital program, a large-scale assessment project by the Italian government. ","PeriodicalId":36754,"journal":{"name":"Journal of Learning Analytics","volume":"34 14","pages":""},"PeriodicalIF":3.9,"publicationDate":"2023-12-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139007236","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Effectiveness of a Learning Analytics Dashboard for Increasing Student Engagement Levels","authors":"Gomathy Ramaswami, Teo Susnjak, A. Mathrani","doi":"10.18608/jla.2023.7935","DOIUrl":"https://doi.org/10.18608/jla.2023.7935","url":null,"abstract":"Learning Analytics Dashboards (LADs) are gaining popularity as a platform for providing students with insights into their learning behaviour patterns in online environments. Existing LAD studies are mainly centred on displaying students’ online behaviours with simplistic descriptive insights. Only a few studies have integrated predictive components, while none possess the ability to explain how the predictive models work and how they have arrived at specific conclusions for a given student. A further gap exists within existing LADs with respect to prescriptive analytics that generate data-driven feedback to students on how to adjust their learning behaviour. The LAD in this study attempts to address this gap and integrates a full spectrum of current analytics technologies for sense-making while anchoring them within theoretical educational frameworks. This study’s LAD (SensEnablr) was evaluated for its effectiveness in impacting learning in a student cohort at a tertiary institution. Our findings demonstrate that student engagement with learning technologies and course resources increased significantly immediately following interactions with the dashboard. Meanwhile, results showed that the dashboard boosted the respondents’ learning motivation levels and that the novel analytics insights drawn from predictive and prescriptive analytics were beneficial to their learning. This study, therefore, has implications for future research when investigating student outcomes and optimizing student learning using LAD technologies.","PeriodicalId":36754,"journal":{"name":"Journal of Learning Analytics","volume":"2 4","pages":""},"PeriodicalIF":3.9,"publicationDate":"2023-12-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139008502","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Enriching Multimodal Data","authors":"Yiqiu Zhou, Jina Kang","doi":"10.18608/jla.2023.7989","DOIUrl":"https://doi.org/10.18608/jla.2023.7989","url":null,"abstract":"Collaboration is a complex, multidimensional process; however, details of how multimodal features intersect and mediate group interactions have not been fully unpacked. Characterizing and analyzing the temporal patterns based on multimodal features is a challenging yet important work to advance our understanding of computer-supported collaborative learning (CSCL). This paper highlights the affordances, as well as the limitations, of different temporal approaches in terms of analyzing multimodal data. To tackle the remaining challenges, we present an empirical example of multimodal temporal analysis that leverages multi-level vector autoregression (mlVAR) to identify temporal patterns of the collaborative problem-solving (CPS) process in an immersive astronomy simulation. We extend previous research on joint attention with a particular focus on the added value from a multimodal, temporal account of the CPS process. We incorporate verbal discussion to contextualize joint attention, examine the sequential and contemporaneous associations between them, and identify significant differences in temporal patterns between low- and high-achieving groups. Our paper does the following: 1) creates interpretable multimodal group interaction patterns, 2) advances understanding of CPS through examination of verbal and non-verbal interactions, and 3) demonstrates the added value of a complete account of temporality including both duration and sequential order.","PeriodicalId":36754,"journal":{"name":"Journal of Learning Analytics","volume":"7 5","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135773698","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Mladen Rakovic, Nora'ayu Ahmad Uzir, Wannisa Matcha, Dragan Gašević, Brendan Eagan, Jelena Jovanović, David Williamson Shaffer, Abelardo Pardo
{"title":"Network Analytics to Unveil Links of Learning Strategies, Time Management, and Academic Performance in a Flipped Classroom","authors":"Mladen Rakovic, Nora'ayu Ahmad Uzir, Wannisa Matcha, Dragan Gašević, Brendan Eagan, Jelena Jovanović, David Williamson Shaffer, Abelardo Pardo","doi":"10.18608/jla.2023.7843","DOIUrl":"https://doi.org/10.18608/jla.2023.7843","url":null,"abstract":"Preparatory learning tasks are considered critical for student success in flipped classroom courses. However, less isknown regarding which learning strategies students use and when they use those strategies in a flipped classroomcourse. In this study, we aimed to address this research gap. In particular, we investigated mutual connectionsbetween learning strategies and time management, and their combined effects on students’ performance in flippedclassrooms. To this end, we harnessed a network analytic approach based on epistemic network analysis (ENA) toanalyze student trace data collected in an undergraduate engineering course (N = 290) with a flipped classroomdesign. Our findings suggest that high-performing students effectively used their study time and enacted learningstrategies mainly linked to formative and summative assessment tasks. The students in the low-performing groupenacted less diverse learning strategies and typically focused on video watching. We discuss several implicationsfor research and instructional practice.","PeriodicalId":36754,"journal":{"name":"Journal of Learning Analytics","volume":"189 10","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135371570","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Lixiang Yan, Linxuan Zhao, D. Gašević, Xinyu Li, Roberto Martínez-Maldonado
{"title":"Socio-spatial Learning Analytics in Co-located Collaborative Learning Spaces:","authors":"Lixiang Yan, Linxuan Zhao, D. Gašević, Xinyu Li, Roberto Martínez-Maldonado","doi":"10.18608/jla.2023.7991","DOIUrl":"https://doi.org/10.18608/jla.2023.7991","url":null,"abstract":"Socio-spatial learning analytics (SSLA) is an emerging area within learning analytics research that seeks to un-cover valuable educational insights from individuals’ social and spatial data traces. These traces are capturedautomatically through sensing technologies in physical learning spaces, and the research is commonly based onthe theoretical foundations of social constructivism and cultural anthropology. With its growing empirical basis, asystematic literature review is timely in order to provide educational researchers and practitioners with a detailedsummary of the emerging works and the opportunities enabled by SSLA. This paper presents a systematic review of25 peer-reviewed articles on SSLA published between 2011 and 2023. Descriptive, network, and thematic analyseswere conducted to identify the citation networks, essential components, opportunities, and challenges enabled bySSLA. The findings illustrated that SSLA provides the opportunity to (1) contribute unobtrusive and unsupervisedresearch methodologies, (2) support educators’ classroom orchestration through visualizations, (3) support learnerreflection with continuous and reliable evidence, (4) develop novel theories about social and collaborative learning,and (5) empower educational stakeholders with the quantitative data to evaluate different learning spaces. Thesefindings could support learning analytics and educational technology scholars and practitioners to better understandand utilize SSLA to support future educational research and practice.","PeriodicalId":36754,"journal":{"name":"Journal of Learning Analytics","volume":" ","pages":""},"PeriodicalIF":3.9,"publicationDate":"2023-09-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45739308","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Rebecca Ferguson, Hassan Khosravi, Vitomir Kovanovíc, Olga Viberg, A. Aggarwal, Matthieu J. S. Brinkhuis, S. Buckingham Shum, Lujie Karen Chen, H. Drachsler, Valerie A. Guerrero, Michael Hanses, Caitlin Hayward, Bentley G. Hicks, I. Jivet, Kirsty Kitto, René F. Kizilcec, J. Lodge, Catherine A. Manly, Rebecca L. Matz, M. Meaney, X. Ochoa, Brendan A. Schuetze, Marco Spruit, Max van Haastrecht, Anouschka van Leeuwen, Lars Van Rijn, Yi-Shan Tsai, Joshua Weidlich, K. Williamson, Veronica X. Yan
{"title":"Aligning the Goals of Learning Analytics with its Research Scholarship: An Open Peer Commentary Approach","authors":"Rebecca Ferguson, Hassan Khosravi, Vitomir Kovanovíc, Olga Viberg, A. Aggarwal, Matthieu J. S. Brinkhuis, S. Buckingham Shum, Lujie Karen Chen, H. Drachsler, Valerie A. Guerrero, Michael Hanses, Caitlin Hayward, Bentley G. Hicks, I. Jivet, Kirsty Kitto, René F. Kizilcec, J. Lodge, Catherine A. Manly, Rebecca L. Matz, M. Meaney, X. Ochoa, Brendan A. Schuetze, Marco Spruit, Max van Haastrecht, Anouschka van Leeuwen, Lars Van Rijn, Yi-Shan Tsai, Joshua Weidlich, K. Williamson, Veronica X. Yan","doi":"10.18608/jla.2023.8197","DOIUrl":"https://doi.org/10.18608/jla.2023.8197","url":null,"abstract":"<jats:p>NA</jats:p>","PeriodicalId":36754,"journal":{"name":"Journal of Learning Analytics","volume":"10 1","pages":"14-50"},"PeriodicalIF":3.9,"publicationDate":"2023-08-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"67542777","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}