{"title":"Studying with Learners’ Own Music: Preliminary Findings on Concentration and Task Load","authors":"Fanjie Li, Zuo Wang, Tzi-Dong Jeremy Ng, Xiao Hu","doi":"10.1145/3448139.3448206","DOIUrl":"https://doi.org/10.1145/3448139.3448206","url":null,"abstract":"Through profiling learners’ music usage in everyday learning settings and depicting their learning experience when studying with a music app powered by a large-scale and real-world music library, this study revealed preliminary observations on how background music impacts learning under varying task load, and manifested intriguing patterns of learners’ music usage and music preferences in various task load conditions. Specifically, we piloted a three-day field experiment in students’ everyday learning environment. During the experiment, participants performed learning tasks with music in the background and completed a set of online surveys before and after each learning session. Our results suggested that learners’ self-selected, real-life background music could enhance their learning effectiveness, while the beneficial effect of background music was more apparent when the learning task was less mentally or temporally demanding. Towards a closer look at the characteristics of preferable music pieces under various task load conditions, our findings showed that music preferred by participants under high versus low temporal demand differs in a number of characteristics, including speechiness, acousticness, danceability, and energy. This study further reveals the effects of background music on learning under varying task load levels and provides implications for context-aware background music selection when designing musically enriched learning environments.","PeriodicalId":108600,"journal":{"name":"LAK21: 11th International Learning Analytics and Knowledge Conference","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-04-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128315816","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Stakeholder Informed Professional Development Framework to Support Engagement with Learning Analytics","authors":"G. Gray, A. Schalk, Pauline Rooney, Charles Lang","doi":"10.1145/3448139.3448162","DOIUrl":"https://doi.org/10.1145/3448139.3448162","url":null,"abstract":"This paper reports on a study aimed at identifying training requirements for both staff and students in higher education to enable more widespread use of learning analytics. Opinions of staff and students were captured through ten focus groups (37 students; 40 staff) and two surveys (1,390 students; 160 staff). Participants were predominantly from two higher education institutions in Ireland. Analysis of the results informed a framework for continuous professional development in learning analytics focusing on aspects of using data, legal and ethical considerations, policy, and workload. The framework presented here differentiates between the training needs of students, academic staff and professional services staff.","PeriodicalId":108600,"journal":{"name":"LAK21: 11th International Learning Analytics and Knowledge Conference","volume":"21 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-04-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128222900","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Kirk P. Vanacore, Kevin Dieter, Lisa B. Hurwitz, Jamie Studwell
{"title":"Longitudinal Clusters of Online Educator Portal Access: Connecting Educator Behavior to Student Outcomes","authors":"Kirk P. Vanacore, Kevin Dieter, Lisa B. Hurwitz, Jamie Studwell","doi":"10.1145/3448139.3448195","DOIUrl":"https://doi.org/10.1145/3448139.3448195","url":null,"abstract":"The rising prevalence of blended learning programs has provided educators with an abundance of information about students' specific educational needs through educator portals. Full implementation of blended learning models requires educators to utilize these data to inform their teaching practices, yet most research on blended learning programs focuses solely on student engagement with the digital learning environment. In this paper, we utilize a longitudinal clustering method to identify patterns of educator portal usage and examine the associations between these clusters and student program outcomes. The clusters of educators varied in intensity and consistency of educator portal access across a school year and were associated with significant differences in student usage and progress in the program. The analyses allowed us to identify preferable educator usage patterns based upon their associated students’ program outcomes, which provides novel information about the potential impact of educator engagement on overall implementation fidelity of blended learning programs.","PeriodicalId":108600,"journal":{"name":"LAK21: 11th International Learning Analytics and Knowledge Conference","volume":"291 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-04-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123553656","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Timing of Support in One-on-one Math Problem Solving Coaching: A Survival Analysis Approach with Multimodal Data","authors":"L. Chen","doi":"10.1145/3448139.3448197","DOIUrl":"https://doi.org/10.1145/3448139.3448197","url":null,"abstract":"In this paper, we explore a kind of teaching-oriented temporal analytics on the timing of support in the context of one-on-one math problem-solving coaching. We build the analytical framework upon the human-human multimodal interaction data collected from the naturalist environments. We demonstrated the potential utility of leveraging survival analysis, a class of statistical methods to model time-to-event data, to gain insights into the timing decisions. We shed light on the heterogeneity of coaching decisions as to when to render support in connection to the problem-solving stages, coaching dyads, as well as the pre-intervention event characteristics. This work opens future avenues into a different type of human tutoring study supported by multimodal data, computational models, and statistical frameworks. This model framework may yield useful reflective teaching analytics to tutors, coaches, or teachers when further developed. We also envision that those analyses may ultimately inform the design of AI-supported autonomous agents that could learn the tutorial interaction logic from empirical data.","PeriodicalId":108600,"journal":{"name":"LAK21: 11th International Learning Analytics and Knowledge Conference","volume":"98 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-04-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129548780","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Yet Another Predictive Model? Fair Predictions of Students’ Learning Outcomes in an Online Math Learning Platform","authors":"Chenglu Li, Wanli Xing, Walter L. Leite","doi":"10.1145/3448139.3448200","DOIUrl":"https://doi.org/10.1145/3448139.3448200","url":null,"abstract":"To support online learners at a large scale, extensive studies have adopted machine learning (ML) techniques to analyze students’ artifacts and predict their learning outcomes automatically. However, limited attention has been paid to the fairness of prediction with ML in educational settings. This study intends to fill the gap by introducing a generic algorithm that can orchestrate with existing ML algorithms while yielding fairer results. Specifically, we have implemented logistic regression with the Seldonian algorithm and compared the fairness-aware model with fairness-unaware ML models. The results show that the Seldonian algorithm can achieve comparable predictive performance while producing notably higher fairness.","PeriodicalId":108600,"journal":{"name":"LAK21: 11th International Learning Analytics and Knowledge Conference","volume":"3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-04-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129904774","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Why Birds of a Feather Flock Together: Factors Triaging Students in Online Forums","authors":"Oleksandra Poquet","doi":"10.1145/3448139.3448185","DOIUrl":"https://doi.org/10.1145/3448139.3448185","url":null,"abstract":"Peer effects, an influence that peers can have on one’s learning and development, have been shown to affect student achievement and attitudes. A large-scale analysis of social influences in digital online interactions showed that students interact in online university forums with peers of similar performance. Mechanisms driving this observed similarity remain unclear. To shed light as to why similar peers interact online, the current study examined the role of organizing factors in the formation of similarity patterns in online university forums, using four-years of forum interaction data of a university cohort. In the study, experiments randomized the timing of student activity, relationship between student activity levels within specific courses, and relationship between student activity and performance. Analysis suggests that similarity between students interacting online is shaped by implications of the course design on individual student behaviour, less so by social processes of selection. Social selection may drive observed similarity in later years of student experience, but its role is relatively small compared to other factors. The results highlight the need to consider what social influences are enacted by the course design and technological scaffolding of learner behaviour in online interactions, towards diversifying student social influences.","PeriodicalId":108600,"journal":{"name":"LAK21: 11th International Learning Analytics and Knowledge Conference","volume":"52 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-04-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121280584","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Einat Shusterman, H. G. Kim, M. Facciotti, Michele Igo, Kamali N. Sripathi, David R Karger, A. Segal, Y. Gal
{"title":"Seeding Course Forums using the Teacher-in-the-Loop","authors":"Einat Shusterman, H. G. Kim, M. Facciotti, Michele Igo, Kamali N. Sripathi, David R Karger, A. Segal, Y. Gal","doi":"10.1145/3448139.3448142","DOIUrl":"https://doi.org/10.1145/3448139.3448142","url":null,"abstract":"Online forums are an integral part of modern day courses, but motivating students to participate in educationally beneficial discussions can be challenging. Our proposed solution is to initialize (or “seed”) a new course forum with comments from past instances of the same course that are intended to trigger discussion that is beneficial to learning. In this work, we develop methods for selecting high-quality seeds and evaluate their impact over one course instance of a 186-student biology class. We designed a scale for measuring the “seeding suitability” score of a given thread (an opening comment and its ensuing discussion). We then constructed a supervised machine learning (ML) model for predicting the seeding suitability score of a given thread. This model was evaluated in two ways: first, by comparing its performance to the expert opinion of the course instructors on test/holdout data; and second, by embedding it in a live course, where it was actively used to facilitate seeding by the course instructors. For each reading assignment in the course, we presented a ranked list of seeding recommendations to the course instructors, who could review the list and filter out seeds with inconsistent or malformed content. We then ran a randomized controlled study, in which one group of students was shown seeds that were recommended by the ML model, and another group was shown seeds that were recommended by an alternative model that ranked seeds purely by the length of discussion that was generated in previous course instances. We found that the group of students that received posts from either seeding model generated more discussion than a control group in the course that did not get seeded posts. Furthermore, students who received seeds selected by the ML-based model showed higher levels of engagement, as well as greater learning gains, than those who received seeds ranked by length of discussion.","PeriodicalId":108600,"journal":{"name":"LAK21: 11th International Learning Analytics and Knowledge Conference","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-04-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115993026","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Gloria Milena Fernández Nieto, Roberto Martínez Maldonado, Kirsty Kitto, S. B. Shum
{"title":"Modelling Spatial Behaviours in Clinical Team Simulations using Epistemic Network Analysis: Methodology and Teacher Evaluation","authors":"Gloria Milena Fernández Nieto, Roberto Martínez Maldonado, Kirsty Kitto, S. B. Shum","doi":"10.1145/3448139.3448176","DOIUrl":"https://doi.org/10.1145/3448139.3448176","url":null,"abstract":"In nursing education through team simulations, students must learn to position themselves correctly in coordination with colleagues. However, with multiple student teams in action, it is difficult for teachers to give detailed, timely feedback on these spatial behaviours to each team. Indoor-positioning technologies can now capture student spatial behaviours, but relatively little work has focused on giving meaning to student activity traces, transforming low-level x/y coordinates into language that makes sense to teachers. Even less research has investigated if teachers can make sense of that feedback. This paper therefore makes two contributions. (1) Methodologically, we document the use of Epistemic Network Analysis (ENA) as an approach to model and visualise students’ movements. To our knowledge, this is the first application of ENA to analyse human movement. (2) We evaluated teachers’ responses to ENA diagrams through qualitative analysis of video-recorded sessions. Teachers constructed consistent narratives about ENA diagrams’ meaning, and valued the new insights ENA offered. However, ENA’s abstract visualisation of spatial behaviours was not intuitive, and caused some confusions. We propose, therefore, that the power of ENA modelling can be combined with other spatial representations such as a classroom map, by overlaying annotations to create a more intuitive user experience.","PeriodicalId":108600,"journal":{"name":"LAK21: 11th International Learning Analytics and Knowledge Conference","volume":"46 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-04-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130985229","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Destiny Williams-Dobosz, R. F. Azevedo, Amos Jeng, Vyom Thakkar, Suma Bhat, Nigel Bosch, M. Perry
{"title":"A Social Network Analysis of Online Engagement for College Students Traditionally Underrepresented in STEM","authors":"Destiny Williams-Dobosz, R. F. Azevedo, Amos Jeng, Vyom Thakkar, Suma Bhat, Nigel Bosch, M. Perry","doi":"10.1145/3448139.3448159","DOIUrl":"https://doi.org/10.1145/3448139.3448159","url":null,"abstract":"Little is known about the online learning behaviors of students traditionally underrepresented in STEM fields (i.e., UR-STEM students), as well as how those behaviors impact important learning outcomes. The present study examined the relationship between online discussion forum engagement and success for UR-STEM and non-UR-STEM students, using the Community of Inquiry (CoI) model as our theoretical framework. Social network analysis and nested regression models were used to explore how three different measures of forum engagement—1) total number of posts written, 2) number of help-seeking posts written and replied to, and 3) level of connectivity—were related to improvement (i.e., relative performance gains) for 70 undergraduate students enrolled in an online introductory STEM course. We found a significant positive relationship between help-seeking and improvement and nonsignificant effects of general posting and connectivity; these results held for UR-STEM and non-UR-STEM students alike. Our findings suggest that online help-seeking has benefits for course improvement beyond what can be predicted by posting alone and that one need not be well connected in a class network to achieve positive learning outcomes. Finally, UR-STEM students demonstrated greater grade improvement than their non-UR-STEM counterparts, which suggests that the online environment has the potential to combat barriers to success that disproportionately affect underrepresented students.","PeriodicalId":108600,"journal":{"name":"LAK21: 11th International Learning Analytics and Knowledge Conference","volume":"92 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-04-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126589490","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Harrison Kyriacou, Anand Ramakrishnan, J. Whitehill
{"title":"Learning to Work in a Materials Recovery Facility: Can Humans and Machines Learn from Each Other?","authors":"Harrison Kyriacou, Anand Ramakrishnan, J. Whitehill","doi":"10.1145/3448139.3448183","DOIUrl":"https://doi.org/10.1145/3448139.3448183","url":null,"abstract":"Workplace learning often requires workers to learn new perceptual and motor skills. The future of work will increasingly feature human users who cooperate with machines, both to learn the tasks and to perform them. In this paper, we examine workplace learning in Materials Recovery Facilities (MRFs), i.e., recycling plants, where workers separate waste items on conveyer belts before they are formed into bales and reprocessed. Using a simulated MRF, we explored the benefit of machine learning assistants (MLAs) that help workers, and help train them, to sort objects efficiently by providing automated perceptual guidance. In a randomized experiment (n = 140), we found: (1) A low-accuracy MLA is worse than no MLA at all, both in terms of task performance and learning. (2) A perfect MLA led to the best task performance, but was no better in helping users to learn than having no MLA at all. (3) Users tend to follow the MLA’s judgments too often, even when they were incorrect. Finally, (4) we devised a novel learning analytics algorithm to assess the worker’s accuracy, with the goal of obtaining additional training labels that can be used for fine-tuning the machine. A simulation study illustrates how even noisy labels can increase the machine’s accuracy.","PeriodicalId":108600,"journal":{"name":"LAK21: 11th International Learning Analytics and Knowledge Conference","volume":"16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-04-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126738565","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}