{"title":"The effects of learning analytics hint system in supporting students problem-solving","authors":"Zilong Pan, Min Liu","doi":"10.1145/3506860.3506871","DOIUrl":"https://doi.org/10.1145/3506860.3506871","url":null,"abstract":"This mixed-method study examined the impacts of a learning-analytics (LA) hints system on middle school students’ problem-solving performance and self-efficacy (SE). Students in condition A received the LA hint system, students in condition B received a static hint system that contains the same set of hints but without the LA mechanism, condition C was a control group that no hints were provided. The statistical results showed that the problem-solving SE for students who engaged with the LA hint system improved significantly. Student interviews revealed that real-time supports and in-time positive feedback played key roles in supporting their SE growth. Moreover, student-generated quantitative and qualitative log data were collected for interpreting the research outcomes. The quantitative logs provided an in-depth examination of problem-solving strategies across the conditions while the qualitative logs provided another perspective to understand students’ problem-solving status. Implications for future implementation of LA-hint system in virtual PBL environments were provided.","PeriodicalId":185465,"journal":{"name":"LAK22: 12th International Learning Analytics and Knowledge Conference","volume":"44 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-03-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130715641","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
M. Khalil, Jacqueline Wong, Erkan Er, Martin Heitmann, Gleb Belokrys
{"title":"Tweetology of Learning Analytics: What does Twitter tell us about the trends and development of the field?","authors":"M. Khalil, Jacqueline Wong, Erkan Er, Martin Heitmann, Gleb Belokrys","doi":"10.1145/3506860.3506914","DOIUrl":"https://doi.org/10.1145/3506860.3506914","url":null,"abstract":"Twitter is a very popular microblogging platform that has been actively used by scientific communities to exchange scientific information and to promote scholarly discussions. The present study aimed to leverage the tweet data to provide valuable insights into the development of the learning analytics field since its initial days. Descriptive analysis, geocoding analysis, and topic modeling were performed on over 1.6 million tweets related to learning analytics posted between 2010-2021. The descriptive analysis reveals an increasing popularity of the field on the Twittersphere in terms of number of users, twitter posts, and hashtags emergence. The topic modeling analysis uncovers new insights of the major topics in the field of learning analytics. Emergent themes in the field were identified, and the increasing (e.g., Artificial Intelligence) and decreasing (e.g., Education) trends were shared. Finally, the geocoding analysis indicates an increasing participation in the field from more diverse countries all around the world. Further findings are discussed in the paper.","PeriodicalId":185465,"journal":{"name":"LAK22: 12th International Learning Analytics and Knowledge Conference","volume":"20 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-03-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125520367","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Knowledge Construction and Uncertainty in Real World Argumentation: A Text Analysis Approach","authors":"Ha Nguyen, William Young","doi":"10.1145/3506860.3506864","DOIUrl":"https://doi.org/10.1145/3506860.3506864","url":null,"abstract":"Collaborative argumentation is key to promoting understanding of scientific issues. However, classroom structures may not always prepare students to engage in argumentation. To address this challenge, education researchers have examined the importance of social knowledge construction and managing uncertainty in group understanding. In this study, we explore these processes using data from /r/ChangeMyView, an online forum on Reddit where users present their opinions, engage others in critiquing ideas, and acknowledge when the discussion has modified their opinions. This unfacilitated environment can illuminate how argumentation evolves naturally towards refined opinions. We employ automated text analyses (LIWC) and discourse analyses to understand the features and discourse sequences of successful arguments. We find that argumentative threads are more likely to be successful if they focus on idea articulation, coherence, and semantic diversity. Findings highlight the role of uncertainty: threads with more certainty words are less likely to be successful. Furthermore, successful arguments are characterized by cycles of raising, managing, and reducing uncertainty, with more occurrences of evidence and idea incorporation. We discuss how learning environments can create norms for idea construction, coherence, and uncertainty, and the potential to provide adaptive prompts to maintain and reduce uncertainty when unproductive argumentative sequences are detected.","PeriodicalId":185465,"journal":{"name":"LAK22: 12th International Learning Analytics and Knowledge Conference","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-03-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127586507","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Time-on-Task Estimation by data-driven Outlier Detection based on Learning Activities","authors":"D. Rotelli, A. Monreale","doi":"10.1145/3506860.3506913","DOIUrl":"https://doi.org/10.1145/3506860.3506913","url":null,"abstract":"Temporal analysis has been demonstrated to be relevant in Learning Analytics research, and capturing time-on-task, i.e., the amount of time spent by students in quality learning, as a proxy to model learning behaviour, predict performance, and avoid drop-out has been the focus of a number of investigations. Nonetheless, most studies do not provide enough information on how their data were prepared for their findings to be easily replicated, even though data pre-processing decisions have an impact on the analysis’ outcomes and can lead to inaccurate predictions. One of the key aspects in the preparation of learning data for temporal analysis is the detection of anomalous values of temporal duration of students’ activities. Most of the works in the literature address this problem without taking into account the fact that different activities can have very different typical execution times. In this paper, we propose a methodology for estimating time-on-task that starts with a well-defined data consolidation and then applies an outlier detection strategy to the data based on a distinct study of each learning activity and its peculiarities. Our real-world data experiments show that the proposed methodology outperforms the current state of the art, providing more accurate time estimations for students’ learning tasks.","PeriodicalId":185465,"journal":{"name":"LAK22: 12th International Learning Analytics and Knowledge Conference","volume":"28 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-03-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114296115","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Experimental Evidence of Performance Feedback vs. Mastery Feedback on Students’ Academic Motivation","authors":"Stephen J. Aguilar","doi":"10.1145/3506860.3506916","DOIUrl":"https://doi.org/10.1145/3506860.3506916","url":null,"abstract":"Work throughout the learning analytics community has examined associations between Learning Analytics Dashboard (LAD) features and a number of important student outcomes, including academic motivation and self-regulated learning strategies. While there are many potential implications of visualized academic information within a LAD on student outcomes, there remains an unanswered question: are there causal differences between showing performance information (e.g., comparing students’ progress to the class average) vs. mastery information (e.g., their individual score) on students’ motivation? Grounded in Achievement Goal Theory, this study answers this question experimentally by analyzing the difference between college students’ (n=445) reported achievement goal orientations as well as their motivated information seeking orientations after being presented with performance or mastery feedback. Results indicate that students in a performance condition which displayed ”above average” achievement on an academic measure reported lower performance-avoidance goals (e.g., not wanting to do worse than everyone else), and performance-avoidance information-seeking goals (e.g., not wanting to seek out information showing that one does worse than peers) when compared to students in the mastery control condition. This study contributes to our understanding of the motivational implications of academic feedback presented to students, and suggests that comparative information has direct effects on student motivation. Results thus uncover a potential tension between what might seem intuitive feedback to give students versus what might be more motivationally appropriate. The implications of this work point to the need to understand LADs not simply as feedback mechanisms, but as embedded features of a learning environment that influence how students engage with course content.","PeriodicalId":185465,"journal":{"name":"LAK22: 12th International Learning Analytics and Knowledge Conference","volume":"24 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-03-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114202696","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Namrata Srivastava, Yizhou Fan, Mladen Raković, Shaveen Singh, J. Jovanović, J. Graaf, Lyn Lim, Surya Surendrannair, J. Kilgour, I. Molenaar, M. Bannert, Johanna D. Moore, D. Gašević
{"title":"Effects of Internal and External Conditions on Strategies of Self-regulated Learning: A Learning Analytics Study","authors":"Namrata Srivastava, Yizhou Fan, Mladen Raković, Shaveen Singh, J. Jovanović, J. Graaf, Lyn Lim, Surya Surendrannair, J. Kilgour, I. Molenaar, M. Bannert, Johanna D. Moore, D. Gašević","doi":"10.1145/3506860.3506972","DOIUrl":"https://doi.org/10.1145/3506860.3506972","url":null,"abstract":"Self-regulated learning (SRL) skills are essential for successful learning in a technology-enhanced learning environment. Learning Analytics techniques have shown a great potential in identifying and exploring SRL strategies from trace data in various learning environments. However, these strategies have been mainly identified through analysis of sequences of learning actions, and thus interpretation of the strategies is heavily task and context dependent. Further, little research has been done on the association of SRL strategies with different influencing factors or conditions. To address these gaps, we propose an analytic method for detecting SRL strategies from theoretically supported SRL processes and applied the method to a dataset collected from a multi-source writing task. The detected SRL strategies were explored in terms of their association with the learning outcome, internal conditions (prior-knowledge, metacognitive knowledge and motivation) and external conditions (scaffolding). The study results showed our analytic method successfully identified three theoretically meaningful SRL strategies. The study results revealed small effect size in the association between the internal conditions and the identified SRL strategies, but revealed a moderate effect size in the association between external conditions and the SRL strategy use.","PeriodicalId":185465,"journal":{"name":"LAK22: 12th International Learning Analytics and Knowledge Conference","volume":"17 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-03-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126653798","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Stanislav Pozdniakov, Roberto Martínez-Maldonado, Yi-Shan Tsai, M. Cukurova, Tom Bartindale, Peter Chen, Harrison Marshall, D. Richardson, D. Gašević
{"title":"The Question-driven Dashboard: How Can We Design Analytics Interfaces Aligned to Teachers’ Inquiry?","authors":"Stanislav Pozdniakov, Roberto Martínez-Maldonado, Yi-Shan Tsai, M. Cukurova, Tom Bartindale, Peter Chen, Harrison Marshall, D. Richardson, D. Gašević","doi":"10.1145/3506860.3506885","DOIUrl":"https://doi.org/10.1145/3506860.3506885","url":null,"abstract":"One of the ultimate goals of several learning analytics (LA) initiatives is to close the loop and support students’ and teachers’ reflective practices. Although there has been a proliferation of end-user interfaces (often in the form of dashboards), various limitations have already been identified in the literature such as key stakeholders not being involved in their design, little or no account for sense-making needs, and unclear effects on teaching and learning. There has been a recent call for human-centred design practices to create LA interfaces in close collaboration with educational stakeholders to consider the learning design, and their authentic needs and pedagogical intentions. This paper addresses the call by proposing a question-driven LA design approach to ensure that end-user LA interfaces explicitly address teachers’ questions. We illustrate the approach in the context of synchronous online activities, orchestrated by pairs of teachers using audio-visual and text-based tools (namely Zoom and Google Docs). This study led to the design and deployment of an open-source monitoring tool to be used in real-time by teachers when students work collaboratively in breakout rooms, and across learning spaces.","PeriodicalId":185465,"journal":{"name":"LAK22: 12th International Learning Analytics and Knowledge Conference","volume":"250 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-03-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122634545","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
R. F. Mello, G. Fiorentino, Hilário Oliveira, P. Miranda, Mladen Raković, D. Gašević
{"title":"Towards automated content analysis of rhetorical structure of written essays using sequential content-independent features in Portuguese","authors":"R. F. Mello, G. Fiorentino, Hilário Oliveira, P. Miranda, Mladen Raković, D. Gašević","doi":"10.1145/3506860.3506977","DOIUrl":"https://doi.org/10.1145/3506860.3506977","url":null,"abstract":"Brazilian universities have included essay writing assignments in the entrance examination procedure to select prospective students. The essay scorers manually look for the presence of required Rhetorical Structure Theory (RST) categories and evaluate essay coherence. However, identifying RST categories is a time-consuming task. The literature reported several attempts to automate the identification of RST categories in essays with machine learning. Still, previous studies have focused on using machine learning algorithms trained on content-dependent features that can diminish classification performance, leading to over-fitting and hindering model generalisability. Therefore, this paper proposes: (i) the analysis of state-of-the-art classifiers and content-independent features to the task of RST rhetorical moves; (ii) a new approach that considers the sequence of the text to extract features – i.e. sequential content-independent features; (iii) an empirical study about the generalisability of the machine learning models and sequential content-independent features for this context; (iv) the identification of the most predictive features for automated identification of RST categories in essays written in Portuguese. The best performing classifier, XGBoost, based on sequential content-independent features, outperformed the classifiers used in the literature and are based on traditional content-dependent features. The XGBoost classifier based on sequential content-independent features also reached promising accuracy when tested for generalisability.","PeriodicalId":185465,"journal":{"name":"LAK22: 12th International Learning Analytics and Knowledge Conference","volume":"69 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-03-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132792138","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"“Beautiful work, you're rock stars!”: Teacher Analytics to Uncover Discourse that Supports or Undermines Student Motivation, Identity, and Belonging in Classrooms","authors":"Nicholas C. Hunkins, Sean Kelly, S. D’Mello","doi":"10.1145/3506860.3506896","DOIUrl":"https://doi.org/10.1145/3506860.3506896","url":null,"abstract":"From carefully crafted messages to flippant remarks, warm expressions to unfriendly grunts, teachers’ behaviors set the tone, expectations, and attitudes of the classroom. Thus, it is prudent to identify the ways in which teachers foster motivation, positive identity, and a strong sense of belonging through inclusive messaging and other interactions. We leveraged a new coding of teacher supportive discourse in 156 video clips from 73 6th to 8th grade math teachers from the archival Measures of Effective Teaching (MET) project. We trained Random Forest classifiers using verbal (words used) and paraverbal (acoustic-prosodic cues, e.g., speech rate) features to detect seven features of teacher discourse (e.g., public admonishment, autonomy supportive messages) from transcripts and audio, respectively. While both modalities performed over chance guessing, the specific language content was more predictive than paraverbal cues (mean correlation = .546 vs. .276); combining the two yielded no improvement. We examined the most predictive cues in order to gain a deeper understanding of the underlying messages in teacher talk. We discuss implications of our work for teacher analytics tools that aim to provide educators and researchers with insight into supportive discourse.","PeriodicalId":185465,"journal":{"name":"LAK22: 12th International Learning Analytics and Knowledge Conference","volume":"249 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-03-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114286851","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Sambit Praharaj, Maren Scheffel, Marcel Schmitz, M. Specht, H. Drachsler
{"title":"Towards Collaborative Convergence: Quantifying Collaboration Quality with Automated Co-located Collaboration Analytics","authors":"Sambit Praharaj, Maren Scheffel, Marcel Schmitz, M. Specht, H. Drachsler","doi":"10.1145/3506860.3506922","DOIUrl":"https://doi.org/10.1145/3506860.3506922","url":null,"abstract":"Collaboration is one of the four important 21st-century skills. With the pervasive use of sensors, interest on co-located collaboration (CC) has increased lately. Most related literature used the audio modality to detect indicators of collaboration (such as total speaking time and turn taking). CC takes place in physical spaces where group members share their social (i.e., non-verbal audio indicators like speaking time, gestures) and epistemic space (i.e., verbal audio indicators like the content of the conversation). Past literature has mostly focused on the social space to detect the quality of collaboration. In this study, we focus on both social and epistemic space with an emphasis on the epistemic space to understand different evolving collaboration patterns and collaborative convergence and quantify collaboration quality. We conduct field trials by collecting audio recordings in 14 different sessions in a university setting while the university staff and students collaborate over playing a board game to design a learning activity. This collaboration task consists of different phases with each collaborating member having been assigned a pre-fixed role. We analyze the collected group speech data to do role-based profiling and visualize it with the help of a dashboard.","PeriodicalId":185465,"journal":{"name":"LAK22: 12th International Learning Analytics and Knowledge Conference","volume":"41 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-03-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132945316","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}