Meiling Jin , Qiang Jiang , Weiyan Xiong , Qi Li , Yanan Feng , Wei Zhao
{"title":"Investigating behavioral patterns to facilitate performance predictions during online peer assessment through learning analytics approach","authors":"Meiling Jin , Qiang Jiang , Weiyan Xiong , Qi Li , Yanan Feng , Wei Zhao","doi":"10.1016/j.stueduc.2024.101394","DOIUrl":"10.1016/j.stueduc.2024.101394","url":null,"abstract":"<div><p>Insights into student behavioral patterns yield benefits for both educators and learners. Nevertheless, only a few studies have investigated the behavioral patterns of high-scoring (HS) and low-scoring (LS) students to facilitate predictions during peer assessment (PA). Therefore, we performed learning analytics to explore the behavioral patterns of HS and LS students in 52 university students from affective, cognitive, and metacognitive perspectives as these students engage in online PAs. The results indicated that on the affective dimension, HS students tended to exhibit negative affection while LS students tended to display positive affection. On the cognitive dimension, HS students demonstrated more intricate transformations compared to the LS students. Regarding the metacognitive dimension, LS students more likely reflected upon and accepted the reviews provided by others than the HS. Additionally, the findings also revealed that HS students were supported by focused social networks and overall stable activity engagement, whereas the LS students were less engaged at the beginning of the activity but had evolving social networks and were more engaged in the later stages of the activity. These findings offer valuable insights for educators in effectively predicting student performance based on behavioral patterns while providing new perspectives to support instructional decision-making processes.</p></div>","PeriodicalId":47539,"journal":{"name":"Studies in Educational Evaluation","volume":"83 ","pages":"Article 101394"},"PeriodicalIF":2.6,"publicationDate":"2024-08-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141940888","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Understanding changes in learner engagement with peer feedback giving in EFL writing: A longitudinal study","authors":"Jing Li , Lianjiang Jiang","doi":"10.1016/j.stueduc.2024.101391","DOIUrl":"10.1016/j.stueduc.2024.101391","url":null,"abstract":"<div><p>This longitudinal case study investigates three EFL learners’ engagement changes with peer feedback giving from affective, behavioral and cognitive perspectives over an academic year. Data were collected from multiple sources, including learners’ texts, feedback messages, open-ended questionnaire, stimulated recalls, and semi-structured interviews. The findings show that the three dimensions of learner engagement with peer feedback giving interacted with each other dynamically during the academic year. While learners’ positive affective engagement promoted their cognitive and behavioral engagement, their negative feelings did not necessarily restrain the enhancement of the other two aspects. In turn, learners’ cognitive and behavioral engagement could either promote or hinder their affective engagement. This study advances our understanding of the multifaceted and dynamic nature of learner engagement with peer feedback giving. It provides pedagogical implications for writing educators to develop teaching strategies to promote learner engagement and maximize learning opportunities in the feedback giving process.</p></div>","PeriodicalId":47539,"journal":{"name":"Studies in Educational Evaluation","volume":"83 ","pages":"Article 101391"},"PeriodicalIF":2.6,"publicationDate":"2024-08-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141940889","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Research on the effect of mixed teaching strategies on students' ambidextrous innovation","authors":"Xiao Meng , Deqiang Niu , Lili Ding , Lei Wang","doi":"10.1016/j.stueduc.2024.101390","DOIUrl":"10.1016/j.stueduc.2024.101390","url":null,"abstract":"<div><p>Mixed teaching strategies have caused a fierce debate about teaching quality in China's higher education. Based on theories of constructivism and self-regulation, we examine the impact of mixed teaching strategies on learners' ambidextrous innovation capacities. Utilizing a questionnaire survey of 412 undergraduate and graduate students from 36 Chinese higher education institutions, this research demonstrates that combining metacognitive strategies with motivational and collaborative approaches enhances exploratory innovation ability, while the pairing of metacognitive and collaborative strategies promotes exploitative innovation. Furthermore, the study identifies significant differences in the effectiveness of mixed teaching strategies based on students' self-efficacy and teaching acceptance levels. These findings highlight the importance of tailoring mixed teaching strategies to individual characteristics for optimal learning outcomes and fostering students' innovative potential. This research contributes to the discourse on effective teaching practices in Chinese higher education by proposing a mixed teaching strategies matrix informed by self-efficacy and teaching acceptance, offering valuable guidance for educators seeking to cultivate students' innovative abilities.</p></div>","PeriodicalId":47539,"journal":{"name":"Studies in Educational Evaluation","volume":"83 ","pages":"Article 101390"},"PeriodicalIF":2.6,"publicationDate":"2024-08-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141961293","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Improving sensitivity to change in self-report items: The role of alignment and item formulation in a longitudinal evaluation","authors":"Sonja Hahn , Samuel Merk , Olga Kunina-Habenicht","doi":"10.1016/j.stueduc.2024.101387","DOIUrl":"10.1016/j.stueduc.2024.101387","url":null,"abstract":"<div><p>The capability of scales to detect change, i.e. sensitivity to change, is important in longitudinal studies. We present a quantitative study comparing the effects of alignment to course content and item granularity (generic vs. concrete wording) on the capability of different scales to capture changes in ICT self-efficacy. In an evaluation of a course on teaching with ICT, we used ICT self-report questionnaires for pre-service teachers at the beginning and end of a one-semester in a treatment and a control group (<em>N</em><sub>intervention</sub> = 278 vs. <em>N</em><sub>control</sub> = 395). After propensity score matching on the imputed data, we applied descriptive, frequentist, and Bayesian methods to examine differential scale behavior. Results show more pronounced longitudinal changes for scales on instructional ICT self-efficacy, especially for scales formulated for concrete tasks. Our results suggest that better alignment and specific formulation enhance sensitivity to change in self-reports, and relate to scale construction and selection in educational evaluations.</p></div>","PeriodicalId":47539,"journal":{"name":"Studies in Educational Evaluation","volume":"83 ","pages":"Article 101387"},"PeriodicalIF":2.6,"publicationDate":"2024-08-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0191491X2400066X/pdfft?md5=d9a6313cea4b117a726077cb47b51e3c&pid=1-s2.0-S0191491X2400066X-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141940890","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Evaluating technological interventions for developing teamwork competency in higher education: A systematic review and meta-ethnography","authors":"Wenjie Hu , Cecilia Ka Yuk Chan","doi":"10.1016/j.stueduc.2024.101382","DOIUrl":"10.1016/j.stueduc.2024.101382","url":null,"abstract":"<div><p>Recent years have seen a surge in higher education educators employing technology to support students' holistic competency development. A rigorous evaluation is essential to ensure these technological interventions achieve their intended outcomes. This systematic review of 37 studies from January 2008 to March 2024 evaluates how technology assists in enhancing teamwork competencies in higher education. Using meta-ethnography, it synthesizes qualitative student feedback to assess the effectiveness and challenges of technology use. The findings reveal that technology integration helps students realize the importance of teamwork, understand team dynamics, develop practical skills, and foster team spirit. Additionally, technology's contribution to teamwork training includes boosting student engagement, facilitating the execution of team tasks, and advancing student self-reflection. However, technological interventions also present challenges, including increased training complexity, deviation from learning objectives, and negative impacts on motivation. These findings are crucial for educators aiming to develop effective technology-enhanced training programs to improve teamwork competencies.</p></div>","PeriodicalId":47539,"journal":{"name":"Studies in Educational Evaluation","volume":"83 ","pages":"Article 101382"},"PeriodicalIF":2.6,"publicationDate":"2024-07-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141940893","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Moderation effects of teacher support and disciplinary climate on SES and reading relation in high and low globally competitive economies","authors":"Leifeng Xiao , Aohua Ni , Kit-Tai Hau","doi":"10.1016/j.stueduc.2024.101389","DOIUrl":"10.1016/j.stueduc.2024.101389","url":null,"abstract":"<div><p>We examined how teacher support and disciplinary climate may have additional benefits for students from low (or high) socioeconomic status (SES) families in high and low globally competitive economies. Multilevel regression analyses on 184102 students from 6397 schools from diversified globally competitive economies in the Programme for International Student Assessment (PISA) showed that: (a) students in high SES families benefited more in reading in higher than lower globally competitive economies; (b) greater teacher support improved reading in high than low globally competitive economies; (c) teacher support would alleviate the negative impact of low SES on reading; and (d) a positive disciplinary climate enhances reading performance in both high and low globally competitive economies, with SES effects being more pronounced in schools with a positive disciplinary climate, especially in low globally competitive economies. Practical implications, limitations, and future research directions are discussed.</p></div>","PeriodicalId":47539,"journal":{"name":"Studies in Educational Evaluation","volume":"83 ","pages":"Article 101389"},"PeriodicalIF":2.6,"publicationDate":"2024-07-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141940891","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Do learners learn from corrective peer feedback? Insights from students","authors":"Xue Jiang , Sarah Solomon Ironsi","doi":"10.1016/j.stueduc.2024.101385","DOIUrl":"10.1016/j.stueduc.2024.101385","url":null,"abstract":"<div><p>Understanding what students do with the feedback they receive is paramount in the influence of feedback on the learning experience. Likewise, while research recommends peer feedback in classroom settings, understanding what students do with corrective peer feedback is instrumental in studying the impact of peer feedback on students learning. This study explored this understudied aspect of peer feedback through mixed-method research. One hundred fifty participants administered with peer feedback were recruited from 5 universities. After collecting data through questionnaires and interviews, it was analyzed. Results indicate that while corrective peer feedback is overly perceived as effective, students perceive it as an unfair and improper assessment model. Other novel findings were used to make pedagogical conclusions on this topic.</p></div>","PeriodicalId":47539,"journal":{"name":"Studies in Educational Evaluation","volume":"83 ","pages":"Article 101385"},"PeriodicalIF":2.6,"publicationDate":"2024-07-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141953954","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"New evidence on the impact of learning in a foreign language on educational outcomes","authors":"Luis Pires , María-Jesús Mancebón , Mauro Mediavilla , José-María Gómez-Sancho","doi":"10.1016/j.stueduc.2024.101386","DOIUrl":"10.1016/j.stueduc.2024.101386","url":null,"abstract":"<div><p>A Content and Language Integrated Learning (CLIL) method, in which some subjects are taught in a foreign language (English), was initiated in Spain in 2005 and has progressively extended to half of public schools. The results have been very positive; however, it has been argued that studying subjects in a foreign language may reduce the educational outcomes of students. This paper evaluates this criticism in the Madrid bilingual program adopting a Propensity Score Matching approach, with the PISA 2015 and 2018 data. The model defines a homogenous student subsample from bilingual and nonbilingual schools in terms of the observable characteristics that may jointly influence both the selection of school type and educational scores. Our results, robust to the sample and unobservable hidden bias, indicate that the Madrid bilingual program, in addition to improving students' English level, does not reduce the skills of subjects taught in English or in Spanish.</p></div>","PeriodicalId":47539,"journal":{"name":"Studies in Educational Evaluation","volume":"83 ","pages":"Article 101386"},"PeriodicalIF":2.6,"publicationDate":"2024-07-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0191491X24000658/pdfft?md5=c5de668aa4453e1ecd08f5c0bd14adfb&pid=1-s2.0-S0191491X24000658-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141960582","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Vivian E. Morssink-Santing , Symen van der Zee , Lida T. Klaver , Jaap de Brouwer , Patrick H.M. Sins
{"title":"The long-term effect of alternative education on self-regulated learning: A comparison between Montessori, Dalton, and traditional education","authors":"Vivian E. Morssink-Santing , Symen van der Zee , Lida T. Klaver , Jaap de Brouwer , Patrick H.M. Sins","doi":"10.1016/j.stueduc.2024.101380","DOIUrl":"https://doi.org/10.1016/j.stueduc.2024.101380","url":null,"abstract":"<div><p>With the rise of secondary alternative schools in the Netherlands, such as Montessori and Dalton, research into its effectiveness has become crucial. Alternative schools’ greater focus on autonomy and independence may better equip students with the self-regulated learning (SRL) skills essential for higher education. The aim of this study is to investigate the transition of secondary (alternative) education to higher education. More specifically, former Dalton (<em>N</em> = 184), Montessori (<em>N</em> = 150), and regular (<em>N</em> = 8614) secondary education students are compared on the perceived fit between secondary education and higher education, for example in regard to learning to work together and work alone, and SRL skills in a bootstrapped hierarchical regression. Results show that former Dalton students rate the fit of their secondary education to their higher education significantly better than students who attended regular education. No other significant differences were found when comparing Montessori and regular education students regarding fit, or when comparing both Montessori and Dalton to regular education regarding the SRL skills of students. Alternative explanations regarding the differences between alternative and regular educations are discussed.</p></div>","PeriodicalId":47539,"journal":{"name":"Studies in Educational Evaluation","volume":"83 ","pages":"Article 101380"},"PeriodicalIF":2.6,"publicationDate":"2024-07-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0191491X24000592/pdfft?md5=093d8493b1f1b5db3cde447c61b696d4&pid=1-s2.0-S0191491X24000592-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141541990","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Admission testing, pre-academic exam self-efficacy, and retention. A prospective cohort study","authors":"Lotte Dyhrberg O’Neill , Tine Nielsen","doi":"10.1016/j.stueduc.2024.101383","DOIUrl":"https://doi.org/10.1016/j.stueduc.2024.101383","url":null,"abstract":"<div><p>Several studies have indicated the presence of a retention effect of admission testing in higher education, but so far little is known about the possible mechanisms behind such an effect. The aim of this study was to examine the associations between admission testing, selected psychological attributes, program performance and early dropout at university for two admission tracks (grade-based admission and test-based admission). Students from five faculties were surveyed at baseline in a prospective cohort study design. Multivariate logistic regression was used to model first-year dropout for 499 students. Results indicated the presence of a retention effect of pre-academic exam self-efficacy (PAE-SE) for the subgroup of students admitted via test-based admission. Implications of the results for research and admission testing practice are discussed.</p></div>","PeriodicalId":47539,"journal":{"name":"Studies in Educational Evaluation","volume":"83 ","pages":"Article 101383"},"PeriodicalIF":2.6,"publicationDate":"2024-07-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0191491X24000622/pdfft?md5=05f388d3d57524cdc972241d373e491d&pid=1-s2.0-S0191491X24000622-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141541988","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}