Educational Assessment Evaluation and Accountability最新文献

Enhanced conceptual understanding through formative assessment: results of a randomized controlled intervention study in physics classes.

IF 2.8 3区教育学

Educational Assessment Evaluation and Accountability Pub Date : 2025-01-01 Epub Date: 2024-12-27 DOI: 10.1007/s11092-024-09445-6

Andreas Lichtenberger, Sarah I Hofer, Elsbeth Stern, Andreas Vaterlaus

{"title":"Enhanced conceptual understanding through formative assessment: results of a randomized controlled intervention study in physics classes.","authors":"Andreas Lichtenberger, Sarah I Hofer, Elsbeth Stern, Andreas Vaterlaus","doi":"10.1007/s11092-024-09445-6","DOIUrl":"10.1007/s11092-024-09445-6","url":null,"abstract":"While formative assessment is a widely valued instructional approach to support meaningful learning, putting it into classroom practice remains a challenge, also because the time resources required may conflict with other goals. In a cluster-randomized controlled intervention study with 29 teachers and 604 students (mean age 15.6 years) at secondary school level, we examined the yield of formative assessment with regard to students' conceptual understanding and quantitative problem-solving skills in physics. Ten teachers applied formative assessment (FA group) in a 14-lesson curriculum on kinematics after having undergone a training that focused on the implementation of multiple-choice concept questions together with monitoring tools, clicker sessions, and reflective lessons. In the frequent testing group (FT group), ten teachers had no training on formative assessment but implemented the same concept questions as those used in the FA group. Nine teachers taught kinematics in their traditional way (TT group). The results revealed that students in the FA group outperformed students in the other two groups in a test on conceptual understanding immediately after the intervention as well as 3 months later, whereas students from the FT group and the TT group did not differ. Importantly, a better conceptual understanding in the FA group was not at the expense of performance in quantitative problem solving, as students of this group better integrated both kinds of knowledge. Our study has shown that a short but well-structured formative assessment teacher training could unfold its potential in terms of students' learning of challenging content.Supplementary information: The online version contains supplementary material available at 10.1007/s11092-024-09445-6.","PeriodicalId":46725,"journal":{"name":"Educational Assessment Evaluation and Accountability","volume":"37 1","pages":"5-33"},"PeriodicalIF":2.8,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11958380/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143774607","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

IRTrees for skipping items in PIRLS 跳过 PIRLS 项目的 IRTrees

IF 3.9 3区教育学

Educational Assessment Evaluation and Accountability Pub Date : 2024-07-24 DOI: 10.1007/s11092-024-09439-4

Andrés Christiansen, Rianne Janssen

{"title":"IRTrees for skipping items in PIRLS","authors":"Andrés Christiansen, Rianne Janssen","doi":"10.1007/s11092-024-09439-4","DOIUrl":"https://doi.org/10.1007/s11092-024-09439-4","url":null,"abstract":"In international large-scale assessments, students may not be compelled to answer every test item: a student can decide to skip a seemingly difficult item or may drop out before the end of the test is reached. The way these missing responses are treated will affect the estimation of the item difficulty and student ability, and ultimately affect the country’s score. In the Progress in International Reading Literacy Study (PIRLS), incorrect answer substitution is used. This means that skipped and omitted items are treated as incorrect responses. In the present study, the effect of this approach is investigated. The data of 2006, 2011, and 2016 cycles of PIRLS were analyzed using IRTree models in which a sequential tree structure is estimated to model the full response process. Item difficulty, students’ ability, and country means were estimated and compared with results from a Rasch model using the standard PIRLS approach to missing values. Results showed that the IRTree model was able to disentangle the students’ ability and their propensity to skip items, reducing the correlation between ability and the proportion of skipped items in comparison to the Rasch model. Nevertheless, at the country level, the aggregated scores showed no important differences between models for the pooled sample, but some differences within countries across cycles.","PeriodicalId":46725,"journal":{"name":"Educational Assessment Evaluation and Accountability","volume":"9 1","pages":""},"PeriodicalIF":3.9,"publicationDate":"2024-07-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141785924","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

How representative is the Swedish PISA sample? A comparison of PISA and register data 瑞典国际学生评估项目（PISA）样本的代表性如何？PISA 与登记数据的比较

IF 3.9 3区教育学

Educational Assessment Evaluation and Accountability Pub Date : 2024-07-06 DOI: 10.1007/s11092-024-09438-5

Linda Borger, Stefan Johansson, Rolf Strietholt

引用次数: 0

Dimensions of teachers’ data literacy: A systematic review of literature from 1990 to 2021 教师的数据素养：对 1990 年至 2021 年文献的系统回顾

IF 3.9 3区教育学

Educational Assessment Evaluation and Accountability Pub Date : 2024-05-06 DOI: 10.1007/s11092-024-09435-8

Jihyun Lee, Dennis Alonzo, Kim Beswick, Jan Michael Vincent Abril, Adrian W. Chew, Cherry Zin Oo

{"title":"Dimensions of teachers’ data literacy: A systematic review of literature from 1990 to 2021","authors":"Jihyun Lee, Dennis Alonzo, Kim Beswick, Jan Michael Vincent Abril, Adrian W. Chew, Cherry Zin Oo","doi":"10.1007/s11092-024-09435-8","DOIUrl":"https://doi.org/10.1007/s11092-024-09435-8","url":null,"abstract":"The current study presents a systematic review of teachers’ data literacy, arising from a synthesis of 83 empirical studies published between 1990 to 2021. Our review identified 95 distinct indicators across five dimensions: (a) knowledge about data, (b) skills in using data, (c) dispositions towards data use, (d) data application for various purposes, and (e) data-related behaviors. Our findings indicate that teachers' data literacy goes beyond addressing the needs of supporting student learning and includes elements such as teacher reflection, collaboration, communication, and participation in professional development. Considering these findings, future policies should acknowledge the significance of teacher dispositions and behaviors in relation to data, recognizing that they are as important as knowledge and skills acquisition. Additionally, prioritizing the provision of system-level support to foster teacher collaboration within in-school professional development programs may prove useful in enhancing teachers’ data literacy.","PeriodicalId":46725,"journal":{"name":"Educational Assessment Evaluation and Accountability","volume":"51 1","pages":""},"PeriodicalIF":3.9,"publicationDate":"2024-05-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140884447","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Examining pre-service teachers’ feedback on low- and high-quality written assignments 研究职前教师对低质量和高质量书面作业的反馈意见

IF 3.9 3区教育学

Educational Assessment Evaluation and Accountability Pub Date : 2024-04-16 DOI: 10.1007/s11092-024-09432-x

Ignacio Máñez, Anastasiya A. Lipnevich, Carolina Lopera-Oquendo, Raquel Cerdán

{"title":"Examining pre-service teachers’ feedback on low- and high-quality written assignments","authors":"Ignacio Máñez, Anastasiya A. Lipnevich, Carolina Lopera-Oquendo, Raquel Cerdán","doi":"10.1007/s11092-024-09432-x","DOIUrl":"https://doi.org/10.1007/s11092-024-09432-x","url":null,"abstract":"Assessing student writing assignments and providing effective feedback are a complex pedagogical skill teacher candidates need to master. Scarce literature has closely examined the type of feedback that pre-service high-school teachers spontaneously deliver when assessing student writings, which is the main goal of our study. In a sample of 255 high school teacher candidates, we examined the type of feedback that they provided when assessing two writing assignments that differed in quality. One thousand eight hundred thirty-five comments were analyzed and coded into 11 sub-categories. Results showed that candidates’ feedback not only focused on task performance but also on the writing process. Although candidates provided critical and past-oriented evaluations frequently, they also crafted feedback in a neutral tone and included future-oriented suggestions. Further, feedback varied as a function of candidates’ gender, academic discipline, and students’ quality of writing. Teacher training programs may use this information to design resources to address nuances of feedback provision.","PeriodicalId":46725,"journal":{"name":"Educational Assessment Evaluation and Accountability","volume":"15 1","pages":""},"PeriodicalIF":3.9,"publicationDate":"2024-04-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140614642","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Legitimising capital: parent organisations and their resistance to testing in England 资本合法化：英格兰家长组织及其对考试的抵制

IF 3.9 3区教育学

Educational Assessment Evaluation and Accountability Pub Date : 2024-03-06 DOI: 10.1007/s11092-024-09431-y

引用次数: 0

Signal, error, or bias? exploring the uses of scores from observation systems 信号、误差还是偏差？探索观察系统评分的用途

IF 3.9 3区教育学

Educational Assessment Evaluation and Accountability Pub Date : 2024-01-30 DOI: 10.1007/s11092-024-09427-8

Mark White, Kirsti Klette

{"title":"Signal, error, or bias? exploring the uses of scores from observation systems","authors":"Mark White, Kirsti Klette","doi":"10.1007/s11092-024-09427-8","DOIUrl":"https://doi.org/10.1007/s11092-024-09427-8","url":null,"abstract":"Scores from observational measures of teaching have recently been put to many uses within school systems, including communicating a standard of practice and providing teacher feedback, identifying teachers for professional development, monitoring system equity, and making employment decisions. In each of these uses, observation scores are interpreted as representing some aspect of the enacted instruction or teachers’ capacity to enact instruction, as seen through the observation systems lens for understanding teaching quality. The quality of these interpretations, or the extent to which observation scores are composed of a signal that accurately reflects the interpretation, has important implications for the overall validity of uses of observation systems. Starting from an explicit conceptualization of instruction, this paper combines generalizability theory and hierarchical linear modelling approaches to decompose observation scores to explore the extent to which scores from observation systems are composed of signal, error, and bias across four different uses (i.e., teacher feedback, professional development, monitoring system equity, and employment decisions) of scores. We show that the quality of observation scores may depend more on what scores are interpreted as representing (i.e., the proposed use) than on the specific observation rubric being used. Further, we show that rater errors and biases are a major threat to any attempt to interpret observation scores as capturing the observation system’s understanding of teaching quality. We discuss implications for using scores from observation systems.","PeriodicalId":46725,"journal":{"name":"Educational Assessment Evaluation and Accountability","volume":"15 1","pages":""},"PeriodicalIF":3.9,"publicationDate":"2024-01-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139648079","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Key stakeholder voices: Investigating student perceptions of teachers’ use of assessment for learning 主要利益相关者的声音：调查学生对教师使用学习评价的看法

IF 3.9 3区教育学

Educational Assessment Evaluation and Accountability Pub Date : 2024-01-15 DOI: 10.1007/s11092-024-09428-7

Christel H. D. Wolterinck-Broekhuis, Cindy L. Poortman, Kim Schildkamp, Adrie J. Visscher

{"title":"Key stakeholder voices: Investigating student perceptions of teachers’ use of assessment for learning","authors":"Christel H. D. Wolterinck-Broekhuis, Cindy L. Poortman, Kim Schildkamp, Adrie J. Visscher","doi":"10.1007/s11092-024-09428-7","DOIUrl":"https://doi.org/10.1007/s11092-024-09428-7","url":null,"abstract":"Many schools aim to implement Assessment for Learning (AfL) to stimulate students to take more ownership of their learning and develop self-regulatory skills. This survey-based study is among the few in the field showing how students experience the extent of implementation of AfL, here in English language and mathematics classes in 12 Dutch secondary schools. Analysis showed no differences between the subjects. Students (N = 685) experienced activities to clarify the learning intentions and success criteria and eliciting evidence on their learning progress regularly (between 50–74% of the lessons). Students hardly ever experienced activities aimed at peer- and self-assessment (less than 25% of the lessons). However, cluster analysis revealed three distinct clusters related to the extent of AfL strategy use students experienced. Overall, we can conclude that AfL is not yet fully integrated into teaching practices. Teachers’ skills, knowledge and attitudes required to increase student engagement in AfL practices and strengthen students’ self-regulated learning need more attention in future teacher professional development trajectories.","PeriodicalId":46725,"journal":{"name":"Educational Assessment Evaluation and Accountability","volume":"15 1","pages":""},"PeriodicalIF":3.9,"publicationDate":"2024-01-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139469655","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Comparing the teaching quality of alternatively certified teachers and traditionally certified teachers: findings from a large-scale study 比较另类认证教师和传统认证教师的教学质量：一项大规模研究的结果

IF 3.9 3区教育学

Educational Assessment Evaluation and Accountability Pub Date : 2024-01-12 DOI: 10.1007/s11092-023-09426-1

Christin Lucksnat, Eric Richter, Sofie Henschel, Lars Hoffmann, Stefan Schipolowski, Dirk Richter

引用次数: 0

Re-contextualization of school quality assessment policies: an ethnography approach to SIMCE on Chilean disadvantaged schools 学校质量评估政策的再语境化：智利贫困学校 SIMCE 的人种学方法

IF 3.9 3区教育学

Educational Assessment Evaluation and Accountability Pub Date : 2023-12-28 DOI: 10.1007/s11092-023-09425-2

{"title":"Re-contextualization of school quality assessment policies: an ethnography approach to SIMCE on Chilean disadvantaged schools","authors":"","doi":"10.1007/s11092-023-09425-2","DOIUrl":"https://doi.org/10.1007/s11092-023-09425-2","url":null,"abstract":"<h3>Abstract</h3> Accountability policies through standardized testing are widespread in diverse educational systems. Based on an ethnographic research study, we sought to understand how a learning assessment policy, used for over three decades in primary and lower secondary education, is lived and interpreted in daily school life in Chile. This article analyzes how learning assessment policies are re-contextualized (translated and interpreted) in three elementary and high schools located in poor neighborhoods with different performance categorization according to students’ test results. We conducted interviews and observations with teachers and school leadership team members. Results show that school actors experience the effects of testing both as a menace and as a pressure despite their different school categorization performance. They consider the tests a decontextualized and unjust measurement and therefore respond with similar strategies that seek to avoid these negative consequences. This study contributes to understanding the effects of tight accountability systems that attempt to place different levels of pressure through categorization.","PeriodicalId":46725,"journal":{"name":"Educational Assessment Evaluation and Accountability","volume":"25 1","pages":""},"PeriodicalIF":3.9,"publicationDate":"2023-12-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139078118","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0