Educational Assessment Evaluation and Accountability最新文献

IRTrees for skipping items in PIRLS 跳过 PIRLS 项目的 IRTrees

IF 3.9 3区教育学

Educational Assessment Evaluation and Accountability Pub Date : 2024-07-24 DOI: 10.1007/s11092-024-09439-4

Andrés Christiansen, Rianne Janssen

{"title":"IRTrees for skipping items in PIRLS","authors":"Andrés Christiansen, Rianne Janssen","doi":"10.1007/s11092-024-09439-4","DOIUrl":"https://doi.org/10.1007/s11092-024-09439-4","url":null,"abstract":"In international large-scale assessments, students may not be compelled to answer every test item: a student can decide to skip a seemingly difficult item or may drop out before the end of the test is reached. The way these missing responses are treated will affect the estimation of the item difficulty and student ability, and ultimately affect the country’s score. In the Progress in International Reading Literacy Study (PIRLS), incorrect answer substitution is used. This means that skipped and omitted items are treated as incorrect responses. In the present study, the effect of this approach is investigated. The data of 2006, 2011, and 2016 cycles of PIRLS were analyzed using IRTree models in which a sequential tree structure is estimated to model the full response process. Item difficulty, students’ ability, and country means were estimated and compared with results from a Rasch model using the standard PIRLS approach to missing values. Results showed that the IRTree model was able to disentangle the students’ ability and their propensity to skip items, reducing the correlation between ability and the proportion of skipped items in comparison to the Rasch model. Nevertheless, at the country level, the aggregated scores showed no important differences between models for the pooled sample, but some differences within countries across cycles.","PeriodicalId":46725,"journal":{"name":"Educational Assessment Evaluation and Accountability","volume":"9 1","pages":""},"PeriodicalIF":3.9,"publicationDate":"2024-07-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141785924","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

How representative is the Swedish PISA sample? A comparison of PISA and register data 瑞典国际学生评估项目（PISA）样本的代表性如何？PISA 与登记数据的比较

IF 3.9 3区教育学

Educational Assessment Evaluation and Accountability Pub Date : 2024-07-06 DOI: 10.1007/s11092-024-09438-5

Linda Borger, Stefan Johansson, Rolf Strietholt

引用次数: 0

Dimensions of teachers’ data literacy: A systematic review of literature from 1990 to 2021 教师的数据素养：对 1990 年至 2021 年文献的系统回顾

IF 3.9 3区教育学

Educational Assessment Evaluation and Accountability Pub Date : 2024-05-06 DOI: 10.1007/s11092-024-09435-8

Jihyun Lee, Dennis Alonzo, Kim Beswick, Jan Michael Vincent Abril, Adrian W. Chew, Cherry Zin Oo

{"title":"Dimensions of teachers’ data literacy: A systematic review of literature from 1990 to 2021","authors":"Jihyun Lee, Dennis Alonzo, Kim Beswick, Jan Michael Vincent Abril, Adrian W. Chew, Cherry Zin Oo","doi":"10.1007/s11092-024-09435-8","DOIUrl":"https://doi.org/10.1007/s11092-024-09435-8","url":null,"abstract":"The current study presents a systematic review of teachers’ data literacy, arising from a synthesis of 83 empirical studies published between 1990 to 2021. Our review identified 95 distinct indicators across five dimensions: (a) knowledge about data, (b) skills in using data, (c) dispositions towards data use, (d) data application for various purposes, and (e) data-related behaviors. Our findings indicate that teachers' data literacy goes beyond addressing the needs of supporting student learning and includes elements such as teacher reflection, collaboration, communication, and participation in professional development. Considering these findings, future policies should acknowledge the significance of teacher dispositions and behaviors in relation to data, recognizing that they are as important as knowledge and skills acquisition. Additionally, prioritizing the provision of system-level support to foster teacher collaboration within in-school professional development programs may prove useful in enhancing teachers’ data literacy.","PeriodicalId":46725,"journal":{"name":"Educational Assessment Evaluation and Accountability","volume":"51 1","pages":""},"PeriodicalIF":3.9,"publicationDate":"2024-05-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140884447","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Examining pre-service teachers’ feedback on low- and high-quality written assignments 研究职前教师对低质量和高质量书面作业的反馈意见

IF 3.9 3区教育学

Educational Assessment Evaluation and Accountability Pub Date : 2024-04-16 DOI: 10.1007/s11092-024-09432-x

Ignacio Máñez, Anastasiya A. Lipnevich, Carolina Lopera-Oquendo, Raquel Cerdán

{"title":"Examining pre-service teachers’ feedback on low- and high-quality written assignments","authors":"Ignacio Máñez, Anastasiya A. Lipnevich, Carolina Lopera-Oquendo, Raquel Cerdán","doi":"10.1007/s11092-024-09432-x","DOIUrl":"https://doi.org/10.1007/s11092-024-09432-x","url":null,"abstract":"Assessing student writing assignments and providing effective feedback are a complex pedagogical skill teacher candidates need to master. Scarce literature has closely examined the type of feedback that pre-service high-school teachers spontaneously deliver when assessing student writings, which is the main goal of our study. In a sample of 255 high school teacher candidates, we examined the type of feedback that they provided when assessing two writing assignments that differed in quality. One thousand eight hundred thirty-five comments were analyzed and coded into 11 sub-categories. Results showed that candidates’ feedback not only focused on task performance but also on the writing process. Although candidates provided critical and past-oriented evaluations frequently, they also crafted feedback in a neutral tone and included future-oriented suggestions. Further, feedback varied as a function of candidates’ gender, academic discipline, and students’ quality of writing. Teacher training programs may use this information to design resources to address nuances of feedback provision.","PeriodicalId":46725,"journal":{"name":"Educational Assessment Evaluation and Accountability","volume":"15 1","pages":""},"PeriodicalIF":3.9,"publicationDate":"2024-04-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140614642","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Legitimising capital: parent organisations and their resistance to testing in England 资本合法化：英格兰家长组织及其对考试的抵制

IF 3.9 3区教育学

Educational Assessment Evaluation and Accountability Pub Date : 2024-03-06 DOI: 10.1007/s11092-024-09431-y

引用次数: 0

Signal, error, or bias? exploring the uses of scores from observation systems 信号、误差还是偏差？探索观察系统评分的用途

IF 3.9 3区教育学

Educational Assessment Evaluation and Accountability Pub Date : 2024-01-30 DOI: 10.1007/s11092-024-09427-8

Mark White, Kirsti Klette

{"title":"Signal, error, or bias? exploring the uses of scores from observation systems","authors":"Mark White, Kirsti Klette","doi":"10.1007/s11092-024-09427-8","DOIUrl":"https://doi.org/10.1007/s11092-024-09427-8","url":null,"abstract":"Scores from observational measures of teaching have recently been put to many uses within school systems, including communicating a standard of practice and providing teacher feedback, identifying teachers for professional development, monitoring system equity, and making employment decisions. In each of these uses, observation scores are interpreted as representing some aspect of the enacted instruction or teachers’ capacity to enact instruction, as seen through the observation systems lens for understanding teaching quality. The quality of these interpretations, or the extent to which observation scores are composed of a signal that accurately reflects the interpretation, has important implications for the overall validity of uses of observation systems. Starting from an explicit conceptualization of instruction, this paper combines generalizability theory and hierarchical linear modelling approaches to decompose observation scores to explore the extent to which scores from observation systems are composed of signal, error, and bias across four different uses (i.e., teacher feedback, professional development, monitoring system equity, and employment decisions) of scores. We show that the quality of observation scores may depend more on what scores are interpreted as representing (i.e., the proposed use) than on the specific observation rubric being used. Further, we show that rater errors and biases are a major threat to any attempt to interpret observation scores as capturing the observation system’s understanding of teaching quality. We discuss implications for using scores from observation systems.","PeriodicalId":46725,"journal":{"name":"Educational Assessment Evaluation and Accountability","volume":"15 1","pages":""},"PeriodicalIF":3.9,"publicationDate":"2024-01-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139648079","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Key stakeholder voices: Investigating student perceptions of teachers’ use of assessment for learning 主要利益相关者的声音：调查学生对教师使用学习评价的看法

IF 3.9 3区教育学

Educational Assessment Evaluation and Accountability Pub Date : 2024-01-15 DOI: 10.1007/s11092-024-09428-7

Christel H. D. Wolterinck-Broekhuis, Cindy L. Poortman, Kim Schildkamp, Adrie J. Visscher

{"title":"Key stakeholder voices: Investigating student perceptions of teachers’ use of assessment for learning","authors":"Christel H. D. Wolterinck-Broekhuis, Cindy L. Poortman, Kim Schildkamp, Adrie J. Visscher","doi":"10.1007/s11092-024-09428-7","DOIUrl":"https://doi.org/10.1007/s11092-024-09428-7","url":null,"abstract":"Many schools aim to implement Assessment for Learning (AfL) to stimulate students to take more ownership of their learning and develop self-regulatory skills. This survey-based study is among the few in the field showing how students experience the extent of implementation of AfL, here in English language and mathematics classes in 12 Dutch secondary schools. Analysis showed no differences between the subjects. Students (N = 685) experienced activities to clarify the learning intentions and success criteria and eliciting evidence on their learning progress regularly (between 50–74% of the lessons). Students hardly ever experienced activities aimed at peer- and self-assessment (less than 25% of the lessons). However, cluster analysis revealed three distinct clusters related to the extent of AfL strategy use students experienced. Overall, we can conclude that AfL is not yet fully integrated into teaching practices. Teachers’ skills, knowledge and attitudes required to increase student engagement in AfL practices and strengthen students’ self-regulated learning need more attention in future teacher professional development trajectories.","PeriodicalId":46725,"journal":{"name":"Educational Assessment Evaluation and Accountability","volume":"15 1","pages":""},"PeriodicalIF":3.9,"publicationDate":"2024-01-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139469655","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Comparing the teaching quality of alternatively certified teachers and traditionally certified teachers: findings from a large-scale study 比较另类认证教师和传统认证教师的教学质量：一项大规模研究的结果

IF 3.9 3区教育学

Educational Assessment Evaluation and Accountability Pub Date : 2024-01-12 DOI: 10.1007/s11092-023-09426-1

Christin Lucksnat, Eric Richter, Sofie Henschel, Lars Hoffmann, Stefan Schipolowski, Dirk Richter

引用次数: 0

Re-contextualization of school quality assessment policies: an ethnography approach to SIMCE on Chilean disadvantaged schools 学校质量评估政策的再语境化：智利贫困学校 SIMCE 的人种学方法

IF 3.9 3区教育学

Educational Assessment Evaluation and Accountability Pub Date : 2023-12-28 DOI: 10.1007/s11092-023-09425-2

{"title":"Re-contextualization of school quality assessment policies: an ethnography approach to SIMCE on Chilean disadvantaged schools","authors":"","doi":"10.1007/s11092-023-09425-2","DOIUrl":"https://doi.org/10.1007/s11092-023-09425-2","url":null,"abstract":"<h3>Abstract</h3> Accountability policies through standardized testing are widespread in diverse educational systems. Based on an ethnographic research study, we sought to understand how a learning assessment policy, used for over three decades in primary and lower secondary education, is lived and interpreted in daily school life in Chile. This article analyzes how learning assessment policies are re-contextualized (translated and interpreted) in three elementary and high schools located in poor neighborhoods with different performance categorization according to students’ test results. We conducted interviews and observations with teachers and school leadership team members. Results show that school actors experience the effects of testing both as a menace and as a pressure despite their different school categorization performance. They consider the tests a decontextualized and unjust measurement and therefore respond with similar strategies that seek to avoid these negative consequences. This study contributes to understanding the effects of tight accountability systems that attempt to place different levels of pressure through categorization.","PeriodicalId":46725,"journal":{"name":"Educational Assessment Evaluation and Accountability","volume":"25 1","pages":""},"PeriodicalIF":3.9,"publicationDate":"2023-12-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139078118","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Teachers gaming the system: exploring opportunistic behaviours in a low-stakes accountability system 教师博弈制度:探索低风险问责制中的机会主义行为

IF 3.9 3区教育学

Educational Assessment Evaluation and Accountability Pub Date : 2023-12-06 DOI: 10.1007/s11092-023-09424-3

Gerard Ferrer-Esteban, Marcel Pagès

{"title":"Teachers gaming the system: exploring opportunistic behaviours in a low-stakes accountability system","authors":"Gerard Ferrer-Esteban, Marcel Pagès","doi":"10.1007/s11092-023-09424-3","DOIUrl":"https://doi.org/10.1007/s11092-023-09424-3","url":null,"abstract":"Based on the theoretical approaches of social capital and institutional trust, this paper seeks to identify contextual factors and conditions behind teacher behaviours which aim to alter the results of standardised tests in the Italian low-stakes accountability system. Numerous studies report significant factors associated with student cheating, but research into the factors of teacher-led opportunistic actions is scarce. Logistic regression models with fixed-effects at classroom level, with interaction terms, were carried out to identify factors increasing the likelihood of teacher misbehaviour. Models included approximately 79,100 primary, lower and upper secondary classrooms. Indicators of teacher cheating were estimated through algorithms based on suspicious answer strings from standardised tests. The results suggest that teacher cheating may be understood as a form of support for the most vulnerable students, since it is, to a greater extent, found helping low-income students, grade-retained students, as well as students in socially homogenous school settings. The findings also reveal that teacher cheating is consistently related to collectively share non-civic-minded behaviours and practices undertaken by teachers, which do not match legal requirements, such as within-school social segregation and exclusion of students from tests. Heterogeneous effects show that, even in classrooms with external controllers, the lower the civic capital in a school, the more misbehaviour are found. Relevant implications for research, social theory and policy are discussed.","PeriodicalId":46725,"journal":{"name":"Educational Assessment Evaluation and Accountability","volume":" 30","pages":""},"PeriodicalIF":3.9,"publicationDate":"2023-12-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138492587","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0