International Journal of Testing最新文献_第5页

Is there bias in alternatives to standardized tests? An investigation into letters of recommendation 标准化考试的替代方案是否存在偏见?对推荐信的调查

IF 1.7

International Journal of Testing Pub Date : 2022-01-02 DOI: 10.1080/15305058.2021.2019751

Dev K. Dalal, Jason G. Randall, Ho Kwan Cheung, Brandon Gorman, Sylvia G. Roch, K. Williams

{"title":"Is there bias in alternatives to standardized tests? An investigation into letters of recommendation","authors":"Dev K. Dalal, Jason G. Randall, Ho Kwan Cheung, Brandon Gorman, Sylvia G. Roch, K. Williams","doi":"10.1080/15305058.2021.2019751","DOIUrl":"https://doi.org/10.1080/15305058.2021.2019751","url":null,"abstract":"Abstract Individuals concerned with subgroup differences on standardized tests suggest replacing these tests with holistic evaluations of unstructured application materials, such as letters of recommendation (LORs), which they posit show less bias. We empirically investigate this proposition that LORs are bias-free, and argue that LORs might actually invite systematic, race and gender subgroup differences in the content and evaluation of LORs. We text analyzed over 37,000 LORs submitted on behalf of over 10,000 graduate school applicants. Results showed that LOR content does differ across applicants. Furthermore, we see some systematic gender, race, and gender-race intersection differences in LOR content. Content of LORs also systematically differed between degree programs (S.T.E.M. vs. non-S.T.E.M.) and degree sought (doctoral vs. masters). Finally, LOR content alone did not predict an appreciable amount of variance in offers of admission (the first barrier to increasing diversity and inclusion in graduate programs). Our results, combined with past research on LOR content bias, highlight concerns that LORs can be biased against marginalized groups. We conclude with suggestions for reducing potential bias in LOR and for increasing diversity in graduate programs.","PeriodicalId":46615,"journal":{"name":"International Journal of Testing","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2022-01-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42116647","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

Introduction to International Journal of Testing special issue on equity and fairness in testing and assessment in school admissions 《国际考试杂志》关于招生考试和评估的公平性和公正性的特刊简介

IF 1.7

International Journal of Testing Pub Date : 2022-01-02 DOI: 10.1080/15305058.2021.2019753

S. E. Woo, B. Wille, S. Sireci

{"title":"Introduction to International Journal of Testing special issue on equity and fairness in testing and assessment in school admissions","authors":"S. E. Woo, B. Wille, S. Sireci","doi":"10.1080/15305058.2021.2019753","DOIUrl":"https://doi.org/10.1080/15305058.2021.2019753","url":null,"abstract":"Across the globe, educational tests are used for admissions decisions to competitive colleges, universities, high schools, and other programs. The high stakes associated with these tests have important consequences for students, and performance on them can determine whether students reach their academic and career aspirations. For this reason, their use is both widespread and contentious. Recently, the debates over the use of standardized tests in college and graduate admissions has increased, due in large part to concerns about score disparities resulting in disparate admissions outcomes. The International Journal of Testing has published many examples of criticisms and research with respect to admissions testing around the world, including in Chile (Ramirez et al., 2020), Israel (Rapp & Allalouf, 2003), Saudi Arabia (Tsaousis et al., 2018), Sweden (Wiberg & von Davier, 2017), and the United States (e.g., TalentoMiller, 2008). In the U.S. and Chile, public outcry against disparate outcomes for certain groups of students have marshaled in changes in admissions testing programs and the policies associated with them (Koljatic et al., 2021). In the U.S., several colleges and universities have suspended the SAT and ACT requirements for their applicants, which generated a number of heated discussions both within and outside academia. The use of the Graduate Record Examinations (GREs) in graduate admissions is also being hotly debated for similar reasons, and a number of graduate programs in the U.S. have opted to remove the GRE requirement from https://doi.org/10.1080/15305058.2021.2019753","PeriodicalId":46615,"journal":{"name":"International Journal of Testing","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2022-01-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48665709","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Using third-party evaluations to assess socioemotional skills in graduate and professional school admissions 使用第三方评估来评估研究生和专业学校招生中的社会情感技能

IF 1.7

International Journal of Testing Pub Date : 2022-01-02 DOI: 10.1080/15305058.2021.2019748

David Klieger, Jennifer L. Bochenek, Chelsea Ezzo, Steven Holtzman, Frederick Cline, Margarita Olivera-Aguilar

{"title":"Using third-party evaluations to assess socioemotional skills in graduate and professional school admissions","authors":"David Klieger, Jennifer L. Bochenek, Chelsea Ezzo, Steven Holtzman, Frederick Cline, Margarita Olivera-Aguilar","doi":"10.1080/15305058.2021.2019748","DOIUrl":"https://doi.org/10.1080/15305058.2021.2019748","url":null,"abstract":"Abstract Consideration of socioemotional skills in admissions potentially can increase representation of racial and ethnic minorities and women in graduate and professional education as well as identify candidates more likely to succeed in graduate and professional school. Research on one such assessment, the ETS Personal Potential Index (PPI), showed that the PPI produced much smaller racial/ethnic-gender group mean score differences than undergraduate grade point average (UGPA) and the Graduate Record Examinations (GRE) did. Across levels of institutional selectivity, the PPI can promote racial/ethnic and gender diversity in graduate and professional school in ways that UGPA and GRE scores do not. Predictive validity analyses showed that for doctoral STEM programs the PPI dimensions of (1) Planning and Organization and (2) Communication Skills positively predict school grade point average as well as a lower risk of academic probation, a determinant of degree progress, both alone and incrementally over UGPA and GRE scores. Supplemental data for this article is available online at https://doi.org/10.1080/15305058.2021.2019748 .","PeriodicalId":46615,"journal":{"name":"International Journal of Testing","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2022-01-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43931141","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Test efficacy: Refocusing validation from college exams to candidates 考试效能:将验证的焦点从大学考试转移到考生身上

IF 1.7

International Journal of Testing Pub Date : 2022-01-02 DOI: 10.1080/15305058.2021.2019752

Alvaro J. Arce, M. J. Young

引用次数: 1

Using personal statements in college admissions: An investigation of gender bias and the effects of increased structure 在大学录取中使用个人陈述:性别偏见和增加结构影响的调查

IF 1.7

International Journal of Testing Pub Date : 2021-12-15 DOI: 10.1080/15305058.2021.2019749

Susan Niessen, Marvin Neumann

{"title":"Using personal statements in college admissions: An investigation of gender bias and the effects of increased structure","authors":"Susan Niessen, Marvin Neumann","doi":"10.1080/15305058.2021.2019749","DOIUrl":"https://doi.org/10.1080/15305058.2021.2019749","url":null,"abstract":"Abstract Personal statements are among the most commonly used instruments in college admissions procedures. Yet, little research on their reliability, validity, and fairness exists. The first aim of this paper was to investigate hypotheses about adverse impact and underprediction for female applicants, which could result from lower tendencies to use agentic language compared to male applicants. Second, we examined if rating personal statements in a more structured manner would increase reliability and validity. Using personal statements (250 words) from a large cohort of applicants to an undergraduate psychology program at a Dutch University, we found no evidence for adverse impact for female applicants or more agentic language use by male applicants, and no relationship between agentic language use and personal statement ratings. In contrast, we found that personal statements of female applicants were rated slightly more positively than those of males. Exploratory analyses suggest that female applicants’ better writing skills might explain this difference. A more structured approach to rating personal statements yielded higher, but still only ‘moderate’ inter-rater reliability, and virtually identical, negligible predictive validity for first year GPA and dropout.","PeriodicalId":46615,"journal":{"name":"International Journal of Testing","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2021-12-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44509062","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

Metacognitive skills inventory (MSI): development and validation 元认知技能量表(MSI):开发与验证

IF 1.7

International Journal of Testing Pub Date : 2021-10-02 DOI: 10.1080/15305058.2021.1986051

Haja Hameed, Reena Cheruvalath

{"title":"Metacognitive skills inventory (MSI): development and validation","authors":"Haja Hameed, Reena Cheruvalath","doi":"10.1080/15305058.2021.1986051","DOIUrl":"https://doi.org/10.1080/15305058.2021.1986051","url":null,"abstract":"Abstract Metacognitive skills help to control and regulate negative thoughts, emotions, beliefs and sad memories. The objective of the study was to develop and validate an inventory-Metacognitive Skills Inventory (MSI) to assess the variance in adopting metacognitive strategies between those who have depressive symptoms and those who have not. Two studies were carried out among Indian youth (study 1—N = 269, MeanAge= 21.1 and study 2—N = 745, MeanAge= 20.9). They completed the MSI as well as measures of depression and negative emotions. Item response theory (IRT) analysis, and exploratory (EFA) and confirmatory factor analysis (CFA) were carried out for the scale development. The analyses derived a meaningful four-factor structure [(i) Navigation of negative thoughts by adopting metacognitive strategies, (ii) Channelizing negative emotions constructively, (iii) Recognizing ruminative tendencies, (iv) Knowledge of strengths and weaknesses in regulating emotions] of a 12-item MSI. An MSI could be used to identify patient-specific metacognitive skills in people with depressive symptoms, which need to be improved while doing Metacognitive Therapy (MCT) after validating clinical samples.","PeriodicalId":46615,"journal":{"name":"International Journal of Testing","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2021-10-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44737070","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Cross-country comparability of a social-emotional skills assessment designed for youth in low-resource environments 为低资源环境中的青年设计的社会情感技能评估的跨国可比性

IF 1.7

International Journal of Testing Pub Date : 2021-10-02 DOI: 10.1080/15305058.2021.1995867

Nina Menezes Cunha, Andres Martinez, P. Kyllonen, Sarah Gates

引用次数: 1

Examining severity and centrality effects in TestDaF writing and speaking assessments: An extended Bayesian many-facet Rasch analysis 检验TestDaF写作和口语评估中的严重性和中心性效应：一种扩展的贝叶斯多方面Rasch分析

IF 1.7

International Journal of Testing Pub Date : 2021-09-03 DOI: 10.1080/15305058.2021.1963260

T. Eckes, K. Jin

引用次数: 6

Exploring task features that predict psychometric quality of test items: the case for the Dutch driving theory exam 探索预测测试项目心理测量质量的任务特征：以荷兰驾驶理论考试为例

IF 1.7

International Journal of Testing Pub Date : 2021-06-15 DOI: 10.1080/15305058.2021.1916506

E. Roelofs, Wilco H M Emons, Angela J. Verschoor

{"title":"Exploring task features that predict psychometric quality of test items: the case for the Dutch driving theory exam","authors":"E. Roelofs, Wilco H M Emons, Angela J. Verschoor","doi":"10.1080/15305058.2021.1916506","DOIUrl":"https://doi.org/10.1080/15305058.2021.1916506","url":null,"abstract":"Abstract This study reports on an Evidence Centered Design (ECD) project in the Netherlands, involving the theory exam for prospective car drivers. In particular, we illustrate how cognitive load theory, task-analysis, response process models, and explanatory item-response theory can be used to systematically develop and refine task models. Based on a cognitive model for driving, 353 existing items involving rules of priority at intersections, were coded on intrinsic task features and task presentation features. Hierarchical regression analyses were carried out to determine the contribution of task features to item difficulty and item discrimination. A substantial proportion of variance in both item difficulty and item discrimination parameters could be explained by intrinsic task-features, including rules and signs (25%, 18.6%), task-intersection features (13.4%, 14.1%), and a smaller small proportion to item presentation features (3.5%, 7.1%) of the total variance. It is concluded that the systematic approach of discerning task features and determining the impact on item parameters has added value as an ECD-tool for evaluating existing assessments that are planned to be innovated. The paper concludes with a discussion of practical implications.","PeriodicalId":46615,"journal":{"name":"International Journal of Testing","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2021-06-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1080/15305058.2021.1916506","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47064095","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Validating theoretical assumptions about reading with cognitive diagnosis models 用认知诊断模型验证阅读的理论假设

IF 1.7

International Journal of Testing Pub Date : 2021-06-15 DOI: 10.1080/15305058.2021.1931238

A. George, A. Robitzsch

引用次数: 8