ETS Research Report Series最新文献_第3页

Methods for Imputing Scores When All Responses Are Missing for One or More Polytomous Items: Accuracy and Impact on Psychometric Property 当一个或多个多同构项目的所有回答都缺失时的计分方法:准确性和对心理测量属性的影响

ETS Research Report Series Pub Date : 2023-05-04 DOI: 10.1002/ets2.12369

Yanxuan Qu, Sandip Sinharay

引用次数: 0

Equity Levers: What Predicts Enrollment in and Number of College-Level Courses Taken in High School? 公平杠杆：什么可以预测高中的入学人数和大学水平课程的数量？

ETS Research Report Series Pub Date : 2023-04-10 DOI: 10.1002/ets2.12368

Marisol J. C. Kevelson, Catherine M. Millett, Carly Slutzky, Stephanie R. Saunders

{"title":"Equity Levers: What Predicts Enrollment in and Number of College-Level Courses Taken in High School?","authors":"Marisol J. C. Kevelson, Catherine M. Millett, Carly Slutzky, Stephanie R. Saunders","doi":"10.1002/ets2.12368","DOIUrl":"10.1002/ets2.12368","url":null,"abstract":"This study explores the extent to which student, family, peer, and school factors predict (a) whether students take Advanced Placement® (AP®) courses, International Baccalaureate (IB) courses, and dual enrollment courses and (b) in models limited to course takers, how many courses they completed. Our findings, based on a nationally representative, longitudinal sample, suggest that, when it comes to college-level high school course taking, the relative advantage of higher socioeconomic status (SES) is less for African American students than it is for White and Asian students. Ninth-grade math skills are the strongest predictor of AP or IB and dual enrollment course taking, above and beyond demographic background characteristics like SES and race or ethnicity. High school girls take AP/IB and dual enrollment courses at a higher rate than boys, and they take more of these courses. The level of academic focus of students and their peers is associated with both AP or IB and dual enrollment course taking, whereas having parents focused on college preparation and course taking only predicts AP or IB course taking. School factors associated with AP or IB course taking include U.S. region and rural location; the percentage of math teachers with a master's degree is also positively associated with the number of AP or IB courses students take. These findings highlight the importance of equitable educational opportunities starting from a young age. They also indicate a need for increased early attention to student math skills and for more supports for parents and school staff to enable them to encourage and prepare all students, especially those from historically marginalized groups, to take college-level courses in high school.","PeriodicalId":11972,"journal":{"name":"ETS Research Report Series","volume":"2023 1","pages":"1-61"},"PeriodicalIF":0.0,"publicationDate":"2023-04-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/ets2.12368","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46713183","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A Hybrid Model for Orthogonal Regression 正交回归的混合模型

ETS Research Report Series Pub Date : 2023-02-27 DOI: 10.1002/ets2.12367

Michael Kane

{"title":"A Hybrid Model for Orthogonal Regression","authors":"Michael Kane","doi":"10.1002/ets2.12367","DOIUrl":"10.1002/ets2.12367","url":null,"abstract":"Linear functional relationships are intended to be symmetric and therefore cannot generally be accurately estimated using ordinary least squares regression equations. Orthogonal regression (OR) models allow for errors in both Y and X and therefore can provide symmetric estimates of these relationships. The most well-established OR model, the errors-in-variables (EIV) model, assumes that the observed scatter around the line is due entirely to errors of measurement in Y and X and that the ratio of the error variances is known. If most of the variance around the line is known to be due to the errors of measurement in Y and X, the EIV model can provide an unbiased maximum likelihood estimate for a functional relationship. However, if a substantial part of the variability around the line is due to natural variability, which is not attributable to errors of measurement in Y or X, the ratio of the measurement error variances is not well defined and the EIV model is not directly applicable. The main contribution of this report is the development of a hybrid model that provides plausible estimates for linear functional relationships in cases with substantial natural variability and substantial errors of measurement. An analysis of female and male differential test functioning between an essay test and an objective test used as parts of a licensure examination provides an illustration of the use of the hybrid model.","PeriodicalId":11972,"journal":{"name":"ETS Research Report Series","volume":"2023 1","pages":"1-19"},"PeriodicalIF":0.0,"publicationDate":"2023-02-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/ets2.12367","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45600512","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Simulating Real-World Context in an Email Writing Task: Implications for Task-Based Language Assessment 在电子邮件写作任务中模拟真实世界环境：对基于任务的语言评估的启示

ETS Research Report Series Pub Date : 2023-02-16 DOI: 10.1002/ets2.12366

John M. Norris, Shoko Sasayama, Michelle Kim

{"title":"Simulating Real-World Context in an Email Writing Task: Implications for Task-Based Language Assessment","authors":"John M. Norris, Shoko Sasayama, Michelle Kim","doi":"10.1002/ets2.12366","DOIUrl":"10.1002/ets2.12366","url":null,"abstract":"Accomplishing a communication task in the real world requires the ability not only to do the task per se but also to manage aspects of the context in which it occurs. For this reason, simulations of target language use contexts have been incorporated into the design of communicative language tests as a way of enhancing the authenticity of assessment task performance. Although some contextual factors may increase extraneous cognitive load and distract learners from focusing on the task at hand (Sweller, 1994), they represent important design considerations in task-based language assessment (TBLA), where the purpose of assessment is to determine what second language (L2) learners can do with the target language in the real world. In that sense, the extraneous cognitive load might well be part of the construct we are interested in assessing. Accordingly, the current study simulated aspects of a real-world task performance context as part of an email writing task assessment. Simulated context was operationalized as (a) additional information about the task scenario, (b) a visual image to simulate the physical context, and (c) an audio to replicate the real-world experience. A total of 276 L2 English learners performed the email task, half with simulated context and the other half without it. Findings revealed that, when presented with simulated context, the tasks were perceived by participants to have induced more time pressure and to be more interesting. In terms of performance effects, the provision of simulated context negatively affected the syntactic complexity of participants' writing but positively affected their syntactic fluency. It also led to greater discrimination among learners at different proficiency levels on various measures of language performance. The paper concludes by highlighting implications for task design and validity evaluation, especially in TBLA.","PeriodicalId":11972,"journal":{"name":"ETS Research Report Series","volume":"2023 1","pages":"1-22"},"PeriodicalIF":0.0,"publicationDate":"2023-02-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/ets2.12366","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45714264","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Employer Expectations of 21st-Century High School Graduates: Analyzing Online Job Advertisements 21世纪高中毕业生的雇主期望:网络招聘广告分析

ETS Research Report Series Pub Date : 2023-01-28 DOI: 10.1002/ets2.12365

Kevin M. Williams, Tao Wang, Steven Holtzman, Tak Ming Leung, Gernissia Cherfrere, Guangming Ling

{"title":"Employer Expectations of 21st-Century High School Graduates: Analyzing Online Job Advertisements","authors":"Kevin M. Williams, Tao Wang, Steven Holtzman, Tak Ming Leung, Gernissia Cherfrere, Guangming Ling","doi":"10.1002/ets2.12365","DOIUrl":"10.1002/ets2.12365","url":null,"abstract":"Individuals with a high school education represent the largest subset of the U.S. workforce. However, little is known about the employer expectations of these individuals, particularly in the area of soft skills—also known as 21st-century skills. Online job advertisements offer useful data for examining these expectations, as they may supplement employer survey data and reflect actual recruitment practices. Our analysis of 68,505 online job advertisements suggests that employers hold generally lower expectations for the soft skills of high school–educated individuals than they do for postsecondary-educated individuals. However, employer expectations for two soft skills—professionalism and customer service skills—appear to be substantially higher for high school–educated individuals than for postsecondary-educated individuals. Additional results highlight similarities and differences within the high school–educated workforce across nine workplace industries. We discuss the implications of these results not only for high school–educated individuals and the organizations that employ them but also for practitioners and educators charged with assessing and providing training for these skills.","PeriodicalId":11972,"journal":{"name":"ETS Research Report Series","volume":"2023 1","pages":"1-19"},"PeriodicalIF":0.0,"publicationDate":"2023-01-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/ets2.12365","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46676646","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Evaluating the Use and Interpretation of the TOEIC® Listening and Reading Test Score Report: Perspectives of Test Takers in Japan 评估TOEIC®听力和阅读考试成绩报告的使用和解释：日本考生的观点

ETS Research Report Series Pub Date : 2023-01-23 DOI: 10.1002/ets2.12364

Ching-Ni Hsieh

{"title":"Evaluating the Use and Interpretation of the TOEIC® Listening and Reading Test Score Report: Perspectives of Test Takers in Japan","authors":"Ching-Ni Hsieh","doi":"10.1002/ets2.12364","DOIUrl":"10.1002/ets2.12364","url":null,"abstract":"Researchers suggest that claims about the meaningfulness of test score interpretations and consequences of test use should be backed by evidence that stakeholders understand the definition of the construct assessed (meaningfulness) and score reports (consequences). Evaluation of stakeholders' actual uses and interpretations of score reports in large-scale standardized language proficiency tests, however, remains limited in the score reporting literature. This study investigates how test takers, as an important stakeholder group, use and interpret the score report of the TOEIC® Listening and Reading (TOEIC L&R) test. Data were collected from 834 TOEIC L&R test takers in Japan, who represented a wide range of English language proficiency based on their TOEIC L&R total scores. The participants responded to an online survey that asked about their uses and interpretations of the test results and their comprehension of the performance feedback presented in the score report. The results showed that the participants used the test results largely as intended, providing an important piece of validity evidence to support the proposed uses of the TOEIC L&R test. Some of the score reporting elements such as the performance feedback and footnote message, however, were not easy to understand for all participants, revealing a need to improve the interpretability of the score report. The study findings have implications for designing informative score reports and the usefulness around reporting test performance feedback.","PeriodicalId":11972,"journal":{"name":"ETS Research Report Series","volume":"2023 1","pages":"1-16"},"PeriodicalIF":0.0,"publicationDate":"2023-01-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/ets2.12364","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44897565","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Evaluating Targeted Double Scoring for the Performance Assessment for School Leaders Using Imputation and Decision Theory 运用归因与决策理论评价学校领导绩效评估的目标双分

ETS Research Report Series Pub Date : 2023-01-19 DOI: 10.1002/ets2.12363

Jing Miao, Sandip Sinharay, Chris Kelbaugh, Yi Cao, Wei Wang

引用次数: 0

Investigating the Relationship Between Career and Technical Education High School Course-Taking and Early Job Outcomes 职业与技术教育高中课程选择与早期工作结果之间的关系研究

ETS Research Report Series Pub Date : 2022-12-08 DOI: 10.1002/ets2.12361

Margarita Olivera-Aguilar, Harrison J. Kell, Chelsea Ezzo, Steven B. Robbins

{"title":"Investigating the Relationship Between Career and Technical Education High School Course-Taking and Early Job Outcomes","authors":"Margarita Olivera-Aguilar, Harrison J. Kell, Chelsea Ezzo, Steven B. Robbins","doi":"10.1002/ets2.12361","DOIUrl":"10.1002/ets2.12361","url":null,"abstract":"This study examined how high school course-taking patterns (i.e., career and technical education [CTE] vs. academic vs. no concentration), personal characteristics embedded in a social cognitive theory framework (e.g., self-efficacy, academic expectations), and contextual variables (e.g., parental expectations, socioeconomic status [SES]) interact with each other in the prediction of students' income and job satisfaction 8 years after graduating from high school. Using a nationally representative data set (the Educational Longitudinal Study of 2002), we found significant differences by sex and course-taking pattern in the prediction of income: Among men, CTE concentrators had the highest income, whereas among women, academic concentrators reported the greatest earnings. We observed similar levels of job satisfaction among academic and CTE concentrators. We also found that SES significantly moderated the effect of English self-efficacy and academic expectations in the prediction of income and general effort in the prediction of job satisfaction. Our findings highlight how a social cognitive framework can be used to investigate the links between high school course-taking, personal and contextual factors, and job outcomes. They additionally suggest the need to consider a broader set of outcomes for evaluating the benefits of CTE participation.","PeriodicalId":11972,"journal":{"name":"ETS Research Report Series","volume":"2022 1","pages":"1-18"},"PeriodicalIF":0.0,"publicationDate":"2022-12-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/ets2.12361","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44351231","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Longitudinal Stability and Change of the Dark Triad: A Call for Research in Postsecondary, Occupational, and Community Settings 黑暗三合会的纵向稳定性和变化：呼吁在中学后、职业和社区环境中进行研究

ETS Research Report Series Pub Date : 2022-11-09 DOI: 10.1002/ets2.12362

Kevin M. Williams, Michelle P. Martin-Raugh, Jennifer E. Lentini

{"title":"Longitudinal Stability and Change of the Dark Triad: A Call for Research in Postsecondary, Occupational, and Community Settings","authors":"Kevin M. Williams, Michelle P. Martin-Raugh, Jennifer E. Lentini","doi":"10.1002/ets2.12362","DOIUrl":"10.1002/ets2.12362","url":null,"abstract":"Researchers, theorists, and practitioners have expressed a renewed interest in the longitudinal dynamics of personality characteristics in adulthood, including organic life span trajectories and their amenability to volitional change. However, this research has apparently not yet expanded to include the Dark Triad (psychopathy, narcissism, Machiavellianism), despite approximately 2 decades of research that has thoroughly examined other important issues related to construct validity and interpersonal behavior. We argue that researchers in postsecondary, occupational, and community-based settings are in a unique position to study the important phenomenon of Dark Triad malleability, as they are less hindered by obstacles in clinical and forensic contexts that have generated largely inconclusive results. In this article, we discuss several examples of methods for evaluating, quantifying, and interpreting Dark Triad malleability, examples of relevant extant training programs, possibilities for developing new programs, and factors that may moderate training efficacy, including Dark Triad levels themselves. Beyond addressing a fundamental question regarding the nature of these traits, the Dark Triad's destructive tendencies suggest that efforts to reduce them would provide myriad societal benefits and could propel Dark Triad research in an important new direction.","PeriodicalId":11972,"journal":{"name":"ETS Research Report Series","volume":"2022 1","pages":"1-22"},"PeriodicalIF":0.0,"publicationDate":"2022-11-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/ets2.12362","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43813598","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Mapping TOEFL® Essentials™ Test Scores to the Canadian Language Benchmarks 映射托福®基础™ 加拿大语言基准测试成绩

ETS Research Report Series Pub Date : 2022-10-22 DOI: 10.1002/ets2.12357

Spiros Papageorgiou, Larry Davis, Renka Ohta, Pablo Garcia Gomez

{"title":"Mapping TOEFL® Essentials™ Test Scores to the Canadian Language Benchmarks","authors":"Spiros Papageorgiou, Larry Davis, Renka Ohta, Pablo Garcia Gomez","doi":"10.1002/ets2.12357","DOIUrl":"10.1002/ets2.12357","url":null,"abstract":"In this research report, we describe a study to map the scores of the TOEFL® Essentials™ test to the Canadian Language Benchmarks (CLB). The TOEFL Essentials test is a four-skills assessment of foundational English language skills and communication abilities in academic and general (daily life) contexts. At the time of writing this report, the test was the most recent addition to the TOEFL® Family of Assessments. TOEFL Essentials test scores are intended to provide academic programs and other users with reliable information regarding the test taker's ability to understand and use English. Mapping of scores to widely used language frameworks such as the CLB provides additional support for interpreting test results and for making inferences regarding test-taker abilities. The score mapping process consisted of the following steps, as recommended in the literature: (a) establishing construct congruence between the test content and the performance descriptors of the CLB; (b) establishing recommended minimum test scores (cut scores) required to classify language learners into CLB levels, based on the judgments of local experts; and (c) providing evidence of procedural, internal, and external validation of the recommended cut scores.","PeriodicalId":11972,"journal":{"name":"ETS Research Report Series","volume":"2022 1","pages":"1-42"},"PeriodicalIF":0.0,"publicationDate":"2022-10-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/ets2.12357","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45056261","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0