ETS Research Report Series最新文献_第8页

Examining the Accuracy of a Conversation-Based Assessment in Interpreting English Learners' Written Responses 考察基于会话的评估在解释英语学习者书面回答中的准确性

ETS Research Report Series Pub Date : 2021-03-05 DOI: 10.1002/ets2.12315

Alexis A. Lopez, Danielle Guzman-Orth, Diego Zapata-Rivera, Carolyn M. Forsyth, Christine Luce

{"title":"Examining the Accuracy of a Conversation-Based Assessment in Interpreting English Learners' Written Responses","authors":"Alexis A. Lopez, Danielle Guzman-Orth, Diego Zapata-Rivera, Carolyn M. Forsyth, Christine Luce","doi":"10.1002/ets2.12315","DOIUrl":"10.1002/ets2.12315","url":null,"abstract":"Substantial progress has been made toward applying technology enhanced conversation-based assessments (CBAs) to measure the English-language proficiency of English learners (ELs). CBAs are conversation-based systems that use conversations among computer-animated agents and a test taker. We expanded the design and capability of prior conversation-based instructional and assessment systems and developed a CBA designed to measure the English language skills and the mathematics knowledge of middle school ELs. The prototype CBA simulates an authentic and engaging mathematics classroom where the test taker interacts with two virtual agents to solve math problems. We embedded feedback and supports that are triggered by how the CBA interprets students' written responses. In this study, we administered the CBA to middle school ELs (N = 82) residing in the United States. We examined the extent to which the CBA system was able to consistently interpret the students' responses (722 responses for the 82 students). The study findings helped us to understand the factors that affect the accuracy of the CBA system's interpretations and shed light on how to improve CBA systems that incorporate scaffolding.","PeriodicalId":11972,"journal":{"name":"ETS Research Report Series","volume":"2021 1","pages":"1-15"},"PeriodicalIF":0.0,"publicationDate":"2021-03-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1002/ets2.12315","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45411329","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 5

EEG Correlates of Engagement During Assessment 评估过程中参与的脑电图相关性

ETS Research Report Series Pub Date : 2021-01-09 DOI: 10.1002/ets2.12312

Laura K. Halderman, Bridgid Finn, J.R. Lockwood, Nicole M. Long, Michael J. Kahana

{"title":"EEG Correlates of Engagement During Assessment","authors":"Laura K. Halderman, Bridgid Finn, J.R. Lockwood, Nicole M. Long, Michael J. Kahana","doi":"10.1002/ets2.12312","DOIUrl":"10.1002/ets2.12312","url":null,"abstract":"In educational assessment, low engagement is problematic when tests are low stakes for students but have significant consequences for teachers or schools. In the current study, we sought to establish the electroencephalographic (EEG) correlates of engagement and to distinguish engagement from mental effort. Forty university students participated in a simulated GRE® General Test session while scalp EEG was recorded from 128 channels. Participants completed two verbal and two quantitative GRE test blocks for a total of 40 items each, and after each item, rated either their engagement or mental effort on a scale of 1–6. We computed power for seven frequency bands (delta, theta, alpha, beta, and low, medium, and high gamma) across six regions of interest: left hemisphere (LH) and right hemisphere (RH) frontal, temporal, and parietal. Preliminary results suggested that gamma power (30–150 hertz [Hz]) indexed differences between high- and low-engagement ratings. This pattern was similar but weaker for mental effort. A cumulative logit model with cross-classified random effects determined that high gamma (90–150 Hz) over the LH temporal cortex predicted engagement ratings, while controlling for reaction time and accuracy. However, for effort ratings, reaction time was the sole significant predictor. These results suggest that high gamma may be a correlate of engagement during complex cognitive tasks, but not a correlate of effort. The findings are a promising step toward the goal of objectively measuring engagement during assessment tasks.","PeriodicalId":11972,"journal":{"name":"ETS Research Report Series","volume":"2021 1","pages":"1-17"},"PeriodicalIF":0.0,"publicationDate":"2021-01-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1002/ets2.12312","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45584417","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

Effect of Statistically Matching Equating Samples for Common-Item Equating 统计匹配等式样本对公共项目等式的影响

ETS Research Report Series Pub Date : 2021-01-09 DOI: 10.1002/ets2.12313

Ru Lu, Sooyeon Kim

引用次数: 2

The Development of a Content Assessment of Basic Electronics Knowledge 电子基础知识内容评估的开发

ETS Research Report Series Pub Date : 2020-12-01 DOI: 10.1002/ets2.12311

Jonathan Steinberg, Jessica Andrews-Todd, Carol M. Forsyth, J. Chamberlain, P. Horwitz, Al Koon, A. Rupp, Laura McCulla

引用次数: 1

Issue Information 问题信息

ETS Research Report Series Pub Date : 2020-12-01 DOI: 10.1002/ets2.12314

引用次数: 0