Applied Measurement in Education最新文献_第8页

Reviewing the Test Reviews: Quality Judgments and Reviewer Agreements in the Mental Measurements Yearbook 复习测验复习：心理测量年鉴中的质量判断和复习者协议

IF 1.5 4区教育学

Applied Measurement in Education Pub Date : 2021-02-25 DOI: 10.1080/08957347.2021.1890742

T. Hogan, Marissa DeStefano, Caitlin Gilby, Dana C. Kosman, Joshua Peri

引用次数: 0

Think Alouds: Informing Scholarship and Broadening Partnerships through Assessment 大声思考:通过评估通知奖学金和扩大伙伴关系

IF 1.5 4区教育学

Applied Measurement in Education Pub Date : 2021-01-02 DOI: 10.1080/08957347.2020.1835914

J. Bostic

引用次数: 3

Formative Assessment of Computational Thinking: Cognitive and Metacognitive Processes 计算思维的形成性评估:认知和元认知过程

IF 1.5 4区教育学

Applied Measurement in Education Pub Date : 2021-01-02 DOI: 10.1080/08957347.2020.1835912

Sarah M. Bonner, Peggy P. Chen, Kristi Jones, Brandon Milonovich

引用次数: 6

Using Think-Alouds for Response Process Evidence of Teacher Attentiveness 用思考声作为教师注意力的反应过程证据

IF 1.5 4区教育学

Applied Measurement in Education Pub Date : 2021-01-02 DOI: 10.1080/08957347.2020.1835910

Ya Mo, Michele B. Carney, L. Cavey, Tatia Totorica

{"title":"Using Think-Alouds for Response Process Evidence of Teacher Attentiveness","authors":"Ya Mo, Michele B. Carney, L. Cavey, Tatia Totorica","doi":"10.1080/08957347.2020.1835910","DOIUrl":"https://doi.org/10.1080/08957347.2020.1835910","url":null,"abstract":"ABSTRACT There is a need for assessment items that assess complex constructs but can also be efficiently scored for evaluation of teacher education programs. In an effort to measure the construct of teacher attentiveness in an efficient and scalable manner, we are using exemplar responses elicited by constructed-response item prompts to develop selected-response assessment items. Through analyses of think-aloud interview data, this study examines the alignment between participant responses to, and scores arising from, the two item types. The interview protocol was administered to 12 mathematics teachers and teacher candidates who were first presented a constructed-response version of an item followed by the selected-response version of the same item stem. Our analyses focus on the alignment between responses and scores for eight item stems across the two item types and the identification of items in need of modification. The results have the potential to influence the way test developers generate and use response process evidence to support or refute the assumptions inherent in a particular score interpretation and use.","PeriodicalId":51609,"journal":{"name":"Applied Measurement in Education","volume":"34 1","pages":"10 - 26"},"PeriodicalIF":1.5,"publicationDate":"2021-01-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1080/08957347.2020.1835910","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42248907","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Gathering Response Process Data for a Problem-Solving Measure through Whole-Class Think Alouds 通过全班大声思考来收集解决问题的反应过程数据

IF 1.5 4区教育学

Applied Measurement in Education Pub Date : 2021-01-02 DOI: 10.1080/08957347.2020.1835913

J. Bostic, T. Sondergeld, G. Matney, G. Stone, Tiara Hicks

{"title":"Gathering Response Process Data for a Problem-Solving Measure through Whole-Class Think Alouds","authors":"J. Bostic, T. Sondergeld, G. Matney, G. Stone, Tiara Hicks","doi":"10.1080/08957347.2020.1835913","DOIUrl":"https://doi.org/10.1080/08957347.2020.1835913","url":null,"abstract":"ABSTRACT Response process validity evidence provides a window into a respondent’s cognitive processing. The purpose of this study is to describe a new data collection tool called a whole-class think aloud (WCTA). This work is performed as part of test development for a series of problem-solving measures to be used in elementary and middle grades. Data from third-grade students were collected in a 1–1 think-aloud setting and compared to data from similar students as part of WCTAs. Findings indicated that students performed similarly on the items when the two think-aloud settings were compared. Respondents also needed less encouragement to share ideas aloud during the WCTA compared to the 1–1 think aloud. They also communicated feeling more comfortable in the WCTA setting compared to the 1–1 think aloud. Drawing the findings together, WCTAs functioned as well if not better, than 1–1 think alouds for the purpose of contextualizing third-grade students’ cognitive processes. Future studies using WCTAs are recommended to explore their limitations and other factors that might impact their success as data gathering tools.","PeriodicalId":51609,"journal":{"name":"Applied Measurement in Education","volume":"34 1","pages":"46 - 60"},"PeriodicalIF":1.5,"publicationDate":"2021-01-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1080/08957347.2020.1835913","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47052381","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 3

Rethinking Think-Alouds: The Often-Problematic Collection of Response Process Data 重新思考大声思考:经常有问题的反应过程数据收集

IF 1.5 4区教育学

Applied Measurement in Education Pub Date : 2021-01-02 DOI: 10.1080/08957347.2020.1835911

Jacqueline P. Leighton

引用次数: 5

Asymptotic Standard Errors of Equating Coefficients Using the Characteristic Curve Methods for the Graded Response Model 梯度响应模型特征曲线法方程系数的渐近标准误差

IF 1.5 4区教育学

Applied Measurement in Education Pub Date : 2020-08-25 DOI: 10.1080/08957347.2020.1789142

Zhonghua Zhang

{"title":"Asymptotic Standard Errors of Equating Coefficients Using the Characteristic Curve Methods for the Graded Response Model","authors":"Zhonghua Zhang","doi":"10.1080/08957347.2020.1789142","DOIUrl":"https://doi.org/10.1080/08957347.2020.1789142","url":null,"abstract":"ABSTRACT The characteristic curve methods have been applied to estimate the equating coefficients in test equating under the graded response model (GRM). However, the approaches for obtaining the standard errors for the estimates of these coefficients have not been developed and examined. In this study, the delta method was applied to derive the mathematical formulas for computing the asymptotic standard errors for the parameter scale transformation coefficients and the true score equating coefficients that are estimated using the characteristic curve methods in test equating under the GRM in the context of the common-item nonequivalent groups equating design. Simulated and real data were further used to examine the accuracy of the derivations and compare the performance of the newly developed delta method with that of the multiple imputation method. The results indicated that the standard errors produced by the delta method were extremely close to the criterion empirical standard errors as well as those yielded by the multiple imputation method. The development of the standard error expressions by the delta method in the study has important practical implications.","PeriodicalId":51609,"journal":{"name":"Applied Measurement in Education","volume":"33 1","pages":"309 - 330"},"PeriodicalIF":1.5,"publicationDate":"2020-08-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1080/08957347.2020.1789142","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49604565","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 3

Can Culture Be a Salient Predictor of Test-Taking Engagement? An Analysis of Differential Noneffortful Responding on an International College-Level Assessment of Critical Thinking 文化是考试参与度的显著预测指标吗?一项国际大学水平批判性思维评估中不费力回答的差异分析

IF 1.5 4区教育学

Applied Measurement in Education Pub Date : 2020-07-29 DOI: 10.1080/08957347.2020.1789141

Joseph A. Rios, Hongwen Guo

{"title":"Can Culture Be a Salient Predictor of Test-Taking Engagement? An Analysis of Differential Noneffortful Responding on an International College-Level Assessment of Critical Thinking","authors":"Joseph A. Rios, Hongwen Guo","doi":"10.1080/08957347.2020.1789141","DOIUrl":"https://doi.org/10.1080/08957347.2020.1789141","url":null,"abstract":"ABSTRACT The objective of this study was to evaluate whether differential noneffortful responding (identified via response latencies) was present in four countries administered a low-stakes college-level critical thinking assessment. Results indicated significant differences (as large as .90 SD) between nearly all country pairings in the average number of noneffortful responses per test taker. Furthermore, noneffortful responding was found to be associated with a number of individual-level predictors, such as demographics (both gender and academic year), prior ability, and perceived difficulty of the test, though, these predictors were found to differ across countries. Ignoring the presence of noneffortful responses was associated with: (a) model fit deterioration as well as inflation of reliability, and (b) the inclusion of non-invariant items in the score linking anchor set. However, no meaningful differences in relative performance were noted once accounting for noneffortful responses. Implications for test development and improving the validity of score-based inferences from international assessments are discussed.","PeriodicalId":51609,"journal":{"name":"Applied Measurement in Education","volume":"33 1","pages":"263 - 279"},"PeriodicalIF":1.5,"publicationDate":"2020-07-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1080/08957347.2020.1789141","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"59806029","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 21

On the Reliable Identification and Effectiveness of Computer-Based, Pop-Up Glossaries in Large-Scale Assessments 论大规模评估中基于计算机的弹出式词汇表的可靠识别和有效性

IF 1.5 4区教育学

Applied Measurement in Education Pub Date : 2020-07-27 DOI: 10.1080/08957347.2020.1789137

D. Cohen, Alesha D. Ballman, F. Rijmen, Jon Cohen

{"title":"On the Reliable Identification and Effectiveness of Computer-Based, Pop-Up Glossaries in Large-Scale Assessments","authors":"D. Cohen, Alesha D. Ballman, F. Rijmen, Jon Cohen","doi":"10.1080/08957347.2020.1789137","DOIUrl":"https://doi.org/10.1080/08957347.2020.1789137","url":null,"abstract":"ABSTRACT Computer-based, pop-up glossaries are perhaps the most promising accommodation aimed at mitigating the influence of linguistic structure and cultural bias on the performance of English Learner (EL) students on statewide assessments. To date, there is no established procedure for identifying the words that require a glossary for EL students that is sufficiently reliable. In the coding procedure, we developed a method to reliably identify words and phrases that require a glossary. The method developed in the coding procedure was used to provide glossaries for the field-test items of statewide English language arts (ELA) and mathematics assessments across grades 3–11 (Current Study). In the Current Study, we assess the effectiveness and influence on construct validity of a pop-up glossary of the words identified in the coding procedure in a large scale, randomized controlled trial. The results demonstrated that generally the pop-up glossary accommodation was effective for both the ELA and mathematics assessments and did not harm the construct being measured.","PeriodicalId":51609,"journal":{"name":"Applied Measurement in Education","volume":"33 1","pages":"378 - 389"},"PeriodicalIF":1.5,"publicationDate":"2020-07-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1080/08957347.2020.1789137","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"59806226","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

Applying a Multiple Comparison Control to IRT Item-fit Testing 多重比较控制在IRT项目拟合测试中的应用

IF 1.5 4区教育学

Applied Measurement in Education Pub Date : 2020-07-23 DOI: 10.1080/08957347.2020.1789138

D. Sauder, Christine E. DeMars

引用次数: 1