{"title":"在 PISA 评估的即时多阶段自适应测试中利用反应时间进行项目选择","authors":"Xiuxiu Tang, Yi Zheng, Tong Wu, K. Hau, H. Chang","doi":"10.1111/jedm.12403","DOIUrl":null,"url":null,"abstract":"Multistage adaptive testing (MST) has been recently adopted for international large‐scale assessments such as Programme for International Student Assessment (PISA). MST offers improved measurement efficiency over traditional nonadaptive tests and improved practical convenience over single‐item‐adaptive computerized adaptive testing (CAT). As a third alternative adaptive test design to MST and CAT, Zheng and Chang proposed the “on‐the‐fly multistage adaptive testing” (OMST), which combines the benefits of MST and CAT and offsets their limitations. In this study, we adopted the OMST design while also incorporating response time (RT) in item selection. Via simulations emulating the PISA 2018 reading test, including using the real item attributes and replicating PISA 2018 reading test's MST design, we compared the performance of our OMST designs against the simulated MST design in (1) measurement accuracy of test takers’ ability, (2) test time efficiency and consistency, and (3) expected gains in precision by design. We also investigated the performance of OMST in item bank usage and constraints management. Results show great potential for the proposed RT‐incorporated OMST designs to be used for PISA and potentially other international large‐scale assessments.","PeriodicalId":1,"journal":{"name":"Accounts of Chemical Research","volume":"7 8","pages":""},"PeriodicalIF":16.4000,"publicationDate":"2024-06-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Utilizing Response Time for Item Selection in On‐the‐Fly Multistage Adaptive Testing for PISA Assessment\",\"authors\":\"Xiuxiu Tang, Yi Zheng, Tong Wu, K. Hau, H. Chang\",\"doi\":\"10.1111/jedm.12403\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Multistage adaptive testing (MST) has been recently adopted for international large‐scale assessments such as Programme for International Student Assessment (PISA). MST offers improved measurement efficiency over traditional nonadaptive tests and improved practical convenience over single‐item‐adaptive computerized adaptive testing (CAT). As a third alternative adaptive test design to MST and CAT, Zheng and Chang proposed the “on‐the‐fly multistage adaptive testing” (OMST), which combines the benefits of MST and CAT and offsets their limitations. In this study, we adopted the OMST design while also incorporating response time (RT) in item selection. Via simulations emulating the PISA 2018 reading test, including using the real item attributes and replicating PISA 2018 reading test's MST design, we compared the performance of our OMST designs against the simulated MST design in (1) measurement accuracy of test takers’ ability, (2) test time efficiency and consistency, and (3) expected gains in precision by design. We also investigated the performance of OMST in item bank usage and constraints management. Results show great potential for the proposed RT‐incorporated OMST designs to be used for PISA and potentially other international large‐scale assessments.\",\"PeriodicalId\":1,\"journal\":{\"name\":\"Accounts of Chemical Research\",\"volume\":\"7 8\",\"pages\":\"\"},\"PeriodicalIF\":16.4000,\"publicationDate\":\"2024-06-06\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Accounts of Chemical Research\",\"FirstCategoryId\":\"102\",\"ListUrlMain\":\"https://doi.org/10.1111/jedm.12403\",\"RegionNum\":1,\"RegionCategory\":\"化学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"CHEMISTRY, MULTIDISCIPLINARY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Accounts of Chemical Research","FirstCategoryId":"102","ListUrlMain":"https://doi.org/10.1111/jedm.12403","RegionNum":1,"RegionCategory":"化学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"CHEMISTRY, MULTIDISCIPLINARY","Score":null,"Total":0}
Utilizing Response Time for Item Selection in On‐the‐Fly Multistage Adaptive Testing for PISA Assessment
Multistage adaptive testing (MST) has been recently adopted for international large‐scale assessments such as Programme for International Student Assessment (PISA). MST offers improved measurement efficiency over traditional nonadaptive tests and improved practical convenience over single‐item‐adaptive computerized adaptive testing (CAT). As a third alternative adaptive test design to MST and CAT, Zheng and Chang proposed the “on‐the‐fly multistage adaptive testing” (OMST), which combines the benefits of MST and CAT and offsets their limitations. In this study, we adopted the OMST design while also incorporating response time (RT) in item selection. Via simulations emulating the PISA 2018 reading test, including using the real item attributes and replicating PISA 2018 reading test's MST design, we compared the performance of our OMST designs against the simulated MST design in (1) measurement accuracy of test takers’ ability, (2) test time efficiency and consistency, and (3) expected gains in precision by design. We also investigated the performance of OMST in item bank usage and constraints management. Results show great potential for the proposed RT‐incorporated OMST designs to be used for PISA and potentially other international large‐scale assessments.
期刊介绍:
Accounts of Chemical Research presents short, concise and critical articles offering easy-to-read overviews of basic research and applications in all areas of chemistry and biochemistry. These short reviews focus on research from the author’s own laboratory and are designed to teach the reader about a research project. In addition, Accounts of Chemical Research publishes commentaries that give an informed opinion on a current research problem. Special Issues online are devoted to a single topic of unusual activity and significance.
Accounts of Chemical Research replaces the traditional article abstract with an article "Conspectus." These entries synopsize the research affording the reader a closer look at the content and significance of an article. Through this provision of a more detailed description of the article contents, the Conspectus enhances the article's discoverability by search engines and the exposure for the research.