Chinese/English journal of educational measurement and evaluation最新文献

Non-Parametric CD-CAT Item Selection Strategy and Termination Rules Based on Binary Search Algorithm 基于二元搜索算法的非参数 CD-CAT 项目选择策略和终止规则

Chinese/English journal of educational measurement and evaluation Pub Date : 2024-04-01 DOI: 10.59863/dkui7768

Junjie Li, Huijing Zheng

{"title":"Non-Parametric CD-CAT Item Selection Strategy and Termination Rules Based on Binary Search Algorithm","authors":"Junjie Li, Huijing Zheng","doi":"10.59863/dkui7768","DOIUrl":"https://doi.org/10.59863/dkui7768","url":null,"abstract":"CD-CAT plays a significant role in diagnosing and assessing students, contributing significantly to improving teaching effectiveness. However, in classroom teaching scenarios, unlike large-scale assessments where a large number of samples can be used to accurately estimate item parameters, non-parametric CD-CAT becomes the only feasible choice. Compared to parametric CD-CAT, non-parametric CD-CAT started later, and research mainly focuses on non-parametric item selection strategies. However, the existing non-parametric item selection strategies have the disadvantage of low efficiency, and there is little research on non-parametric termination rules. Therefore, this study proposes two more efficient item selection strategies: Non-Parametric Dynamic Binary Search (NDBS) and General Non-Parametric Dynamic Binary Search (GNDBS), as well as a non-parametric termination rule：Non-parametric Dynamic Binary Searching Index (NDBI). Simulation results show: (1) Under all conditions, the pattern classification accuracy rate of NDBS is higher than that of NPS, so NDBS can be used as the item selection strategy when there are no samples available. (2) In most cases, the performance of GNDBS is better than other item selection strategies, so GNDBS can be chosen as the item selection strategy when there are few samples available. (3) In variable-length tests, when the research objective is to obtain more accurate classification results, the critical value of the NDBI rule can be reduced; conversely, the critical values of the NDBI and GNDBI rules can be appropriately increased.","PeriodicalId":72586,"journal":{"name":"Chinese/English journal of educational measurement and evaluation","volume":"269 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140781312","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

基于二分搜索算法构建的非参数CD-CAT选题策略及终止规则基于二分搜索算法构建的非参数CD-CAT选题策略及终止规则

Chinese/English journal of educational measurement and evaluation Pub Date : 2024-04-01 DOI: 10.59863/yqmx8617

俊杰李, 慧婧郑

{"title":"基于二分搜索算法构建的非参数CD-CAT选题策略及终止规则","authors":"俊杰李, 慧婧郑","doi":"10.59863/yqmx8617","DOIUrl":"https://doi.org/10.59863/yqmx8617","url":null,"abstract":"CD-CAT 能对学生进行诊断和评估，对提升教学效果具有重大意义。但在课堂教学情景中，无法像大规模测评可以使用大量样本对项目参数进行准确估计，此时非参数 CD-CAT 成为唯一可行的选择。相比于参数 CD-CAT，非参数 CD-CAT 起步较晚，且非参数CD-CAT的研究主要集中在选题策略方面，但已有的非参数选题策略存在效率不高的缺点，且非参数终止规则则是鲜有研究。因此本研究提出了两种更高效的选题策略:非参数动态二分选题策略（NDBS）和一般非参数动态二分选题策略（GNDBS），以及一种非参数二分终止规则(NDBI)。模拟研究结果表明：（1）所有条件下，NDBS的模式判准率均高于NPS，因此，在完全没有样本时，可选用NDBS作为选题策略。（2）在大多数情况下，GNDBS的性能优于其他选题策略，因此，在具有少量样本时，可选用GNDBS作为选题策略。（3）在变长测验中，当研究目的是获得更准确的分类结果时，可以降低NDBI规则的临界值；相反，可以适当提高NDBI和GNDBI规则的临界值。","PeriodicalId":72586,"journal":{"name":"Chinese/English journal of educational measurement and evaluation","volume":"696 15","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140782552","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

ETS Skills Taxonomy ETS 技能分类标准

Chinese/English journal of educational measurement and evaluation Pub Date : 2023-12-01 DOI: 10.59863/nmie9603

Ou Lydia Liu, Harrison Kell, Kevin Williams, Guangming Ling, Micah Sanders

引用次数: 0

重访中国初高中教育改革 —— 基于 2012 年与 2022 年两次师生调查对比研究重访中国初高中教育改革 —— 基于 2012 年与 2022 年两次师生调查对比研究

Chinese/English journal of educational measurement and evaluation Pub Date : 2023-12-01 DOI: 10.59863/poen3762

一翔金, 雨田钟, 敏许

引用次数: 0

ETS 2025 技能分类法 ETS 2025 技能分类法

Chinese/English journal of educational measurement and evaluation Pub Date : 2023-12-01 DOI: 10.59863/urzl9477

Ou Lydia Liu, Harrison Kell, Kevin Williams, Guangming Ling, Micah Sanders

引用次数: 0

An Efficient Non-parametric Item Selection Method for Polytomous Scoring CD-CAT 用于 CD-CAT 多变量计分的高效非参数项目选择方法

Chinese/English journal of educational measurement and evaluation Pub Date : 2023-12-01 DOI: 10.59863/indp7038

Junjie Li, Jinghui Zheng, Chunhua Kang, Pingfei Zeng

{"title":"An Efficient Non-parametric Item Selection Method for Polytomous Scoring CD-CAT","authors":"Junjie Li, Jinghui Zheng, Chunhua Kang, Pingfei Zeng","doi":"10.59863/indp7038","DOIUrl":"https://doi.org/10.59863/indp7038","url":null,"abstract":"In educational evaluations at home and abroad, polytomous scoring items are becoming increasingly important. They can provide richer and more valuable information, with unmatched advantages compared to binary (0-1 scoring) items. If used as a tool for teachers to diagnose and assess students in the classroom, CD-CAT (Cognitive Diagnostic Computerized Adaptive Testing) has significant implications for improving teaching effectiveness. However, in classroom teaching situations, it is not feasible to estimate item parameters accurately with a large sample, as in large-scale assessments. In such cases, non-parametric CD-CAT becomes the only viable option. Compared to parametric CD-CAT, non-parametric CD-CAT started later and is particularly lacking in research related to polytomous scoring. Item selection method is at the core of CD-CAT, so it is essential to develop a non-parametric item selection method suitable for polytomous scoring CD-CAT. This study proposes a non-parametric item selection method for polytomous scoring cognitive diagnostic computerized adaptive testing (PCD-CAT): the Manhattan Distance Non-parametric Difference index item selection method (MD-NDI). The results of simulation studies indicate: (1) MD-NDI item selection method is suitable for PCD-CAT scenarios and exhibits better performance when the item bank quality is poor or the sample size for estimating item parameters is limited. (2) MD-NDI does not require pre-testing of items and distributes item usage more evenly, effectively ensuring the security of the item bank. (3) Even in cases of incorrectly specified item bank of Qc-matrix, MD-NDI still shows higher pattern correct classification rates. (4) In the study of variable-length PCD-CAT, MD-NDI not only reduces the test length in most conditions but also has a higher pattern correct classification rates when reaching the test termination rule.","PeriodicalId":72586,"journal":{"name":"Chinese/English journal of educational measurement and evaluation","volume":" 5","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138615954","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

一种高效的且适用于多级计分CD-CAT非参数选题方法一种高效的且适用于多级计分CD-CAT非参数选题方法

Chinese/English journal of educational measurement and evaluation Pub Date : 2023-12-01 DOI: 10.59863/ingv5575

俊杰李, 慧婧郑, 春花康, 平飞曾

引用次数: 0

Revisiting Secondary Education Reform in China: Comparing the Perceptions of Teachers and Students between 2012 and 2022 重新审视中国的中学教育改革：比较 2012 年和 2022 年师生的看法

Chinese/English journal of educational measurement and evaluation Pub Date : 2023-12-01 DOI: 10.59863/zuxn4915

Yixiang Jin, Yee Han Peter Joong, Rose Gibbs

引用次数: 0

ELion: An Intelligent Chinese Composition Tutoring System Based on Large Language Models ELion:基于大语言模型的智能作文辅导系统

Chinese/English journal of educational measurement and evaluation Pub Date : 2023-09-01 DOI: 10.59863/mpjo6480

Chanjin Zheng, Shaoyang Guo, Wei Xia, Shaoguang Mao

{"title":"ELion: An Intelligent Chinese Composition Tutoring System Based on Large Language Models","authors":"Chanjin Zheng, Shaoyang Guo, Wei Xia, Shaoguang Mao","doi":"10.59863/mpjo6480","DOIUrl":"https://doi.org/10.59863/mpjo6480","url":null,"abstract":"For a long time, Chinese language teachers in primary and secondary schools have been confronting challenges of heavy workload, low efficiency, and difficulty in improving the quality of composition evaluations. This article introduces “ELion”, an intelligent Chinese composition tutoring system based on large language models. The system utilizes deep linguistic features to evaluate the quality of compositions and provide interpretable feedback. By discussing the overall design, evaluation framework structure, and scoring algorithm principles of ELion, this paper addresses the theoretical, technical, and engineering issues of intelligent evaluation of Chinese compositions in the educational context. Small-scale experiments conducted in schools demonstrate that ELion performs well in language error detection, rhetorical techniques, and the expression of actions and emotions. It can basically meet the needs of Chinese language teaching in primary and secondary schools. In the future, ELion will further develop algorithms for ”instruction-learning-evaluation” alignment assessment, and personalized precise feedback generation, based on the GPT model. This will improve the evaluation effectiveness in topic analysis, text structure, and genuine emotional expression. Additionally, systematic field experiments for the system will be conducted to explore the application of artificial intelligence in education.","PeriodicalId":72586,"journal":{"name":"Chinese/English journal of educational measurement and evaluation","volume":"51 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"77127152","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Detecting Careless Cases in Practice Tests 检测练习测试中的粗心案例

Chinese/English journal of educational measurement and evaluation Pub Date : 2023-09-01 DOI: 10.59863/lavm1367

Steven Nydick

引用次数: 0