{"title":"PISA 2018 Okuma Becerileri Testinde Yer Alan Maddelerde Değişen Madde Fonksiyonunun İncelenmesi","authors":"Şerife Zeybekoğlu, Ayşe Bilicioğlu Güneş, Evrim Yalçın","doi":"10.15390/eb.2023.12123","DOIUrl":null,"url":null,"abstract":"This study aimed to conduct Differential Item Functioning (DIF) determination studies using different methods on items in the Program for International Student Assessment (PISA) 2018 reading test in Turkey and compare the performance of the methods used. In the analyses, considering the individualized test design, item packages in the same test(s) in the core, first-stage, and second-stage sections were used. The second package (Core RC2), second package (Stage 1-R12H), and third package (Stage 2-R23H) were selected for the core, first, and second stages, respectively. Three partially scored items in this package were excluded from the analysis, and 33 common items with score of 1-0 were included in the analysis. This study included 147 Turkish students who responded to package of items. The variables of gender (ST004D01T), school location (SC001Q01TA) and index of economic, cultural and social status (ESCS) were drawn from student- and school-scale data and combined with the items of the cognitive test. Prior to data analysis, the dataset was organized, missing data and outliers were examined, and the assumptions of the theories were tested. Within the scope of the study, the Mantel-Haenszel (MH), Logistic Regression (LR), SIBTEST, and Raju’s Area Measures methods were employed for two categorical variables, and the Generalized MH, Generalized LR, and Generalized Lord’s χ2 methods were used for three categorical variables. According to gender variables, two, four, and three items were found to show DIF in the MH, SIBTEST, and LR methods, respectively, whereas 17 items were found to display DIF according to the unsigned area test, and seven items were found to display DIF according to the signed area test in Raju’s Area Measures. According to the ESCS variable, two and one items manifested DIF in MH and LR, respectively, while 15 items were found to manifest DIF according to the unsigned area test and eight items manifested DIF according to the signed area test in Raju’s Area Measures. None of the items showed DIF when using the SIBTEST method. According to the school location variable, one, two and 28 item were found to show DIF in Generalized MH, Generalized LR and Generalized Lord’s χ2 method, respectively. The results of the study indicate that although the Classical Test Theory (CTT) -based- and Item Response Theory (IRT)- based DIF methods are compatible, they differ in the level of DIF. IRT- based methods detect more DIF items than CTT- based methods. Additionally, similar results were obtained using the Generalized MH and LR methods.","PeriodicalId":47058,"journal":{"name":"Egitim Ve Bilim-Education and Science","volume":null,"pages":null},"PeriodicalIF":0.5000,"publicationDate":"2023-11-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Egitim Ve Bilim-Education and Science","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.15390/eb.2023.12123","RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q4","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}
引用次数: 0
Abstract
This study aimed to conduct Differential Item Functioning (DIF) determination studies using different methods on items in the Program for International Student Assessment (PISA) 2018 reading test in Turkey and compare the performance of the methods used. In the analyses, considering the individualized test design, item packages in the same test(s) in the core, first-stage, and second-stage sections were used. The second package (Core RC2), second package (Stage 1-R12H), and third package (Stage 2-R23H) were selected for the core, first, and second stages, respectively. Three partially scored items in this package were excluded from the analysis, and 33 common items with score of 1-0 were included in the analysis. This study included 147 Turkish students who responded to package of items. The variables of gender (ST004D01T), school location (SC001Q01TA) and index of economic, cultural and social status (ESCS) were drawn from student- and school-scale data and combined with the items of the cognitive test. Prior to data analysis, the dataset was organized, missing data and outliers were examined, and the assumptions of the theories were tested. Within the scope of the study, the Mantel-Haenszel (MH), Logistic Regression (LR), SIBTEST, and Raju’s Area Measures methods were employed for two categorical variables, and the Generalized MH, Generalized LR, and Generalized Lord’s χ2 methods were used for three categorical variables. According to gender variables, two, four, and three items were found to show DIF in the MH, SIBTEST, and LR methods, respectively, whereas 17 items were found to display DIF according to the unsigned area test, and seven items were found to display DIF according to the signed area test in Raju’s Area Measures. According to the ESCS variable, two and one items manifested DIF in MH and LR, respectively, while 15 items were found to manifest DIF according to the unsigned area test and eight items manifested DIF according to the signed area test in Raju’s Area Measures. None of the items showed DIF when using the SIBTEST method. According to the school location variable, one, two and 28 item were found to show DIF in Generalized MH, Generalized LR and Generalized Lord’s χ2 method, respectively. The results of the study indicate that although the Classical Test Theory (CTT) -based- and Item Response Theory (IRT)- based DIF methods are compatible, they differ in the level of DIF. IRT- based methods detect more DIF items than CTT- based methods. Additionally, similar results were obtained using the Generalized MH and LR methods.
期刊介绍:
Educación y Ciencia en el foco de publicación en línea con los objetivos; - Contribuyendo a la profesionalización de los docentes, - Un puente entre la teoría y la práctica, que proporciona una contribución directa a la práctica escolar y en el aula. - Ofreciendo una visión holística del ecosistema escolar, - Evaluar tanto los problemas educativos locales y actuales de Turquía se encuentra en sus propios estudios privados que examinan tanto la comparación internacional. Además, los siguientes estudios están cubiertos por la publicación de Educación y Ciencia. - Los estudios que examinan los problemas básicos y aplicaciones de la educación en preescolar, educación primaria y secundaria. - Esfuerzos para mejorar las competencias y estrategias educativas de profesores activos y otros profesionales - Todo tipo de estrategias y prácticas innovadoras de formación de docentes con el objetivo de hacer que los docentes candidatos sean docentes más calificados, examinar los modelos de formación docente y limitar la muestra al área de enseñanza relacionada - Metanálisis, análisis de contenido, análisis de perfil con grandes grupos de investigación o muestras - Todo tipo de actividades que pueden afectar el desarrollo individual, educativo, de carrera y social de los estudiantes en los niveles preescolar, elemental, intermedio y secundario. - Estudios que desarrollan y evalúan un nuevo modelo educativo, aunque teóricamente - Estudios que promuevan y apoyen la participación familiar en la educación. - Estudios multidisciplinares y estudios interdisciplinarios. - Estudios donde las herramientas de medición educativa son desarrolladas y probadas para su utilidad al mismo tiempo. - Estudios de integración tecnológica para ayudar a desarrollar aplicaciones educativas.