Korean Society for Educational Evaluation最新文献_第3页

Detection of and Correction for Violation of the Common Trend Assumption in Gain Score Analysis 增益分数分析中违反共同趋势假设的检测与纠正

Korean Society for Educational Evaluation Pub Date : 2022-12-31 DOI: 10.31158/jeev.2022.35.4.743

Yongnam Kim, Sangyun Lee, Naram Gwak

引用次数: 0

Causal Path Analysis of Teaching Competency based on the Structural Causal Model: Focusing on the Comparison across South Korea, England, and Finland using TALIS 2018 data 基于结构因果模型的教学胜任力因果路径分析——以韩国、英国和芬兰为例，基于TALIS 2018数据

Korean Society for Educational Evaluation Pub Date : 2022-12-31 DOI: 10.31158/jeev.2022.35.4.657

H. Yang, Sun-Geun Baek

{"title":"Causal Path Analysis of Teaching Competency based on the Structural Causal Model: Focusing on the Comparison across South Korea, England, and Finland using TALIS 2018 data","authors":"H. Yang, Sun-Geun Baek","doi":"10.31158/jeev.2022.35.4.657","DOIUrl":"https://doi.org/10.31158/jeev.2022.35.4.657","url":null,"abstract":"Based on the structural causal model, this study derived a causal graph that shows the causal relationship between the factors predicting the teaching competency of lower secondary school teachers in South Korea, the UK(England), and Finland. Also, it compared and analyzed the causal path to each country’s teaching competency. To this end, the data of lower secondary school teachers and principals, who participated in TALIS 2018, in Korea, the UK(England), and Finland were analyzed. First, the top 20 factors that predict teaching competency by each country were extracted by applying the mixed-effect random forest technique in consideration of the multi-layer structure of the data. Then, the causal graphs were derived by applying the causal discovery algorithm based on a structural causal model with the extracted predictors. As a result, there were common factors and discrimination factors in the top 20 predictors extracted from each national data, and the causal paths to teaching competency were compared and analyzed in the context of each country based on the causal graph by country. In addition, in the field of education, the possibility of using causal inference based on structural causal models was discussed, and the limitations and implications of this study were presented.","PeriodicalId":207460,"journal":{"name":"Korean Society for Educational Evaluation","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-12-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125875371","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Multidimensional IRT Scale Linking Under a Mixture of Bifactor Models for Mixed-Format Tests 混合格式测试双因子模型下的多维IRT量表链接

Korean Society for Educational Evaluation Pub Date : 2022-09-30 DOI: 10.31158/jeev.2022.35.3.521

S. Kim

{"title":"Multidimensional IRT Scale Linking Under a Mixture of Bifactor Models for Mixed-Format Tests","authors":"S. Kim","doi":"10.31158/jeev.2022.35.3.521","DOIUrl":"https://doi.org/10.31158/jeev.2022.35.3.521","url":null,"abstract":"Like unidimensional item response theory (IRT) models, bifactor models in multidimensional IRT have a scale indeterminacy problem, and due to this problem scale linking methods are needed to place all bifactor model parameter estimates from separate calibrations on a common ability scale. Four bifactor scale linking methods including the direct least squares (DLS), mean/least squares (MLS), item category response function (ICRF), and test response function (TRF) methods have been presented for use with single-format tests. Parallel to the 2006 paper of Kim and Lee, this paper extends the four scale linking methods to a mixture of bifactor models for mixed-format tests. Each linking method extended is intended to handle mixed-format tests using any mixture of the following bifactor extensions of four unidimensional IRT models: the bifactor three-parameter logistic, bifactor graded response, bifactor generalized partial credit, and bifactor nominal response models. For generality, symmetric criterion functions are proposed for the ICRF and TRF methods. Given two sets of parameter estimates for the common items linking two test forms, each linking method estimates the dilation (slope) and translation (intercept) coefficients of a linear transformation. Simulations are conducted to investigate the performance of the four linking methods. The results indicate that overall, the ICRF method performs very well, the MLS and DLS methods perform well (the MLS method is slightly better than the DLS method), and the TRF method performs poorly in estimating the linking coefficients. The inferiority of the TRF method is mainly due to its poor estimation of the translation coefficients.","PeriodicalId":207460,"journal":{"name":"Korean Society for Educational Evaluation","volume":"28 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134268581","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Exploring the Influential Factors of Teacher-student Relationship based on Random Forest Regression and Interpretation using SHAP 基于随机森林回归和SHAP解释的师生关系影响因素探讨

Korean Society for Educational Evaluation Pub Date : 2022-09-30 DOI: 10.31158/jeev.2022.35.3.409

Jae Hyun Kim, Junyeop Kim

{"title":"Exploring the Influential Factors of Teacher-student Relationship based on Random Forest Regression and Interpretation using SHAP","authors":"Jae Hyun Kim, Junyeop Kim","doi":"10.31158/jeev.2022.35.3.409","DOIUrl":"https://doi.org/10.31158/jeev.2022.35.3.409","url":null,"abstract":"The purpose of this study is to explore factors affecting the teacher-student relationship of middle and high school first grade students. It was intended to provide information for improving the teacher-student relationship through the analysis results. This study performed random forest regression analysis using GEPS data. The subjects of this study were 5,586 middle school students and 5,348 high school students. The explanatory variables used in the analysis were 478 items in middle school and 499 items in high school. 24 major factors were derived using SHAP value. The result is as follows: First, the main factors that influenced teacher-student relationships in the first grade of middle school were ‘school satisfaction’, ‘friendship relationship’, ‘teacher’s passion’, ‘teacher’s teaching ability’, ‘teacher’s teaching method’, ‘vacation life’, and ‘parent attachment-trust’. Second, the main factors that influenced teacher-student relationships in the first grade of high school were ‘school satisfaction’, ‘friendship relationship’, ‘teacher’s passion’, ‘teacher’s teaching method’, ‘self-esteem’, and ‘parent attachment-trust’. School satisfaction(8 items), peer relationship(6 items), teacher passion(1 item) and parent attachment-trust(1 item) were the same influencing factors, but the contribution of each question was different depending on the school level. Schools need to approach improving teacher- student relationships considering the results of the study.","PeriodicalId":207460,"journal":{"name":"Korean Society for Educational Evaluation","volume":"2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134015139","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

The Effects of Weighting on the Estimator of 3-Level Growth Model in the analysis of missing longitudinal data 纵向数据缺失分析中权重对三水平增长模型估计量的影响

Korean Society for Educational Evaluation Pub Date : 2022-09-30 DOI: 10.31158/jeev.2022.35.3.379

Seungwon Song, Sang-jin Kang

{"title":"The Effects of Weighting on the Estimator of 3-Level Growth Model in the analysis of missing longitudinal data","authors":"Seungwon Song, Sang-jin Kang","doi":"10.31158/jeev.2022.35.3.379","DOIUrl":"https://doi.org/10.31158/jeev.2022.35.3.379","url":null,"abstract":"This study seeks to clarify the effect of weighting on the fixed effect and random effect parameter estimators of the 3-level growth model. This study considered five weighting methods; ① non-weighting, ② sampling weighting, ③ longitudinal weighting, ④ multi-level weighting, ⑤ scaled multi-level weighting. The simulation study was conducted to statistically reveal the effect of weighting methods on the parameter estimation of the multi-level growth model. For the study, population, sampling data, and missing longitudinal data were each generated. The parameters of the 3-level growth model were estimated for each of the five weighting conditions using the missing longitudinal data. All conditions were repeated 100 times. The properties of the estimator were evaluated with bias, relative bias, and RMSE(root mean square error) from the viewpoint of bias and efficiency. The result is as follows. First, all fixed and random effects parameters were estimated inconsistently when non-weights or scaled multi-level weights were used in the 3-level growth model. Second, the 3-level growth model had the highest efficiency of fixed-effect and random-effect parameter estimators when non-weights or scaled multi-level weights were used. Based on the above results, suggestions for researchers and follow-up studies were presented.","PeriodicalId":207460,"journal":{"name":"Korean Society for Educational Evaluation","volume":"87 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128757069","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A Comparison of Treatment after Detecting insufficient effort respondings in Survey data through Simulation study 通过仿真研究调查数据中发现努力响应不足后的处理方法比较

Korean Society for Educational Evaluation Pub Date : 2022-09-30 DOI: 10.31158/jeev.2022.35.3.489

Wooyoul Na

{"title":"A Comparison of Treatment after Detecting insufficient effort respondings in Survey data through Simulation study","authors":"Wooyoul Na","doi":"10.31158/jeev.2022.35.3.489","DOIUrl":"https://doi.org/10.31158/jeev.2022.35.3.489","url":null,"abstract":"Since insufficient effort responding(IER) in survey data caused by a variety factors threaten the validity of the data and lead to inaccurate research results, it is necessary to detect and treat IER before conducting the main data analysis is a necessary in order to produce more reliable research results. Nevertheless, it has been insufficient to discuss about treatments how to deal with the data after detecting IER. Accordingly, this purpose of this study is to provide methodological implications about more feasible treatments after detecting IER by comparing complete data analysis, casewise- deletion, and multiple imputation which are applicable after detecting IER through simulation in the context of analyzing categorical confirmatory factor analysis. The main results of this study are as follows. First, It is showed that the casewise deletion tends to perform better than other treatments in terms of goodness-of-fit index and the accuracy of parameter estimation. Second, complete data analysis tends to perform poorly in the goodness-of-fit index and showed inaccurate results in estimating the relationship between constructs. Third, It is showed that multiple imputation tends to perform better than analyzing complete data in the model fit index, and it is indicated to be relatively accurate in estimating the relationship between constructs. However, It was inaccurate in terms of estimating factor loadings when multiple imputation was applied. Based on the results, It has been discussed about more efficient treatments how to deal with the survey data after detecting IER.","PeriodicalId":207460,"journal":{"name":"Korean Society for Educational Evaluation","volume":"26 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124274354","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Deep Learning Algorithm Exploration for Automated Korean essay Scoring 韩语作文自动评分的深度学习算法探索

Korean Society for Educational Evaluation Pub Date : 2022-09-30 DOI: 10.31158/jeev.2022.35.3.465

Kang Yun Park, Yong-Sang Lee

引用次数: 0

Structural Relationships of Elementary School Students’ Perception of Formative Feedback, Belief of Intelligence, Attitude Towards Assessment, Behavioral Intention, and Classroom Engagement 小学生形成反馈知觉、智力信念、评价态度、行为意向与课堂投入的结构关系

Korean Society for Educational Evaluation Pub Date : 2022-09-30 DOI: 10.31158/jeev.2022.35.3.439

Hyun-Woo Noh

{"title":"Structural Relationships of Elementary School Students’ Perception of Formative Feedback, Belief of Intelligence, Attitude Towards Assessment, Behavioral Intention, and Classroom Engagement","authors":"Hyun-Woo Noh","doi":"10.31158/jeev.2022.35.3.439","DOIUrl":"https://doi.org/10.31158/jeev.2022.35.3.439","url":null,"abstract":"The aims of this study was to explore the structural relationships among students’ attitudes toward assessment(SATA), the incremental belief of intelligence, the perceived teachers’ formative feedback, behavioral intention, and classroom engagement in either formative or summative assessment situations. In the study the multi-level SEM approach was used to identify the structural relationships among the SATA and its related variables using Mplus. The findings revealed that incremental belief of intelligence and formative feedback had a statistically significant positive effect on expectancy and value and an indirect effect on learners’ participation through value, positive affect, and behavioral intention. Specifically, formative feedback had a positive direct effect on behavioral intention and learners’ participation. The educational implications based on the results of this study are as follows. First, teachers’ formative feedback and students’ incremental beliefs of intelligence are variables with a positive influence on SATA. Second, students ascribe a greater value to and have better expectations from summative assessment rather than formative assessment. They consider summative assessment results to be more important. Although the mean of value and expectation in formative assessment was low, value showed the greatest positive effect on positive affect while expectation reduced negative affect in formative assessment. Therefore, to help improve students’ learning, the role of formative assessment is important. There is a need to emphasize that teachers should use student-centered assessment to consider students’ individual differences in learning and enhance students’ learning by providing formative feedback.","PeriodicalId":207460,"journal":{"name":"Korean Society for Educational Evaluation","volume":"89 4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123170805","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A Comparative Study on the Factors Affecting Middle School Students’ Participation in Private Supplementary Education in Mathematics between Korea and the USA using the TIMSS 2019 Data 基于TIMSS 2019数据的韩美中学生参与私立数学补习教育影响因素比较研究

Korean Society for Educational Evaluation Pub Date : 2022-06-30 DOI: 10.31158/jeev.2022.35.2.219

Sijung Cho

{"title":"A Comparative Study on the Factors Affecting Middle School Students’ Participation in Private Supplementary Education in Mathematics between Korea and the USA using the TIMSS 2019 Data","authors":"Sijung Cho","doi":"10.31158/jeev.2022.35.2.219","DOIUrl":"https://doi.org/10.31158/jeev.2022.35.2.219","url":null,"abstract":"The purpose of this study was to investigate the major factors affecting the participation of Korean and American middle school students in private supplementary education in mathematics using the TIMSS 2019 data. The random forest method was used to extract, compare, and analyze the key factors predicting student’s participation in private supplementary education in mathematics on the data of 3,658 Korean middle school students and 6,960 American middle school students who participated in the TIMSS 2019 survey. Academic achievement, personal characteristics, family background, and school characteristics were found to be important predictors in both countries, but the effect of each factor on the probability of participating in private supplementary education was different between the two countries. The results of this study reconfirmed that students in Korea tend to participate in private supplementary education with 'competitive motives' whereas the United States, students engage in private supplementary education with 'supplementary motives'. Results suggest that when devising policies against private supplementary tutoring, it is necessary to diagnose the state and causes in each individual country and devise customized policies for each country. \u0000","PeriodicalId":207460,"journal":{"name":"Korean Society for Educational Evaluation","volume":"232 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-06-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122374247","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Asymptotic Standard Errors of IRT Linking Coefficients for the Bifactor Extension of the 3PL Model 3PL模型双因子扩展的IRT连接系数的渐近标准误差

Korean Society for Educational Evaluation Pub Date : 2022-06-30 DOI: 10.31158/jeev.2022.35.2.327

S. Kim

{"title":"Asymptotic Standard Errors of IRT Linking Coefficients for the Bifactor Extension of the 3PL Model","authors":"S. Kim","doi":"10.31158/jeev.2022.35.2.327","DOIUrl":"https://doi.org/10.31158/jeev.2022.35.2.327","url":null,"abstract":"The bifactor extension of the three-parameter logistic (B3PL) model has been used in applications of multidimensional item response theory (IRT) such as test equating and vertical scaling. Developing a common multidimensional IRT scale (that is, multidimensional coordinate system) is critical in those applications. Three common-item scale linking methods, the direct least squares (DLS), mean/least squares (MLS), and item response function (IRF) methods, for the B3PL model have been found to be effective in developing a common multidimensional IRT ability scale between two test forms to be linked. In this paper, the asymptotic standard errors (SEs) of IRT linking coefficients estimated by the DLS, MLS, and IRF methods are derived assuming that the B3PL model holds and the asymptotic variance-covariance matrix of item parameter estimates from separate calibrations is available. The delta method is used for the derivations. Computer simulations which investigate the accuracy of the derivations under various conditions are given, showing that the derivations are reasonably accurate when sufficiently large samples are used and that in general the SEs of the IRF method are smaller than those of the DLS and MLS methods. The simulation results also suggest that the SEs of linking coefficient estimates are, approximately, inversely proportional to the square root of the sample size when two test forms are administered to the same number of examinees.","PeriodicalId":207460,"journal":{"name":"Korean Society for Educational Evaluation","volume":"27 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-06-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132571062","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0