Educational and Psychological Measurement最新文献

{"title":"Using Explanatory Item Response Theory to Study Rating-Scale Design Effects on Response Style Discrimination.","authors":"Munevver Ilgun Dibek, Daniel Bolt","doi":"10.1177/00131644261466688","DOIUrl":"10.1177/00131644261466688","url":null,"abstract":"<p><p>This paper develops an explanatory Item Response Theory (IRT) methodology that supports the study of rating-scale design features on item sensitivity to response style. Using item response data from a previous two-part experimental study that systematically manipulated various rating scale features (e.g., the number of rating scale categories, rating scale labeling, and polarity of rating scales), a multidimensional nominal response model (MNRM) methodology is applied in which response style is represented by a latent trait that distinguishes respondents having preferences toward extreme versus middle response categories. Item discrimination parameters on the response style dimension are treated as outcomes within an explanatory IRT framework where the manipulated design features serve as predictors. Results from our analyses suggest that the number of rating scale categories was generally associated with stronger response-style discrimination, while wave, labeling and polarity conditions appeared less influential. Importantly, residual variability in response style discrimination at both the within- and between-item levels was seen in both studies to be quite high, making even statistically significant design features appear relatively weak in their effects. In both studies, the measured construct is seen to play a significant role in explaining some of that between-item heterogeneity. The results point to the feasibility of using the MNRM within an explanatory IRT framework as a basis for exploring item-level sensitivities to response style. The framework also uniquely allows for evaluating design effects against residual within-item and between-item sources of heterogeneity in response style item discrimination. The presence of substantial within- and between-item residual heterogeneity in the explanatory IRT analysis helps explain the frequent inconsistencies seen across regarding the effects of design characteristics.</p>","PeriodicalId":11502,"journal":{"name":"Educational and Psychological Measurement","volume":" ","pages":"00131644261466688"},"PeriodicalIF":4.4,"publicationDate":"2026-07-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13420125/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148629763","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Factor Retention in Ordinal Data Under Local Item Dependence: A Monte Carlo Comparison of Correlation Estimators and Retention Methods.","authors":"Abdullah Faruk Kılıç","doi":"10.1177/00131644261465917","DOIUrl":"10.1177/00131644261465917","url":null,"abstract":"<p><p>Local item dependence (LID) is a persistent threat to dimensionality assessment in ordinal data, yet its effects on factor retention methods remain incompletely understood. Five factor retention methods (minimum average partial [MAP], parallel analysis [PA], Exploratory Graph Analysis [EGA]-Walk, EGA-Leiden, EGA-TMFG) combined with six correlation matrix estimators (Pearson, polychoric, and four fuzzy-based variants) were evaluated in a Monte Carlo simulation spanning 900 conditions and 450,000 datasets. First, pairwise and testlet LID structures produced asymmetric effects on factor retention: pair-LID induced systematic overfactoring in GLASSO-based EGA methods and PA, while testlet-LID produced near-ceiling MAP accuracy (.99) through mechanical error compensation rather than correct structural identification. Second, correlation matrix choice had negligible influence on retention performance. All four fuzzy variants were statistically indistinguishable from Pearson across all conditions, while polychoric showed a condition-dependent profile, with modest advantages under local independence but poorer performance under severe testlet-LID. Third, threshold distribution shape exerted no meaningful influence on retention outcomes, either as a main effect or in interaction with LID condition. EGA-TMFG showed the strongest resilience and is recommended when local dependence is plausible. LID structure and factor loading strength were the primary determinants of retention accuracy; correlation matrix preprocessing provided no practical corrective benefit.</p>","PeriodicalId":11502,"journal":{"name":"Educational and Psychological Measurement","volume":" ","pages":"00131644261465917"},"PeriodicalIF":4.4,"publicationDate":"2026-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13380647/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148520189","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Consensus Among Differential Item Functioning Effect Size Measures: A Simplified Approach to Reporting Effect Size.","authors":"Austin Wyman, Zhiyong Zhang","doi":"10.1177/00131644261458305","DOIUrl":"10.1177/00131644261458305","url":null,"abstract":"<p><p>Differential item functioning (DIF) is an issue of a measure that can lead to differences in item scores between subpopulations (e.g., sex, race, age) while controlling for a latent trait or ability level. It is important to report DIF effect size measures to understand the practical significance of DIF results as well. We reviewed 15 different DIF effect size measures in the literature, and no previous study had examined whether the 15 DIF effect size measures agreed with one another, which has implications in the number and type of measures to report. The present study investigated the relationship among DIF effect size measures in 1PL, 2PL, and GRM models and provide clear recommendations to applied researchers for their reporting. Our simulation results showed that, for each model, three effect size measures can be reported to obtain the most unique information about uniform DIF and the least bias from sampling conditions (e.g., sample size and impact): Mantel-Haenszel Delta, McFadden's Pseudo R-squared, and a signed DIF measure. However, the optimal signed DIF measure varies by model, with Signed Area under the curve for the 1PL model, Standardized P-difference for the 2PL, and Signed Item Difference in the Sample/Normal Distribution for the GRM. The recommended approach to reporting DIF effect size improves the detection of DIF through practical significance and provides researchers with more tools to address bias.</p>","PeriodicalId":11502,"journal":{"name":"Educational and Psychological Measurement","volume":" ","pages":"00131644261458305"},"PeriodicalIF":4.4,"publicationDate":"2026-07-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13375857/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148577481","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Multimodal Test Item Parameter Prediction From Text, Images, and Metadata: Fusing Together AI Vision and Language Models.","authors":"Hotaka Maeda, Yikai Ek Lu","doi":"10.1177/00131644261460779","DOIUrl":"10.1177/00131644261460779","url":null,"abstract":"<p><p>We propose a flexible multimodal model for predicting all dichotomous and polytomous item parameters from text, images, and metadata by fusing representations from encoder Transformer vision and language models. This deep learning model accommodates heterogeneous item formats, including items with any number of components such as correct and incorrect options, stimuli, and images. Answer key indicators distinguish correct options from distractors, while an attention pooling technique weights the relative importance of these components. Based on the two-parameter logistic model and the generalized partial credit model, we predict all item parameters jointly using a masking strategy to ensure that only relevant parameters contribute to the loss. Item-level and item component-level metadata are also included. We evaluate the approach using 40,965 English language arts and mathematics items for Grades 3-11. A single model accommodated both exam subjects, all 11 item types, and all item parameters, eliminating the need for multiple specialized models. However, results indicate that the full model was unable to leverage all input data. Often, prediction accuracy was unchanged as features were removed. Images were good predictors on their own (item intercept <math> <mrow> <msup><mrow><mi>R</mi></mrow> <mrow><mn>2</mn></mrow> </msup> <mo>=</mo> <mo>.</mo> <mn>25</mn></mrow> </math> ) but did not consistently contribute unique information when combined with text and metadata. Of the strongest-performing models, the most parsimonious variant achieved <math> <mrow> <msup><mrow><mi>R</mi></mrow> <mrow><mn>2</mn></mrow> </msup> </mrow> </math> values of .67, .45, .39, .34, .66, .33, and .74 for item intercept, discrimination, difficulty, and four polytomous step threshold parameters, respectively. Findings suggest that current training methods may limit the learnability of complex multimodal deep fusion models. Example Python code from this study is available on Github: https://github.com/hotakamaeda/multimodal_prediction.</p>","PeriodicalId":11502,"journal":{"name":"Educational and Psychological Measurement","volume":" ","pages":"00131644261460779"},"PeriodicalIF":4.4,"publicationDate":"2026-07-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13364774/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148444855","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A Changepoint Rule for Determining Bicluster Retention Cutoffs in Cheating Detection.","authors":"Hyeryung Lee","doi":"10.1177/00131644261459591","DOIUrl":"10.1177/00131644261459591","url":null,"abstract":"<p><p>In test cheating detection, biclustering can be used to identify localized groups of examinees who share unusual response patterns, but an unresolved practical issue is determining how many extracted biclusters should be retained and how many examinees should be flagged. Retaining too many biclusters may increase false positives by capturing weak or noise-driven patterns, whereas retaining too few may miss meaningful cheating structures. This study proposes a changepoint-based retention rule that uses the ordered sequence of bicluster <i>p</i> values to identify where strong cheating-related evidence begins to give way to weaker residual patterns, thereby providing a data-driven cutoff for bicluster retention. The method was evaluated using two operational test forms with known cheating labels and a simulation study that varied cheating type, test length, and the proportion of compromised items. Label-based benchmark cutoffs were defined using the F1-score, balanced accuracy, and Youden's index, with the F1-score treated as the primary benchmark. In the empirical analysis, the estimated changepoints closely aligned with the F1-based benchmark, yielding the same sensitivity and slightly lower specificity across both forms. In the simulation study, the estimated changepoint generally approximated the F1-based benchmark but tended to select more conservative retention cutoffs, resulting in lower sensitivity and small specificity differences across conditions. These findings suggest that the proposed rule can provide a useful basis for determining how many biclusters to flag for further review in operational cheating-detection settings.</p>","PeriodicalId":11502,"journal":{"name":"Educational and Psychological Measurement","volume":" ","pages":"00131644261459591"},"PeriodicalIF":4.4,"publicationDate":"2026-07-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13341680/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148410801","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Reliability of Difference Scores Obtained From Nested Data Within a Multivariate Generalizability Theory Framework.","authors":"Rabia Karatoprak Ersen, Won-Chan Lee, Donald B Yarbrough","doi":"10.1177/00131644261451746","DOIUrl":"10.1177/00131644261451746","url":null,"abstract":"&lt;p&gt;&lt;p&gt;The purpose of this study is to examine the reliability and dependability of difference scores computed as the change between a pretest and a posttest administered to assess the effectiveness of an intervention. The data-collection design involved a nested structure, with persons (&lt;i&gt;p&lt;/i&gt;) nested within groups (&lt;i&gt;g&lt;/i&gt;), and groups nested within sites (&lt;i&gt;s&lt;/i&gt;). Multivariate generalizability theory was employed to estimate the reliability and dependability of difference scores at the levels of persons, groups, and sites. The &lt;i&gt;G&lt;/i&gt; study designs included &lt;math&gt; &lt;mrow&gt;&lt;mrow&gt;&lt;mo&gt;(&lt;/mo&gt; &lt;msup&gt;&lt;mrow&gt;&lt;mi&gt;p&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mo&gt;•&lt;/mo&gt;&lt;/mrow&gt; &lt;/msup&gt; &lt;mo&gt;:&lt;/mo&gt; &lt;msup&gt;&lt;mrow&gt;&lt;mi&gt;g&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mo&gt;•&lt;/mo&gt;&lt;/mrow&gt; &lt;/msup&gt; &lt;mo&gt;:&lt;/mo&gt; &lt;msup&gt;&lt;mrow&gt;&lt;mi&gt;s&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mo&gt;•&lt;/mo&gt;&lt;/mrow&gt; &lt;/msup&gt; &lt;mo&gt;)&lt;/mo&gt;&lt;/mrow&gt; &lt;mspace&gt;&lt;/mspace&gt; &lt;mi&gt;x&lt;/mi&gt; &lt;mspace&gt;&lt;/mspace&gt; &lt;msup&gt;&lt;mrow&gt;&lt;mi&gt;i&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mo&gt;•&lt;/mo&gt;&lt;/mrow&gt; &lt;/msup&gt; &lt;/mrow&gt; &lt;/math&gt; , &lt;math&gt; &lt;mrow&gt;&lt;mrow&gt;&lt;mo&gt;(&lt;/mo&gt; &lt;msup&gt;&lt;mrow&gt;&lt;mi&gt;p&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mo&gt;•&lt;/mo&gt;&lt;/mrow&gt; &lt;/msup&gt; &lt;mo&gt;:&lt;/mo&gt; &lt;msup&gt;&lt;mrow&gt;&lt;mi&gt;s&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mo&gt;•&lt;/mo&gt;&lt;/mrow&gt; &lt;/msup&gt; &lt;mo&gt;)&lt;/mo&gt;&lt;/mrow&gt; &lt;mspace&gt;&lt;/mspace&gt; &lt;mi&gt;x&lt;/mi&gt; &lt;mspace&gt;&lt;/mspace&gt; &lt;msup&gt;&lt;mrow&gt;&lt;mi&gt;i&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mo&gt;•&lt;/mo&gt;&lt;/mrow&gt; &lt;/msup&gt; &lt;/mrow&gt; &lt;/math&gt; , &lt;math&gt; &lt;mrow&gt;&lt;mrow&gt;&lt;mo&gt;(&lt;/mo&gt; &lt;msup&gt;&lt;mrow&gt;&lt;mi&gt;p&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mo&gt;•&lt;/mo&gt;&lt;/mrow&gt; &lt;/msup&gt; &lt;mo&gt;:&lt;/mo&gt; &lt;msup&gt;&lt;mrow&gt;&lt;mi&gt;g&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mo&gt;•&lt;/mo&gt;&lt;/mrow&gt; &lt;/msup&gt; &lt;mo&gt;)&lt;/mo&gt;&lt;/mrow&gt; &lt;mspace&gt;&lt;/mspace&gt; &lt;mi&gt;x&lt;/mi&gt; &lt;mspace&gt;&lt;/mspace&gt; &lt;msup&gt;&lt;mrow&gt;&lt;mi&gt;i&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mo&gt;•&lt;/mo&gt;&lt;/mrow&gt; &lt;/msup&gt; &lt;/mrow&gt; &lt;/math&gt; , &lt;math&gt; &lt;mrow&gt; &lt;msup&gt;&lt;mrow&gt;&lt;mi&gt;g&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mo&gt;•&lt;/mo&gt;&lt;/mrow&gt; &lt;/msup&gt; &lt;mspace&gt;&lt;/mspace&gt; &lt;mi&gt;x&lt;/mi&gt; &lt;mspace&gt;&lt;/mspace&gt; &lt;msup&gt;&lt;mrow&gt;&lt;mi&gt;i&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mo&gt;•&lt;/mo&gt;&lt;/mrow&gt; &lt;/msup&gt; &lt;/mrow&gt; &lt;/math&gt; , &lt;math&gt; &lt;mrow&gt; &lt;msup&gt;&lt;mrow&gt;&lt;mi&gt;p&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mo&gt;•&lt;/mo&gt;&lt;/mrow&gt; &lt;/msup&gt; &lt;mi&gt;x&lt;/mi&gt; &lt;mspace&gt;&lt;/mspace&gt; &lt;msup&gt;&lt;mrow&gt;&lt;mi&gt;i&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mo&gt;•&lt;/mo&gt;&lt;/mrow&gt; &lt;/msup&gt; &lt;/mrow&gt; &lt;/math&gt; , and &lt;math&gt; &lt;mrow&gt; &lt;msup&gt;&lt;mrow&gt;&lt;mi&gt;s&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mo&gt;•&lt;/mo&gt;&lt;/mrow&gt; &lt;/msup&gt; &lt;mi&gt;x&lt;/mi&gt; &lt;mspace&gt;&lt;/mspace&gt; &lt;msup&gt;&lt;mrow&gt;&lt;mi&gt;i&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mo&gt;•&lt;/mo&gt;&lt;/mrow&gt; &lt;/msup&gt; &lt;/mrow&gt; &lt;/math&gt; , with pretest and posttest serving as the two levels of the multivariate facet. In the designs with sites as the object of measurement, &lt;math&gt; &lt;mrow&gt;&lt;mrow&gt;&lt;mo&gt;(&lt;/mo&gt; &lt;msup&gt;&lt;mrow&gt;&lt;mi&gt;P&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mo&gt;•&lt;/mo&gt;&lt;/mrow&gt; &lt;/msup&gt; &lt;mo&gt;:&lt;/mo&gt; &lt;msup&gt;&lt;mrow&gt;&lt;mi&gt;G&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mo&gt;•&lt;/mo&gt;&lt;/mrow&gt; &lt;/msup&gt; &lt;mo&gt;:&lt;/mo&gt; &lt;msup&gt;&lt;mrow&gt;&lt;mi&gt;s&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mo&gt;•&lt;/mo&gt;&lt;/mrow&gt; &lt;/msup&gt; &lt;mo&gt;)&lt;/mo&gt;&lt;/mrow&gt; &lt;mspace&gt;&lt;/mspace&gt; &lt;mi&gt;x&lt;/mi&gt; &lt;mspace&gt;&lt;/mspace&gt; &lt;msup&gt;&lt;mrow&gt;&lt;mi&gt;I&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mo&gt;•&lt;/mo&gt;&lt;/mrow&gt; &lt;/msup&gt; &lt;/mrow&gt; &lt;/math&gt; , &lt;math&gt; &lt;mrow&gt;&lt;mrow&gt;&lt;mo&gt;(&lt;/mo&gt; &lt;msup&gt;&lt;mrow&gt;&lt;mi&gt;P&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mo&gt;•&lt;/mo&gt;&lt;/mrow&gt; &lt;/msup&gt; &lt;mo&gt;:&lt;/mo&gt; &lt;msup&gt;&lt;mrow&gt;&lt;mi&gt;s&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mo&gt;•&lt;/mo&gt;&lt;/mrow&gt; &lt;/msup&gt; &lt;mo&gt;)&lt;/mo&gt;&lt;/mrow&gt; &lt;mspace&gt;&lt;/mspace&gt; &lt;mi&gt;x&lt;/mi&gt; &lt;mspace&gt;&lt;/mspace&gt; &lt;msup&gt;&lt;mrow&gt;&lt;mi&gt;I&lt;/mi&gt;&lt;/mrow&gt; &lt;mrow&gt;&lt;mo&gt;•&lt;/mo&gt;&lt;/mrow&gt; &lt;/msup&gt; &lt;/mrow&gt; &lt;/math&gt; , and &lt;math&gt; &lt;mrow&gt; &lt;m","PeriodicalId":11502,"journal":{"name":"Educational and Psychological Measurement","volume":" ","pages":"00131644261451746"},"PeriodicalIF":4.4,"publicationDate":"2026-07-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13346105/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148419451","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A Kurtosis-Adjusted Bias Correction for the Standardized Mean Difference: Extending Hedges' <i>g</i> to Nonnormal Populations.","authors":"Daiki Nakamura","doi":"10.1177/00131644261461535","DOIUrl":"10.1177/00131644261461535","url":null,"abstract":"<p><p>The standardized mean difference (SMD) is the most widely used effect size measure in the behavioral, educational, and social sciences. Hedges' <i>g</i>, which applies a bias correction factor to Cohen's <i>d</i>, assumes normally distributed populations. This article proposes a kurtosis-adjusted estimator, <math> <mrow> <msubsup><mrow><mi>g</mi></mrow> <mrow><mi>κ</mi></mrow> <mrow><mo>*</mo></mrow> </msubsup> </mrow> </math> , that extends Hedges' correction by incorporating an additional multiplicative factor based on the sample excess kurtosis. The estimator is intended for continuous, common-shape distributions that depart from normality primarily through excess kurtosis with at most moderate skewness; it is not intended for multimodal, infinite-variance, or severely skewed-and-heavy-tailed distributions. The proposed estimator uses the exact Hedges correction factor <math><mrow><mi>J</mi> <mrow><mo>(</mo> <mi>ν</mi> <mo>)</mo></mrow> </mrow> </math> as its anchor, so that it reduces algebraically to Hedges' <i>g</i> when the estimated excess kurtosis equals zero. We provide a formal derivation of the kurtosis-induced bias under explicit regularity conditions, including the treatment of the numerator-denominator dependence under nonnormality and the quantification of plug-in estimation error. A comprehensive Monte Carlo simulation study (400+ conditions × 10,000 replications) demonstrates that (a) under heavy-tailed distributions ( <math> <mrow> <msub><mrow><mi>γ</mi></mrow> <mrow><mn>2</mn></mrow> </msub> <mo>≥</mo> <mn>4</mn></mrow> </math> ), <math> <mrow> <msubsup><mrow><mi>g</mi></mrow> <mrow><mi>κ</mi></mrow> <mrow><mo>*</mo></mrow> </msubsup> </mrow> </math> reduces the residual bias of Hedges' <i>g</i> by 36% to 92% at sample sizes of 20-100 per group; (b) the associated 95% confidence interval achieves coverage of .952-.961 under the main nonnormal distributions examined (excluding severe <i>g</i>-and-<i>h</i> boundary cases), compared with .938-.947 for the classical Hedges interval and .954-.961 for the Bonett kurtosis-adjusted interval, with modestly narrower widths than Bonett; (c) under normality, the estimator's finite-sample behavior is very close to Hedges' <i>g</i>, with small differences induced by sampling variability and finite-sample bias in the moment-based kurtosis estimate; and (d) the estimator performed well across a broad range of symmetric and moderately skewed distributions, while its effectiveness was attenuated when severe skewness and extreme kurtosis co-occurred.</p>","PeriodicalId":11502,"journal":{"name":"Educational and Psychological Measurement","volume":" ","pages":"00131644261461535"},"PeriodicalIF":4.4,"publicationDate":"2026-07-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13328115/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148390173","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A Simple Approach for Differential Test Functioning Based on Sum Scores.","authors":"Yutaro Sakamoto, Ryuichi Kumagai","doi":"10.1177/00131644261455414","DOIUrl":"10.1177/00131644261455414","url":null,"abstract":"<p><p>Differential test functioning (DTF) evaluates whether a test exhibits group differences beyond those attributable to latent trait differences. Its magnitude is often most interpretable on the raw-score scale. However, most existing DTF effect size measures rely on item response theory (IRT) modeling. Building on the renewed practical value of sum scores, this study proposes a simple observed-score-based approach to estimate the DTF magnitude directly in raw-score points. The proposed Index S stratifies examinees by an anchor-based sum score composed of items not flagged for differential item functioning (DIF), summarizes the within-stratum mean differences in total test scores, and aggregates these conditional differences using weights based on the observed distribution of the matching variable. To stabilize the estimation when score strata are sparse, adjacent strata are merged to satisfy a minimum per-group sample size requirement, and Index S_std provides a standardized version. We evaluated the indices using two-parameter logistic (2PL) simulation studies by varying the sample size, test length, DIF type, DIF proportion, DIF direction, and focal group latent trait distribution. The utility was assessed in terms of estimation accuracy, including Bias and root mean square error (RMSE), and correlation with established IRT-based DTF indices. An empirical application to TIMSS 2023 Grade 8 mathematics (Japan vs. the United States) illustrates how the proposed indices provide an accessible raw-score interpretation of the DTF magnitude for both psychometric and non-psychometric stakeholders.</p>","PeriodicalId":11502,"journal":{"name":"Educational and Psychological Measurement","volume":" ","pages":"00131644261455414"},"PeriodicalIF":4.4,"publicationDate":"2026-06-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13314674/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148359914","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Evaluating Factor Retention in Large Factor Analysis Models: A Simulation Study Comparing 15 Methods.","authors":"Ruoqian Wu, Yan Xia","doi":"10.1177/00131644261448083","DOIUrl":"https://doi.org/10.1177/00131644261448083","url":null,"abstract":"<p><p>Exploratory factor analysis (EFA) is widely used to identify latent structures measured by observed indicators. Nevertheless, determining the number of factors remains a methodological challenge, especially when the size of the EFA model is large (e.g., models with more than 10 indicators per factor and more than five factors). This study evaluated the performance of 15 factor-retention methods under large factor analysis models, including parallel analysis (PA) using the mean and 95th percentiles of threshold, exploratory graph analysis (EGA) with GLASSO and TMFG estimations, sequential χ², fit indices (Comparative Fit Index [CFI], Tucker-Lewis Index [TLI], Root Mean Square Error of Approximation [RMSEA]), Kaiser Criterion (K1 rule), the very simple structure (VSS) method, the comparison data (CD) approach, the minimum average partial (MAP) test, the Hull method using the comparative fit index, Cattell's acceleration factor criteria, and Bayesian information criterion (BIC). Specifically, we manipulated the number of factors (5, 6, and 7), indicators per factor (10 and 15), sample sizes (100, 200, 500, 800, 1,000, 1,500, 2,000, and 4,000), inter-factor correlations (.30, .50, and .70), and factor loadings (.40 and .70). Results reveal that PA-M and EGA-TMFG are the most robust methods across varying conditions. PA-M excels under low-to-moderate inter-factor correlations, while EGA-TMFG is more accurate in high-correlation scenarios when sample sizes are sufficient. Notably, EGA-Glasso may fail to converge when sample size is insufficient in large factor models.</p>","PeriodicalId":11502,"journal":{"name":"Educational and Psychological Measurement","volume":" ","pages":"00131644261448083"},"PeriodicalIF":4.4,"publicationDate":"2026-06-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13303264/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148344328","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Agreement and Alignment in Binary Rating Tasks: Strategic Convergence as an Equilibrium Outcome.","authors":"Irene Gianeselli","doi":"10.1177/00131644261455307","DOIUrl":"https://doi.org/10.1177/00131644261455307","url":null,"abstract":"<p><p>Agreement between raters does not, by itself, show how that agreement is produced. In binary judgement tasks, raters may classify cases similarly either because they rely on similar decision thresholds or because those thresholds operate under favourable marginal conditions. The present study examined whether convergence in raters' decision thresholds must be imposed exogenously or can emerge endogenously through strategic adjustment. Within an equal-variance signal-detection framework, raters were modelled as choosing thresholds that balance expected classification accuracy against an incentive to align with the rest of the panel. Equilibrium thresholds were obtained by Nash best-response updating, and threshold variance, the Strategic Convergence Index (SCI), and mean pairwise Cohen's <math><mrow><mi>κ</mi></mrow> </math> were examined as functions of alignment pressure and prevalence. Increasing alignment pressure produced a sharp collapse in threshold dispersion and drove SCI towards 1 under the chosen normalisation, whereas <math><mrow><mi>κ</mi></mrow> </math> increased only within a comparatively narrow range. When alignment pressure was held constant, SCI remained effectively unchanged across prevalence conditions, while <math><mrow><mi>κ</mi></mrow> </math> varied substantially. These findings indicate that convergence in decision thresholds can arise as an equilibrium outcome of strategic interaction yet remains analytically distinct from observable agreement. They also clarify the status of Cohen's <math><mrow><mi>κ</mi></mrow> </math> in this setting: it tracks agreement at the level of realised classifications and therefore remains sensitive to marginal conditions, whereas SCI captures convergence in the latent decision structure from which those classifications arise. SCI is thus not proposed as a replacement for agreement coefficients but as a model-based quantity that makes explicit a form of alignment that <math><mrow><mi>κ</mi></mrow> </math> only partially reflects.</p>","PeriodicalId":11502,"journal":{"name":"Educational and Psychological Measurement","volume":" ","pages":"00131644261455307"},"PeriodicalIF":4.4,"publicationDate":"2026-06-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13303268/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148344403","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}