Applied Psychological Measurement最新文献_第5页

A New Approach to Desirable Responding: Multidimensional Item Response Model of Overclaiming Data. 理想回应的新方法：超额索赔数据的多维项目反应模型。

IF 1.2 4区心理学

Applied Psychological Measurement Pub Date : 2023-05-01 Epub Date: 2023-01-19 DOI: 10.1177/01466216231151704

Kuan-Yu Jin, Delroy L Paulhus, Ching-Lin Shih

{"title":"A New Approach to Desirable Responding: Multidimensional Item Response Model of Overclaiming Data.","authors":"Kuan-Yu Jin, Delroy L Paulhus, Ching-Lin Shih","doi":"10.1177/01466216231151704","DOIUrl":"10.1177/01466216231151704","url":null,"abstract":"A variety of approaches have been presented for assessing desirable responding in self-report measures. Among them, the overclaiming technique asks respondents to rate their familiarity with a large set of real and nonexistent items (foils). The application of signal detection formulas to the endorsement rates of real items and foils yields indices of (a) knowledge accuracy and (b) knowledge bias. This overclaiming technique reflects both cognitive ability and personality. Here, we develop an alternative measurement model based on multidimensional item response theory (MIRT). We report three studies demonstrating this new model's capacity to analyze overclaiming data. First, a simulation study illustrates that MIRT and signal detection theory yield comparable indices of accuracy and bias-although MIRT provides important additional information. Two empirical examples-one based on mathematical terms and one based on Chinese idioms-are then elaborated. Together, they demonstrate the utility of this new approach for group comparisons and item selection. The implications of this research are illustrated and discussed.","PeriodicalId":48300,"journal":{"name":"Applied Psychological Measurement","volume":"47 3","pages":"221-236"},"PeriodicalIF":1.2,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10126390/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9363746","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A Testlet Diagnostic Classification Model with Attribute Hierarchies. 带属性层次的小测试诊断分类模型

IF 1 4区心理学

Applied Psychological Measurement Pub Date : 2023-05-01 Epub Date: 2023-03-21 DOI: 10.1177/01466216231165315

Wenchao Ma, Chun Wang, Jiaying Xiao

{"title":"A Testlet Diagnostic Classification Model with Attribute Hierarchies.","authors":"Wenchao Ma, Chun Wang, Jiaying Xiao","doi":"10.1177/01466216231165315","DOIUrl":"10.1177/01466216231165315","url":null,"abstract":"In this article, a testlet hierarchical diagnostic classification model (TH-DCM) was introduced to take both attribute hierarchies and item bundles into account. The expectation-maximization algorithm with an analytic dimension reduction technique was used for parameter estimation. A simulation study was conducted to assess the parameter recovery of the proposed model under varied conditions, and to compare TH-DCM with testlet higher-order CDM (THO-DCM; Hansen, M. (2013). Hierarchical item response models for cognitive diagnosis (Unpublished doctoral dissertation). UCLA; Zhan, P., Li, X., Wang, W.-C., Bian, Y., & Wang, L. (2015). The multidimensional testlet-effect cognitive diagnostic models. Acta Psychologica Sinica, 47(5), 689. https://doi.org/10.3724/SP.J.1041.2015.00689). Results showed that (1) ignoring large testlet effects worsened parameter recovery, (2) DCMs assuming equal testlet effects within each testlet performed as well as the testlet model assuming unequal testlet effects under most conditions, (3) misspecifications in joint attribute distribution had an differential impact on parameter recovery, and (4) THO-DCM seems to be a robust alternative to TH-DCM under some hierarchical structures. A set of real data was also analyzed for illustration.","PeriodicalId":48300,"journal":{"name":"Applied Psychological Measurement","volume":"47 3","pages":"183-199"},"PeriodicalIF":1.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10126385/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9357116","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

On the Folly of Introducing A (Time-Based UMV), While Designing for B (Time-Based CMV). 在为 B（基于时间的 CMV）设计的同时引入 A（基于时间的 UMV）的愚蠢之举。

IF 1.2 4区心理学

Applied Psychological Measurement Pub Date : 2023-05-01 Epub Date: 2023-03-15 DOI: 10.1177/01466216231165304

Alice Brawley Newlin

引用次数: 0

Enhancing Computerized Adaptive Testing with Batteries of Unidimensional Tests. 用单维测验组增强计算机化自适应测验。

IF 1.2 4区心理学

Applied Psychological Measurement Pub Date : 2023-05-01 Epub Date: 2023-03-24 DOI: 10.1177/01466216231165301

Pasquale Anselmi, Egidio Robusto, Francesca Cristante

引用次数: 0

Confidence Screening Detector: A New Method for Detecting Test Collusion. 信心筛选探测器：检测测试串通的新方法。

IF 1.2 4区心理学

Applied Psychological Measurement Pub Date : 2023-05-01 Epub Date: 2023-03-20 DOI: 10.1177/01466216231165299

Yongze Xu, Ying Cui, Xinyi Wang, Meiwei Huang, Fang Luo

引用次数: 0

A Likelihood Approach to Item Response Theory Equating of Multiple Forms. 项目反应理论多形式等式的可能性方法。

IF 1.2 4区心理学

Applied Psychological Measurement Pub Date : 2023-05-01 Epub Date: 2023-01-24 DOI: 10.1177/01466216231151702

Michela Battauz, Waldir Leôncio

引用次数: 0

A Comparison of Confirmatory Factor Analysis and Network Models for Measurement Invariance Assessment When Indicator Residuals are Correlated. 当指标残差相关时，用于测量不变性评估的确证因子分析与网络模型的比较》（A Comparison of Confirmatory Factor Analysis and Network Models for Measurement Invariance Assessment when Indicator Residuals are Correlated）。

IF 1.2 4区心理学

Applied Psychological Measurement Pub Date : 2023-03-01 Epub Date: 2023-01-14 DOI: 10.1177/01466216231151700

W Holmes Finch, Brian F French, Alicia Hazelwood

{"title":"A Comparison of Confirmatory Factor Analysis and Network Models for Measurement Invariance Assessment When Indicator Residuals are Correlated.","authors":"W Holmes Finch, Brian F French, Alicia Hazelwood","doi":"10.1177/01466216231151700","DOIUrl":"10.1177/01466216231151700","url":null,"abstract":"Social science research is heavily dependent on the use of standardized assessments of a variety of phenomena, such as mood, executive functioning, and cognitive ability. An important assumption when using these instruments is that they perform similarly for all members of the population. When this assumption is violated, the validity evidence of the scores is called into question. The standard approach for assessing the factorial invariance of the measures across subgroups within the population involves multiple groups confirmatory factor analysis (MGCFA). CFA models typically, but not always, assume that once the latent structure of the model is accounted for, the residual terms for the observed indicators are uncorrelated (local independence). Commonly, correlated residuals are introduced after a baseline model shows inadequate fit and inspection of modification indices ensues to remedy fit. An alternative procedure for fitting latent variable models that may be useful when local independence does not hold is based on network models. In particular, the residual network model (RNM) offers promise with respect to fitting latent variable models in the absence of local independence via an alternative search procedure. This simulation study compared the performances of MGCFA and RNM for measurement invariance assessment when local independence is violated, and residual covariances are themselves not invariant. Results revealed that RNM had better Type I error control and higher power compared to MGCFA when local independence was absent. Implications of the results for statistical practice are discussed.","PeriodicalId":48300,"journal":{"name":"Applied Psychological Measurement","volume":"47 2","pages":"106-122"},"PeriodicalIF":1.2,"publicationDate":"2023-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9979199/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10845586","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

The Effects of Rating Designs on Rater Classification Accuracy and Rater Measurement Precision in Large-Scale Mixed-Format Assessments. 在大规模混合格式评估中，评分设计对评分者分类准确性和评分者测量精确度的影响。

IF 1.2 4区心理学

Applied Psychological Measurement Pub Date : 2023-03-01 Epub Date: 2023-01-12 DOI: 10.1177/01466216231151705

Wenjing Guo, Stefanie A Wind

{"title":"The Effects of Rating Designs on Rater Classification Accuracy and Rater Measurement Precision in Large-Scale Mixed-Format Assessments.","authors":"Wenjing Guo, Stefanie A Wind","doi":"10.1177/01466216231151705","DOIUrl":"10.1177/01466216231151705","url":null,"abstract":"In standalone performance assessments, researchers have explored the influence of different rating designs on the sensitivity of latent trait model indicators to different rater effects as well as the impacts of different rating designs on student achievement estimates. However, the literature provides little guidance on the degree to which different rating designs might affect rater classification accuracy (severe/lenient) and rater measurement precision in both standalone performance assessments and mixed-format assessments. Using results from an analysis of National Assessment of Educational Progress (NAEP) data, we conducted simulation studies to systematically explore the impacts of different rating designs on rater measurement precision and rater classification accuracy (severe/lenient) in mixed-format assessments. The results suggest that the complete rating design produced the highest rater classification accuracy and greatest rater measurement precision, followed by the multiple-choice (MC) + spiral link design and the MC link design. Considering that complete rating designs are not practical in most testing situations, the MC + spiral link design may be a useful choice because it balances cost and performance. We consider the implications of our findings for research and practice.","PeriodicalId":48300,"journal":{"name":"Applied Psychological Measurement","volume":"47 2","pages":"91-105"},"PeriodicalIF":1.2,"publicationDate":"2023-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9979195/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10846015","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Evaluating Equating Transformations in IRT Observed-Score and Kernel Equating Methods. 评估 IRT 观察得分和核等价方法中的等价变换。

IF 1 4区心理学

Applied Psychological Measurement Pub Date : 2023-03-01 Epub Date: 2022-10-04 DOI: 10.1177/01466216221124087

Waldir Leôncio, Marie Wiberg, Michela Battauz

{"title":"Evaluating Equating Transformations in IRT Observed-Score and Kernel Equating Methods.","authors":"Waldir Leôncio, Marie Wiberg, Michela Battauz","doi":"10.1177/01466216221124087","DOIUrl":"10.1177/01466216221124087","url":null,"abstract":"Test equating is a statistical procedure to ensure that scores from different test forms can be used interchangeably. There are several methodologies available to perform equating, some of which are based on the Classical Test Theory (CTT) framework and others are based on the Item Response Theory (IRT) framework. This article compares equating transformations originated from three different frameworks, namely IRT Observed-Score Equating (IRTOSE), Kernel Equating (KE), and IRT Kernel Equating (IRTKE). The comparisons were made under different data-generating scenarios, which include the development of a novel data-generation procedure that allows the simulation of test data without relying on IRT parameters while still providing control over some test score properties such as distribution skewness and item difficulty. Our results suggest that IRT methods tend to provide better results than KE even when the data are not generated from IRT processes. KE might be able to provide satisfactory results if a proper pre-smoothing solution can be found, while also being much faster than IRT methods. For daily applications, we recommend observing the sensibility of the results to the equating method, minding the importance of good model fit and meeting the assumptions of the framework.","PeriodicalId":48300,"journal":{"name":"Applied Psychological Measurement","volume":"47 2","pages":"123-140"},"PeriodicalIF":1.0,"publicationDate":"2023-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ftp.ncbi.nlm.nih.gov/pub/pmc/oa_pdf/74/30/10.1177_01466216221124087.PMC9979196.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10846018","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Heywood Cases in Unidimensional Factor Models and Item Response Models for Binary Data. 二元数据的单维因子模型和项目反应模型中的海伍德案例。

IF 1.2 4区心理学

Applied Psychological Measurement Pub Date : 2023-03-01 Epub Date: 2023-01-29 DOI: 10.1177/01466216231151701

Selena Wang, Paul De Boeck, Marcel Yotebieng

{"title":"Heywood Cases in Unidimensional Factor Models and Item Response Models for Binary Data.","authors":"Selena Wang, Paul De Boeck, Marcel Yotebieng","doi":"10.1177/01466216231151701","DOIUrl":"10.1177/01466216231151701","url":null,"abstract":"Heywood cases are known from linear factor analysis literature as variables with communalities larger than 1.00, and in present day factor models, the problem also shows in negative residual variances. For binary data, factor models for ordinal data can be applied with either delta parameterization or theta parametrization. The former is more common than the latter and can yield Heywood cases when limited information estimation is used. The same problem shows up as non convergence cases in theta parameterized factor models and as extremely large discriminations in item response theory (IRT) models. In this study, we explain why the same problem appears in different forms depending on the method of analysis. We first discuss this issue using equations and then illustrate our conclusions using a small simulation study, where all three methods, delta and theta parameterized ordinal factor models (with estimation based on polychoric correlations and thresholds) and an IRT model (with full information estimation), are used to analyze the same datasets. The results generalize across WLS, WLSMV, and ULS estimators for the factor models for ordinal data. Finally, we analyze real data with the same three approaches. The results of the simulation study and the analysis of real data confirm the theoretical conclusions.","PeriodicalId":48300,"journal":{"name":"Applied Psychological Measurement","volume":"47 2","pages":"141-154"},"PeriodicalIF":1.2,"publicationDate":"2023-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9979198/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10846019","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0