Applied Psychological Measurement最新文献_第3页

Online Parameter Estimation for Student Evaluation of Teaching. 学生教学评价的在线参数估计。

IF 1.2 4区心理学

Applied Psychological Measurement Pub Date : 2023-06-01 Epub Date: 2023-03-19 DOI: 10.1177/01466216231165314

Chia-Wen Chen, Chen-Wei Liu

引用次数: 0

Using a Generalized Logistic Regression Method to Detect Differential Item Functioning With Multiple Groups in Cognitive Diagnostic Tests. 使用广义逻辑回归法检测认知诊断测试中多个组别的差异项目功能。

IF 1.2 4区心理学

Applied Psychological Measurement Pub Date : 2023-06-01 Epub Date: 2023-05-13 DOI: 10.1177/01466216231174559

Xiaojian Sun, Shimeng Wang, Lei Guo, Tao Xin, Naiqing Song

{"title":"Using a Generalized Logistic Regression Method to Detect Differential Item Functioning With Multiple Groups in Cognitive Diagnostic Tests.","authors":"Xiaojian Sun, Shimeng Wang, Lei Guo, Tao Xin, Naiqing Song","doi":"10.1177/01466216231174559","DOIUrl":"10.1177/01466216231174559","url":null,"abstract":"Items with the presence of differential item functioning (DIF) will compromise the validity and fairness of a test. Studies have investigated the DIF effect in the context of cognitive diagnostic assessment (CDA), and some DIF detection methods have been proposed. Most of these methods are mainly designed to perform the presence of DIF between two groups; however, empirical situations may contain more than two groups. To date, only a handful of studies have detected the DIF effect with multiple groups in the CDA context. This study uses the generalized logistic regression (GLR) method to detect DIF items by using the estimated attribute profile as matching criteria. A simulation study is conducted to examine the performance of the two GLR methods, GLR-based Wald test (GLR-Wald) and GLR-based likelihood ratio test (GLR-LRT), in detecting the DIF items, the results based on the ordinary Wald test are also reported. Results show that (1) both GLR-Wald and GLR-LRT have more reasonable performance in controlling Type I error rates than the ordinary Wald test in most conditions; (2) the GLR method also produces higher empirical rejection rates than the ordinary Wald test in most conditions; and (3) using the estimated attribute profile as the matching criteria can produce similar Type I error rates and empirical rejection rates for GLR-Wald and GLR-LRT. A real data example is also analyzed to illustrate the application of these DIF detection methods in multiple groups.","PeriodicalId":48300,"journal":{"name":"Applied Psychological Measurement","volume":"47 4","pages":"328-346"},"PeriodicalIF":1.2,"publicationDate":"2023-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10240570/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10300639","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

The Impact of Item Model Parameter Variations on Person Parameter Estimation in Computerized Adaptive Testing With Automatically Generated Items. 在使用自动生成项目的计算机化自适应测试中，项目模型参数变化对人员参数估计的影响。

IF 1.2 4区心理学

Applied Psychological Measurement Pub Date : 2023-06-01 Epub Date: 2023-03-17 DOI: 10.1177/01466216231165313

Chen Tian, Jaehwa Choi

{"title":"The Impact of Item Model Parameter Variations on Person Parameter Estimation in Computerized Adaptive Testing With Automatically Generated Items.","authors":"Chen Tian, Jaehwa Choi","doi":"10.1177/01466216231165313","DOIUrl":"10.1177/01466216231165313","url":null,"abstract":"Sibling items developed through automatic item generation share similar but not identical psychometric properties. However, considering sibling item variations may bring huge computation difficulties and little improvement on scoring. Assuming identical characteristics among siblings, this study explores the impact of item model parameter variations (i.e., within-family variation between siblings) on person parameter estimation in linear tests and Computerized Adaptive Testing (CAT). Specifically, we explore (1) what if small/medium/large within-family variance is ignored, (2) if the effect of larger within-model variance can be compensated by greater test length, (3) if the item model pool properties affect the impact of within-family variance on scoring, and (4) if the issues in (1) and (2) are different in linear vs. adaptive testing. Related sibling model is used for data generation and identical sibling model is assumed for scoring. Manipulated factors include test length, the size of within-model variation, and item model pool characteristics. Results show that as within-family variance increases, the standard error of scores remains at similar levels. For correlations between true and estimated score and RMSE, the effect of the larger within-model variance was compensated by test length. For bias, scores are biased towards the center, and bias was not compensated by test length. Despite the within-family variation is random in current simulations, to yield less biased ability estimates, the item model pool should provide balanced opportunities such that \"fake-easy\" and \"fake-difficult\" item instances cancel their effects. The results of CAT are similar to that of linear tests, except for higher efficiency.","PeriodicalId":48300,"journal":{"name":"Applied Psychological Measurement","volume":"47 4","pages":"275-290"},"PeriodicalIF":1.2,"publicationDate":"2023-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10240571/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10300640","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A New Approach to Desirable Responding: Multidimensional Item Response Model of Overclaiming Data. 理想回应的新方法：超额索赔数据的多维项目反应模型。

IF 1.2 4区心理学

Applied Psychological Measurement Pub Date : 2023-05-01 Epub Date: 2023-01-19 DOI: 10.1177/01466216231151704

Kuan-Yu Jin, Delroy L Paulhus, Ching-Lin Shih

{"title":"A New Approach to Desirable Responding: Multidimensional Item Response Model of Overclaiming Data.","authors":"Kuan-Yu Jin, Delroy L Paulhus, Ching-Lin Shih","doi":"10.1177/01466216231151704","DOIUrl":"10.1177/01466216231151704","url":null,"abstract":"A variety of approaches have been presented for assessing desirable responding in self-report measures. Among them, the overclaiming technique asks respondents to rate their familiarity with a large set of real and nonexistent items (foils). The application of signal detection formulas to the endorsement rates of real items and foils yields indices of (a) knowledge accuracy and (b) knowledge bias. This overclaiming technique reflects both cognitive ability and personality. Here, we develop an alternative measurement model based on multidimensional item response theory (MIRT). We report three studies demonstrating this new model's capacity to analyze overclaiming data. First, a simulation study illustrates that MIRT and signal detection theory yield comparable indices of accuracy and bias-although MIRT provides important additional information. Two empirical examples-one based on mathematical terms and one based on Chinese idioms-are then elaborated. Together, they demonstrate the utility of this new approach for group comparisons and item selection. The implications of this research are illustrated and discussed.","PeriodicalId":48300,"journal":{"name":"Applied Psychological Measurement","volume":"47 3","pages":"221-236"},"PeriodicalIF":1.2,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10126390/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9363746","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A Testlet Diagnostic Classification Model with Attribute Hierarchies. 带属性层次的小测试诊断分类模型

IF 1 4区心理学

Applied Psychological Measurement Pub Date : 2023-05-01 Epub Date: 2023-03-21 DOI: 10.1177/01466216231165315

Wenchao Ma, Chun Wang, Jiaying Xiao

{"title":"A Testlet Diagnostic Classification Model with Attribute Hierarchies.","authors":"Wenchao Ma, Chun Wang, Jiaying Xiao","doi":"10.1177/01466216231165315","DOIUrl":"10.1177/01466216231165315","url":null,"abstract":"In this article, a testlet hierarchical diagnostic classification model (TH-DCM) was introduced to take both attribute hierarchies and item bundles into account. The expectation-maximization algorithm with an analytic dimension reduction technique was used for parameter estimation. A simulation study was conducted to assess the parameter recovery of the proposed model under varied conditions, and to compare TH-DCM with testlet higher-order CDM (THO-DCM; Hansen, M. (2013). Hierarchical item response models for cognitive diagnosis (Unpublished doctoral dissertation). UCLA; Zhan, P., Li, X., Wang, W.-C., Bian, Y., & Wang, L. (2015). The multidimensional testlet-effect cognitive diagnostic models. Acta Psychologica Sinica, 47(5), 689. https://doi.org/10.3724/SP.J.1041.2015.00689). Results showed that (1) ignoring large testlet effects worsened parameter recovery, (2) DCMs assuming equal testlet effects within each testlet performed as well as the testlet model assuming unequal testlet effects under most conditions, (3) misspecifications in joint attribute distribution had an differential impact on parameter recovery, and (4) THO-DCM seems to be a robust alternative to TH-DCM under some hierarchical structures. A set of real data was also analyzed for illustration.","PeriodicalId":48300,"journal":{"name":"Applied Psychological Measurement","volume":"47 3","pages":"183-199"},"PeriodicalIF":1.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10126385/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9357116","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

On the Folly of Introducing A (Time-Based UMV), While Designing for B (Time-Based CMV). 在为 B（基于时间的 CMV）设计的同时引入 A（基于时间的 UMV）的愚蠢之举。

IF 1.2 4区心理学

Applied Psychological Measurement Pub Date : 2023-05-01 Epub Date: 2023-03-15 DOI: 10.1177/01466216231165304

Alice Brawley Newlin

引用次数: 0

Enhancing Computerized Adaptive Testing with Batteries of Unidimensional Tests. 用单维测验组增强计算机化自适应测验。

IF 1.2 4区心理学

Applied Psychological Measurement Pub Date : 2023-05-01 Epub Date: 2023-03-24 DOI: 10.1177/01466216231165301

Pasquale Anselmi, Egidio Robusto, Francesca Cristante

引用次数: 0

Confidence Screening Detector: A New Method for Detecting Test Collusion. 信心筛选探测器：检测测试串通的新方法。

IF 1.2 4区心理学

Applied Psychological Measurement Pub Date : 2023-05-01 Epub Date: 2023-03-20 DOI: 10.1177/01466216231165299

Yongze Xu, Ying Cui, Xinyi Wang, Meiwei Huang, Fang Luo

引用次数: 0

A Likelihood Approach to Item Response Theory Equating of Multiple Forms. 项目反应理论多形式等式的可能性方法。

IF 1.2 4区心理学

Applied Psychological Measurement Pub Date : 2023-05-01 Epub Date: 2023-01-24 DOI: 10.1177/01466216231151702

Michela Battauz, Waldir Leôncio

引用次数: 0

A Comparison of Confirmatory Factor Analysis and Network Models for Measurement Invariance Assessment When Indicator Residuals are Correlated. 当指标残差相关时，用于测量不变性评估的确证因子分析与网络模型的比较》（A Comparison of Confirmatory Factor Analysis and Network Models for Measurement Invariance Assessment when Indicator Residuals are Correlated）。

IF 1.2 4区心理学

Applied Psychological Measurement Pub Date : 2023-03-01 Epub Date: 2023-01-14 DOI: 10.1177/01466216231151700

W Holmes Finch, Brian F French, Alicia Hazelwood

{"title":"A Comparison of Confirmatory Factor Analysis and Network Models for Measurement Invariance Assessment When Indicator Residuals are Correlated.","authors":"W Holmes Finch, Brian F French, Alicia Hazelwood","doi":"10.1177/01466216231151700","DOIUrl":"10.1177/01466216231151700","url":null,"abstract":"Social science research is heavily dependent on the use of standardized assessments of a variety of phenomena, such as mood, executive functioning, and cognitive ability. An important assumption when using these instruments is that they perform similarly for all members of the population. When this assumption is violated, the validity evidence of the scores is called into question. The standard approach for assessing the factorial invariance of the measures across subgroups within the population involves multiple groups confirmatory factor analysis (MGCFA). CFA models typically, but not always, assume that once the latent structure of the model is accounted for, the residual terms for the observed indicators are uncorrelated (local independence). Commonly, correlated residuals are introduced after a baseline model shows inadequate fit and inspection of modification indices ensues to remedy fit. An alternative procedure for fitting latent variable models that may be useful when local independence does not hold is based on network models. In particular, the residual network model (RNM) offers promise with respect to fitting latent variable models in the absence of local independence via an alternative search procedure. This simulation study compared the performances of MGCFA and RNM for measurement invariance assessment when local independence is violated, and residual covariances are themselves not invariant. Results revealed that RNM had better Type I error control and higher power compared to MGCFA when local independence was absent. Implications of the results for statistical practice are discussed.","PeriodicalId":48300,"journal":{"name":"Applied Psychological Measurement","volume":"47 2","pages":"106-122"},"PeriodicalIF":1.2,"publicationDate":"2023-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9979199/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10845586","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0