AssessmentPub Date : 2024-12-01Epub Date: 2024-02-15DOI: 10.1177/10731911241229566
Tiffany Wu, Christina Weiland, Meghan McCormick, JoAnn Hsueh, Catherine Snow, Jason Sachs
{"title":"One Score to Rule Them All? Comparing the Predictive and Concurrent Validity of 30 Hearts and Flowers Scoring Approaches.","authors":"Tiffany Wu, Christina Weiland, Meghan McCormick, JoAnn Hsueh, Catherine Snow, Jason Sachs","doi":"10.1177/10731911241229566","DOIUrl":"10.1177/10731911241229566","url":null,"abstract":"<p><p>The Hearts and Flowers (H&F) task is a computerized executive functioning (EF) assessment that has been used to measure EF from early childhood to adulthood. It provides data on accuracy and reaction time (RT) across three different task blocks (hearts, flowers, and mixed). However, there is a lack of consensus in the field on how to score the task that makes it difficult to interpret findings across studies. The current study, which includes a demographically diverse population of kindergarteners from Boston Public Schools (<i>N</i> = 946), compares the predictive and concurrent validity of 30 ways of scoring H&F, each with a different combination of accuracy, RT, and task block(s). Our exploratory results provide evidence supporting the use of a <i>two-vector average</i> score based on Zelazo et al.'s approach of adding accuracy and RT scores together only after individuals pass a certain accuracy threshold. Findings have implications for scoring future tablet-based developmental assessments.</p>","PeriodicalId":8577,"journal":{"name":"Assessment","volume":" ","pages":"1702-1720"},"PeriodicalIF":3.5,"publicationDate":"2024-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139740249","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
AssessmentPub Date : 2024-12-01Epub Date: 2024-02-12DOI: 10.1177/10731911241229573
Pascal Schlechter, Sharon A S Neufeld
{"title":"Longitudinal and Gender Measurement Invariance of the General Health Questionnaire-12 (GHQ-12) From Adolescence to Emerging Adulthood.","authors":"Pascal Schlechter, Sharon A S Neufeld","doi":"10.1177/10731911241229573","DOIUrl":"10.1177/10731911241229573","url":null,"abstract":"<p><p>Psychological distress often onsets during adolescence, necessitating an accurate understanding of its development. Assessing change in distress is based on the seldom examined premise of longitudinal measurement invariance (MI). Thus, we used three waves of data from Next Steps, a representative cohort of young people in the UK (<i>N</i> = 13,539) to examine MI of the General Health Questionnaire-12 (GHQ-12). We examined MI across time and gender from ages 15 to 25 in four competing latent models: (a) a single-factor model, (b) a three-factor correlated model, (c) a bifactor model of \"general distress\" and two orthogonal specific factors capturing positive and negative wording, and (d) a single-factor model including error covariances of negatively phrased items. We also tested acceptability of assumptions underlying sum score models. For all factor models, residual MI was confirmed from ages 15 to 25 years and across gender. The bifactor model had the best fit. While sum score model fit was not unequivocally acceptable, most mean differences across time and gender were equivalent across sum scores and latent difference scores. Thus, GHQ-12 sum scores may be used to assess change in psychological distress in young people. However, latent scores appear more accurate, and model fit can be improved by accounting for item wording.</p>","PeriodicalId":8577,"journal":{"name":"Assessment","volume":" ","pages":"1687-1701"},"PeriodicalIF":3.5,"publicationDate":"2024-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11492552/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139721339","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
AssessmentPub Date : 2024-11-29DOI: 10.1177/10731911241299723
E David Klonsky
{"title":"How to Produce, Identify, and Motivate Robust Psychological Science: A Roadmap and a Response to Vize et al.","authors":"E David Klonsky","doi":"10.1177/10731911241299723","DOIUrl":"https://doi.org/10.1177/10731911241299723","url":null,"abstract":"<p><p>Some wish to mandate preregistration as a response to the replication crisis, while I and others caution that such mandates inadvertently cause harm and distract from more critical reforms. In this article, after briefly critiquing a recently published defense of preregistration mandates, I propose a three-part vision for cultivating a robust and cumulative psychological science. First, we must know how to produce robust rather than fragile findings. Key ingredients include sufficient sample sizes, valid measurement, and honesty/transparency. Second, we must know how to identify robust (and non-robust) findings. To this end, I reframe robustness checks broadly into four types: across analytic decisions, across measures, across samples, and across investigative teams. Third, we must be motivated to produce and care about robust science. This aim requires marshaling sociocultural forces to support, reward, and celebrate the production of robust findings, just as we once rewarded flashy but fragile findings. Critically, these sociocultural reinforcements must be tied as closely as possible to rigor and robustness themselves-rather than cosmetic indicators of rigor and robustness, as we have done in the past.</p>","PeriodicalId":8577,"journal":{"name":"Assessment","volume":" ","pages":"10731911241299723"},"PeriodicalIF":3.5,"publicationDate":"2024-11-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142754457","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
AssessmentPub Date : 2024-11-28DOI: 10.1177/10731911241298079
Charlie C Su, Camilo J Ruggero, Craig S Neumann, David C Cicero
{"title":"Measurement Equivalence of Family Functioning and Psychosis Risk Measures in the Adolescent Brain Cognitive Development<sup>SM</sup> Study.","authors":"Charlie C Su, Camilo J Ruggero, Craig S Neumann, David C Cicero","doi":"10.1177/10731911241298079","DOIUrl":"https://doi.org/10.1177/10731911241298079","url":null,"abstract":"<p><p>Decades of research show a clear link between family factors and psychopathology. Family functioning varies across cultures, suggesting potential cultural differences in the association between family factors and psychopathology. In addition, assessing family functioning generally involves tools not systematically validated for diverse cultural backgrounds. Using the Adolescent Brain Cognitive Development<sup>SM</sup> data (<i>N</i> = 11,864), this study found: (a) full scalar invariance was tenable for the Children's Reports of Parental Behavior Inventory (CRPBI) and Prodromal Questionnaire-Brief Child Version (PQ-BC) across race/ethnicity, but not for the Family Environment Scale (FES) and Parental Monitoring Survey (PMQ); (b) the CRPBI and PMQ were significantly associated with the PQ-BC, and (c) all three family scales had equivalent relations with the PQ-BC across groups. This highlights the importance of evaluating scales for measurement invariance across race/ethnicity. Results also help to connect specific family factors to the etiology of psychosis risk among U.S. children and adolescents.</p>","PeriodicalId":8577,"journal":{"name":"Assessment","volume":" ","pages":"10731911241298079"},"PeriodicalIF":3.5,"publicationDate":"2024-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142738201","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
AssessmentPub Date : 2024-11-23DOI: 10.1177/10731911241289249
Lindsay N Gabel, Thomas M Olino, Brandon L Goldstein, Daniel N Klein, Kasey Stanton, Elizabeth P Hayden
{"title":"Latent Structure and Item Functioning of Self-Referent Encoding Task Word Stimuli in Preadolescent Youth.","authors":"Lindsay N Gabel, Thomas M Olino, Brandon L Goldstein, Daniel N Klein, Kasey Stanton, Elizabeth P Hayden","doi":"10.1177/10731911241289249","DOIUrl":"https://doi.org/10.1177/10731911241289249","url":null,"abstract":"<p><p>The Self-Referent Encoding Task (SRET) can be used to measure self-concept via endorsement of trait words, a robust metric associated with depression severity. Our study is the first to investigate the structural validity and item functioning of SRET endorsement scores using confirmatory factor analysis and item response theory. Community-dwelling preadolescent youth (<i>N</i> = 508; <i>M</i><sub>age</sub> = 12.39 years, <i>SD</i><sub>age</sub> = .72) were shown a list of positive and negative trait adjectives and made binary ratings of whether words were self-descriptive. The SRET exhibited a two-factor structure, comprising positive and negative factors. Positive items were endorsed by most children and best estimated information about positive self-concepts below average levels of positivity. Conversely, negative items were unendorsed by most children and best estimated information about negative self-concepts above average levels of negativity. We identify standardized, psychometrically sound, and developmentally sensitive SRET items for assessing youth self-concept and its associations with depression risk.</p>","PeriodicalId":8577,"journal":{"name":"Assessment","volume":" ","pages":"10731911241289249"},"PeriodicalIF":3.5,"publicationDate":"2024-11-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142695241","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
AssessmentPub Date : 2024-11-23DOI: 10.1177/10731911241293222
C E Broshek, A J Kurtz, N S Udupa, C J Chang, M L Rogers, B A Feinstein, E A Selby, K L Gratz, M T Tull, T E Joiner, M E Jeon
{"title":"Psychometric Evaluation of the Heterosexist Harassment, Rejection, and Discrimination Scale: Identification of a Unidimensional 11-Item Scale.","authors":"C E Broshek, A J Kurtz, N S Udupa, C J Chang, M L Rogers, B A Feinstein, E A Selby, K L Gratz, M T Tull, T E Joiner, M E Jeon","doi":"10.1177/10731911241293222","DOIUrl":"https://doi.org/10.1177/10731911241293222","url":null,"abstract":"<p><p>Despite the Heterosexist Harassment, Rejection, and Discrimination Scale (HHRDS) being a routinely used measure of discrimination, there is no current consensus regarding its fundamental psychometric properties. This study sought to: (a) test the dimensionality of the scale using a classical test theory approach, and (b), if lacking a clear multidimensional structure, use an item response theory approach to develop a concise unidimensional measure and examine differential item and test functioning across various identity factors. In a sample of sexual minority adults (<i>N</i> = 1,243), evaluation of the HHRDS supported a reduced 11-item unidimensional measure. Assessment of differential item functioning suggested differences for some items between cisgender men and cisgender women, cisgender men and gender-diverse individuals, gay/lesbian and bi+ individuals, and white/non-Latine individuals and people of color. However, differential item functioning had minimal impact on total test scores, supporting the use of the revised 11-item HHRDS across groups.</p>","PeriodicalId":8577,"journal":{"name":"Assessment","volume":" ","pages":"10731911241293222"},"PeriodicalIF":3.5,"publicationDate":"2024-11-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142695242","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
AssessmentPub Date : 2024-11-20DOI: 10.1177/10731911241293939
Martin Sellbom
{"title":"MMPI-3 Assessment of Externalizing Psychopathology in Targeted Community and University Samples.","authors":"Martin Sellbom","doi":"10.1177/10731911241293939","DOIUrl":"https://doi.org/10.1177/10731911241293939","url":null,"abstract":"<p><p>The Minnesota Multiphasic Personality Inventory-3 (MMPI-3) scales that are designed to assess various forms of externalizing psychopathology have received relatively little research attention to date. The goal of this investigation was therefore to examine the validity of these MMPI-3 scales in the measurement of the externalizing spectrum. A community sample (<i>n</i> = 206) with high levels of externalizing psychopathology and a university sample (<i>n</i> = 645) were used. The former sample was administered structured clinical interviews for various forms of externalizing psychopathology, whereas the student sample completed the Externalizing Spectrum Inventory-160. Confirmatory factor analysis was used to operationalize the externalizing spectrum using each set of measures. The results indicated that the externalizing MMPI-3 scales were associated with a general externalizing factor and an expected pattern of associations, while systematic residuals of individual disorder symptoms also emerged. Moreover, in the university sample, specific MMPI-3 scales hypothesized to assess antagonistic-externalizing (Aggressiveness, Aggression, and Cynicism) emerged as key predictors of a callous-aggression residual group factor. The Substance Abuse scale was unsurprisingly the best predictor of such dysfunction. These findings provide guidance for mental health practitioners who use the MMPI-3 for the assessment of externalizing psychopathology symptoms.</p>","PeriodicalId":8577,"journal":{"name":"Assessment","volume":" ","pages":"10731911241293939"},"PeriodicalIF":3.5,"publicationDate":"2024-11-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142680616","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
AssessmentPub Date : 2024-11-20DOI: 10.1177/10731911241289987
S M Boelders, E Butterbrod, L V D E Vogelsmeier, M M Sitskoorn, L L Ong, K Gehring
{"title":"Factor Structure and Validity of Composite Scores Resulting From a Computerized Cognitive Test Battery in Healthy Adults and Patients With Primary Brain Tumors.","authors":"S M Boelders, E Butterbrod, L V D E Vogelsmeier, M M Sitskoorn, L L Ong, K Gehring","doi":"10.1177/10731911241289987","DOIUrl":"https://doi.org/10.1177/10731911241289987","url":null,"abstract":"<p><p>Computerized neuropsychological test batteries (CNTs), such as Central Nervous System Vital Signs (CNS VS), are increasingly used for measuring cognitive functioning, but empirical evidence of how they measure cognition is scarce. We investigated the factor structure of CNS VS using exploratory factor analyses four samples: healthy adults (<i>n</i> = 169), patients with meningioma (392), low-grade glioma (99), and high-grade glioma (247). We tested model fit and investigated measurement invariance. Differences in factor interpretation existed between healthy participants and patients. Factor structures among patient groups were approximately the same but differed in non-zero loadings. Overall, factor structures largely did not support the \"clinical domains\" provided by CNS VS for clinical interpretation. Confirmatory models did not have a good fit, and measurement invariance could not be established. Our results indicate that (weighted) sum scores of CNS VS results may lack validity. We recommend researchers and clinicians to use scores on individual test measures.</p>","PeriodicalId":8577,"journal":{"name":"Assessment","volume":" ","pages":"10731911241289987"},"PeriodicalIF":3.5,"publicationDate":"2024-11-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142680612","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
AssessmentPub Date : 2024-11-19DOI: 10.1177/10731911241283609
Philipp E Sischka, Isabelle Albert, Anna E Kornadt
{"title":"Validation of the 10-Item Social Provision Scale (SPS-10): Evaluating Factor Structure, Reliability, Measurement Invariance, and Nomological Network Across 38 Countries.","authors":"Philipp E Sischka, Isabelle Albert, Anna E Kornadt","doi":"10.1177/10731911241283609","DOIUrl":"10.1177/10731911241283609","url":null,"abstract":"<p><p>The 24-item Social Provision Scale is among the most frequently used self-report scales to assess perceived social support. A shortened 10-item version (SPS-10) with promising psychometric properties has been proposed. However, so far only a few studies in specific cultural contexts investigated its psychometric properties. Thus, the aim of the current study was an extensive investigation of the psychometric properties of the SPS-10 among 38 countries (<i>N</i> = 88,857 respondents) using the COVIDiSTRESS Global Survey data. Item response theory bifactor analysis revealed that the general factor accounts for the largest part of the covariance among the items in all countries (explained common variance ≥ .75) and that the mean/sum score of the SPS-10 can be used as a total score of perceived general social support. Reliability was high for all countries (ρ<sub>unidimensional GRM</sub> ≥ .89). Measurement invariance and differential item/test functioning analysis of the unidimensional graded response model indicated negligible differential test functioning across countries. The association with the external criteria mainly supported its construct validity. The SPS-10 is a psychometrically sound measure of perceived social support for large-scale cross-cultural studies.</p>","PeriodicalId":8577,"journal":{"name":"Assessment","volume":" ","pages":"10731911241283609"},"PeriodicalIF":3.5,"publicationDate":"2024-11-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142666996","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
AssessmentPub Date : 2024-11-14DOI: 10.1177/10731911241286868
Ginette Lafit, Jordan Revol, Leonie Cloos, Peter Kuppens, Eva Ceulemans
{"title":"The Effect of Different Construct Operationalizations, Study Duration, and Preprocessing Choices on Power-Based Sample Size Recommendations in Intensive Longitudinal Research.","authors":"Ginette Lafit, Jordan Revol, Leonie Cloos, Peter Kuppens, Eva Ceulemans","doi":"10.1177/10731911241286868","DOIUrl":"https://doi.org/10.1177/10731911241286868","url":null,"abstract":"<p><p>To shed light on the dynamics of psychological processes, researchers often collect intensive longitudinal (IL) data by asking people to repeatedly report on their momentary experiences in daily life. Two important decisions when designing an IL study concern the number of persons and the number of measurement occasions to be included. These sample size decisions are ideally based on statistical power considerations. When conducting statistical power analysis, it is necessary to provide the value of the effect size of interest as well as of all other model parameters. In IL research, these values have to be based on previous studies. This implies that these values are subject to large heterogeneity due to differences in study design and preprocessing choices. This between-study heterogeneity can severely impact power-based sample size recommendations. In this article, we introduce an approach to investigate the impact of study design and pre-processing of previous studies and to determine a recommended sample size to account for this impact. We demonstrate how to use this approach to investigate the effect of different construct operationalizations, study duration, and preprocessing choices. This approach paves the way for more thoughtful and robust sample-size decisions.</p>","PeriodicalId":8577,"journal":{"name":"Assessment","volume":" ","pages":"10731911241286868"},"PeriodicalIF":3.5,"publicationDate":"2024-11-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142613862","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}