Stephen M. Humphry, Paul Montuoro, Carolyn Maxwell
{"title":"Cumulative Ordering as Evidence of Construct Validity for Assessments of Developmental Attributes","authors":"Stephen M. Humphry, Paul Montuoro, Carolyn Maxwell","doi":"10.1177/07342829231199007","DOIUrl":"https://doi.org/10.1177/07342829231199007","url":null,"abstract":"This article builds upon a proiminent definition of construct validity that focuses on variation in attributes causing variation in measurement outcomes. This article synthesizes the defintion and uses Rasch measurement modeling to explicate a modified conceptualization of construct validity for assessments of developmental attributes. If attributes are conceived as developmental, hypotheses about how new knowledge builds cumulatively upon the cognitive capacity afforded by prior knowledge can be developed. This cumulative ordering of knowledge required to accomplish test items constitutes evidence of a specific form of construct validity. Examples of cumulative ordering appear in the extant literature, but they are rare and confined to the early literature. Furthermore, cumulative ordering has never been explicated, especially its relationship to construct validity. This article describes three of the most complete examples of cumulative ordering in the literature. These examples are used to synthesize a method for assessing cumulative ordering, in which the Rasch model is used to assess the progression of item difficulties which are, in turn, used to review developmental theories and hypotheses, and the tests themselves. We discuss how this conceptualization of construct validity can lead to a more direct relationship between developmental theories and tests which, for practitioners, should result in a clearer understanding of what tests results actually mean. Finally, we discuss how cumulative ordering can be used to facilitate decisions about consequential validity.","PeriodicalId":51446,"journal":{"name":"Journal of Psychoeducational Assessment","volume":" ","pages":""},"PeriodicalIF":1.7,"publicationDate":"2023-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42192450","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Further Validation of the Social Efficacy and Social Outcome Expectations Scale","authors":"Stephen L. Wright, Michael A. Jenkins-Guarnieri","doi":"10.1177/07342829231198277","DOIUrl":"https://doi.org/10.1177/07342829231198277","url":null,"abstract":"The current study sought out to advance the Social Self-Efficacy and Social Outcome Expectations scale using multiple approaches to scale development. Data from 583 undergraduate students were used in two scale development approaches: Classic Test Theory (CTT) and Item Response Theory (IRT). Confirmatory factor analysis suggested a 2-factor structure that aligns with the theoretically based domains for SEOES items and supports previously proposed models of this scale from CTT and psychometric analyses. The IRT analysis indicated that the SEOES items have greater measurement precision at measuring lower levels of the latent constructs. Future research directions are provided and practice implications are discussed.","PeriodicalId":51446,"journal":{"name":"Journal of Psychoeducational Assessment","volume":" ","pages":""},"PeriodicalIF":1.7,"publicationDate":"2023-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45401363","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Do Dimensions of Perfectionism Predict Dimensions of Test Anxiety While Controlling for Depression?","authors":"P. Lowe","doi":"10.1177/07342829221095653","DOIUrl":"https://doi.org/10.1177/07342829221095653","url":null,"abstract":"The present study examined whether cultural differences in different dimensions of perfectionism exist and whether different dimensions of perfectionism (i.e., rigid and self-critical perfectionism) predicted different dimensions of test anxiety while controlling for depression in a sample of Canadian and Singapore higher education students. In addition, culture was examined to determine whether it served as a moderator variable in the relationship between different dimensions of perfectionism and different dimensions of test anxiety. The present study was grounded in DiBartolo and Rendón’s cross-cultural framework for conducting intra- and intercultural research in the area of perfectionism. The sample for the study included 1,095 undergraduate students. Perfectionism, test anxiety, and depression measures were administered to the students online. The results of mean and covariance analyses found the perfectionism measure was invariant across Canadian and Singapore students. In addition, the results of latent mean analyses found no significant differences on the different dimensions of perfectionism between Canadian and Singapore students. The results of analyses of variance also found no significant differences in different ethnic groups on the different dimensions of perfectionism in Canada and Singapore. Furthermore, the results of five hierarchical regression analyses found self-critical perfectionism explained unique variance in the five different test anxiety dimensions while controlling for depression, and culture did not serve as a moderator variable in the relationship between the different dimensions of perfectionism and test anxiety. Implications of the findings are discussed.","PeriodicalId":51446,"journal":{"name":"Journal of Psychoeducational Assessment","volume":"41 1","pages":"634 - 656"},"PeriodicalIF":1.7,"publicationDate":"2023-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49052809","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Kayleigh O’Donnell, Amy L. Reschly, James J. Appleton
{"title":"Assessment of Engagement and Disaffection With the Student Engagement Instrument","authors":"Kayleigh O’Donnell, Amy L. Reschly, James J. Appleton","doi":"10.1177/07342829231199314","DOIUrl":"https://doi.org/10.1177/07342829231199314","url":null,"abstract":"Research suggests the need to assess both positive and negative forms of student engagement. The purpose of this study was to pilot disaffection items with the Student Engagement Instrument (SEI) with a sample of middle school students from a rural area in the Southeastern U.S. This study explored the factor structure of the piloted items alongside the SEI, measurement invariance, and associations between student engagement and disaffection with educational outcomes such as mathematics and reading test scores, discipline referrals, and absences. Results hold implications for our theoretical understanding of engagement, suggesting that engagement and disaffection dimensions are theoretically and psychometrically distinct.","PeriodicalId":51446,"journal":{"name":"Journal of Psychoeducational Assessment","volume":" ","pages":""},"PeriodicalIF":1.7,"publicationDate":"2023-08-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42875913","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Psychometric Properties of the Turkish Version of the Teacher Academic Optimism Scale","authors":"Erdem Karataş, Murat Özdemir","doi":"10.1177/07342829231197460","DOIUrl":"https://doi.org/10.1177/07342829231197460","url":null,"abstract":"This study aimed to adapt Teacher Academic Optimism Scale-Secondary (TAOS-S) to Turkish culture. A total of 453 public school teachers in Turkey participated in the study. We examined the validity, reliability, and measurement invariance of the scale across school levels. The results indicated good internal consistency of the TAOS, suggesting a good measure to assess teacher academic optimism. Confirmatory factor analysis revealed that the three-dimensional individual teacher academic optimism construct showed a good fit with strong reliability evidence. Multigroup confirmatory factor analysis results indicated both configural and metric invariance was observed across school levels; however, scalar invariance was only partially confirmed. Overall, our results show the TAOS has sound psychometric properties, is culturally and linguistically acceptable, and is equally effective in assessing the academic optimism of Turkish teachers.","PeriodicalId":51446,"journal":{"name":"Journal of Psychoeducational Assessment","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-08-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135520511","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Evaluating Phonemic Awareness and Orthographic Mapping With the Woodcock–Johnson IV","authors":"Jason R. Parkin, Daniel B. Hajovsky, V. Alfonso","doi":"10.1177/07342829231196199","DOIUrl":"https://doi.org/10.1177/07342829231196199","url":null,"abstract":"Although phonemic awareness is an essential skill in learning to decode written words, practitioners may question which phonemic awareness tasks best operationalize their relationship with orthographic mapping, the process that converts a decoded word into one instantly recognized on sight. Tests from the Woodcock– Johnson IV were used to evaluate the effects of phonemic awareness tasks and vocabulary on measures of pseudoword decoding, word reading, and spelling in three age groups (ages 6 to 8, 9 to 13, and 14 to 19 years) within the WJ IV normative sample ( N = 4082). Results from path analysis indicated the effects of phonemic awareness tasks and vocabulary varied depending on age and reading task type with mixed results based on theoretical expectations. Results from the 9 to 13 age group appeared closest to conforming to hypotheses. We discuss implications of measuring phonemic awareness, reading, and spelling in the context of comprehensive psychoeducational assessment.","PeriodicalId":51446,"journal":{"name":"Journal of Psychoeducational Assessment","volume":"1 1","pages":""},"PeriodicalIF":1.7,"publicationDate":"2023-08-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41814013","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Examining the Psychometric Properties of the Motivational Scale of Motivated Strategies for Learning Questionnaire for English Learning Among Chinese Secondary Students","authors":"Zhengdong Gan, Zheng Yuan, Randall Schumacker","doi":"10.1177/07342829231193064","DOIUrl":"https://doi.org/10.1177/07342829231193064","url":null,"abstract":"This study examined the psychometric properties of the Motivational scale of the Motivated Strategies for Learning Questionnaire in a sample of 656 Chinese secondary students in an English learning context. Exploratory factor analysis and confirmatory factor analysis results suggested that a five-factor motivational structure fit the data better as opposed to the original six-factor motivational model reported by Pintrich and his colleagues. Reliability coefficients of these five motivational subscales (i.e., intrinsic value, extrinsic goal orientation, control of learning beliefs, self-efficacy for learning and performance , and test anxiety) were in the adequate to good range. All motivational subscales except test anxiety were positively correlated with metacognitive regulation and/or students’ self-rated English proficiency. The second-order CFA further provided empirical evidence to consider a common and broad motivational factor that can be inferred from the five subscales.","PeriodicalId":51446,"journal":{"name":"Journal of Psychoeducational Assessment","volume":"65 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-08-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135015773","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Adam B Lockwood, Kelsey Klatka, Brandon Parker, Nicholas Benson
{"title":"Administration and Scoring Errors on the Woodcock-Johnson IV Tests of Achievement: Before and During COVID-19.","authors":"Adam B Lockwood, Kelsey Klatka, Brandon Parker, Nicholas Benson","doi":"10.1177/07342829231166725","DOIUrl":"10.1177/07342829231166725","url":null,"abstract":"<p><p>Eighty Woodcock-Johnson IV Tests of Achievement protocols from 40 test administrators were examined to determine the types and frequencies of administration and scoring errors made. Non-critical errors (e.g., failure to record verbatim) were found on every protocol (<i>M</i> = 37.2). Critical (e.g., standard score, start point) errors were found on 98.8% of protocols (<i>M</i> = 15.3). Additionally, a series of paired samples t-test were conducted to determine differences in total, critical, and non-critical errors pre- and during-COVID-19. No statistic differences were found. Our findings add to a growing body of research that suggests that errors on norm-referenced tests of achievement are pervasive. However, the frequency of errors did not appear to be affected by COVID-19 stressors or social distancing requirements. Implications of these findings for training and practice are discussed. Suggestions for future research are also provided.</p>","PeriodicalId":51446,"journal":{"name":"Journal of Psychoeducational Assessment","volume":"41 1","pages":"501-513"},"PeriodicalIF":1.5,"publicationDate":"2023-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10080173/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45909087","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jinyan Huang, Yaxin Dong, Chun-yan Han, Xiaojun Wang
{"title":"Evaluating the Language- and Culture-Related Construct-Irrelevant Variance and Reliability of the Sense of School Belonging Scale: Suggestions for Revision","authors":"Jinyan Huang, Yaxin Dong, Chun-yan Han, Xiaojun Wang","doi":"10.1177/07342829231189319","DOIUrl":"https://doi.org/10.1177/07342829231189319","url":null,"abstract":"Using expert reviews and item response theory (IRT), this study evaluated the language- and culture-related construct-irrelevant variance and reliability of the 2019 TIMSS sense of school belonging scale (SSBS) for grades 4 and 8. The five items of the SSBS, which were identical for both grades, were reviewed for the language- and culture-related construct-irrelevant variance evidence by 10 assessment experts who have expertise in Chinese and English languages and cultures. Further, the existing TIMSS 2019 SSBS data associated with the countries of China (Hong Kong), Singapore, Australia, and America were analyzed within the framework of IRT for estimating the item and scale information function values (i.e., reliability) of the SSBS. The experts did find language- and culture-related evidence in each of the five items that might lead to the construct variance of the SSBS across these four countries. The IRT results further indicated that the SSBS item and scale information function values varied considerably across these four countries. Suggestions for revising the SSBS are proposed for TIMSS program developers.","PeriodicalId":51446,"journal":{"name":"Journal of Psychoeducational Assessment","volume":" ","pages":""},"PeriodicalIF":1.7,"publicationDate":"2023-07-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42285042","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"School Connectedness: Examining Measurement Invariance by Gender and Grade","authors":"Anqi Peng, Meagan M. Patterson, Paula J. Fite","doi":"10.1177/07342829231191161","DOIUrl":"https://doi.org/10.1177/07342829231191161","url":null,"abstract":"School connectedness plays a critical role in multiple aspects of child development. The purpose of this study was to operationalize school connectedness as a multidimensional construct and further assess whether this construct was measured similarly across gender and grade level among a sample of upper elementary school children (age M = 9.57; N = 361) in the United States. Based on prior scholarship, we identified and validated three key elements of school connectedness: school attachment, teacher attachment, and school safety. The results of multigroup confirmatory factor analysis showed that these three elements were measured comparably across gender and grade (Grades 3, 4, and 5). As measurement invariances by gender and grade were supported, we further examined differences in latent means among these groups and found that there were no differences in the three elements of school connectedness by gender; fourth graders had lower school attachment than third graders.","PeriodicalId":51446,"journal":{"name":"Journal of Psychoeducational Assessment","volume":" ","pages":""},"PeriodicalIF":1.7,"publicationDate":"2023-07-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49161370","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}