Pui‐wa Lei, Hui Zhao, S. Hart, Xinyue Li, J. DiPerna
{"title":"Examination of Psychometric Evidence for Criterion-Referenced Scores from the SSIS SEL Brief Scales","authors":"Pui‐wa Lei, Hui Zhao, S. Hart, Xinyue Li, J. DiPerna","doi":"10.1177/07342829221144717","DOIUrl":"https://doi.org/10.1177/07342829221144717","url":null,"abstract":"Efficient and intuitive interpretive frameworks for social-emotional learning (SEL) measures are necessary for identifying student needs and informing programming decisions across multitiered systems of support in schools. Though familiar to educators and often used with standardized tests of academic achievement, criterion-referenced frameworks are less common in SEL assessment. As such, the current study examined the psychometric evidence for scores from one such framework, the Competency-Referenced Performance Framework, which was developed to inform universal screening decisions based on the SSIS SEL Brief Scales (Elliott et al., 2020). Specifically, we evaluated stability, test-criterion relationships with academic outcomes, and treatment sensitivity of the CRPF using data from an efficacy trial of a universal SEL program. Results provided preliminary supportive evidence for the CRPF.","PeriodicalId":51446,"journal":{"name":"Journal of Psychoeducational Assessment","volume":"41 1","pages":"311 - 327"},"PeriodicalIF":1.7,"publicationDate":"2022-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48959870","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
F. Peixoto, J. Radišić, Ksenija Krstić, Kajsa Yang Hansen, A. Laine, A. Baucal, Maarja Sõrmus, Lourdes Mata
{"title":"Contribution to the Validation of the Expectancy-Value Scale for Primary School Students","authors":"F. Peixoto, J. Radišić, Ksenija Krstić, Kajsa Yang Hansen, A. Laine, A. Baucal, Maarja Sõrmus, Lourdes Mata","doi":"10.1177/07342829221144868","DOIUrl":"https://doi.org/10.1177/07342829221144868","url":null,"abstract":"Grounded in ‘expectancy-value’ theory, this paper reports on the psychometric properties of an instrument intended to measure students’ motivation in mathematics. The participants were 2045 third-, fourth- and fifth-grade students from Estonia, Finland, Norway, Portugal, Serbia and Sweden. The Expectancy-Value Scale (EVS) was found to be suitable for early grades of primary education in measuring competence self-perceptions and subjective task values relative to the mathematics field. The results indicate a good model fit aligned with the expectancy-value theory. The EVS dimensions showed good reliability, and scalar invariance was established. However, findings also indicated high correlations between some of the EVS dimensions, which is well documented for students at this age. The findings are discussed relative to the ‘expectancy-value’ theory framework and students’ age.","PeriodicalId":51446,"journal":{"name":"Journal of Psychoeducational Assessment","volume":"41 1","pages":"343 - 350"},"PeriodicalIF":1.7,"publicationDate":"2022-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44938063","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Janina Eberhart, Andrew E. Koepp, S. Howard, R. Kok, D. McCoy, Sara T. Baker
{"title":"Advancing Educational Research on Children’s Self-Regulation With Observational Measures","authors":"Janina Eberhart, Andrew E. Koepp, S. Howard, R. Kok, D. McCoy, Sara T. Baker","doi":"10.1177/07342829221143208","DOIUrl":"https://doi.org/10.1177/07342829221143208","url":null,"abstract":"Self-regulation is crucial for children’s development and learning. Almost by convention, it is assumed that self-regulation is a relatively stable skill, and little is known about its dynamic nature and context dependency. Traditional measurement approaches such as single direct assessments and adult reports are not well suited to address questions around variations of self-regulation within individuals and influences from social-contextual factors. Measures relying on child observations are uniquely positioned to address these questions and to advance the field by shedding light on self-regulatory variability and incremental growth. In this paper, we review traditional measurement approaches (direct assessments and adult reports) and recently developed observational measures. We discuss which questions observational measures are best suited to address and why traditional measurement approaches fall short. Finally, we share lessons learned based on our experiences using child observations in educational settings and discuss how measurement approaches should be carefully aligned to the research questions.","PeriodicalId":51446,"journal":{"name":"Journal of Psychoeducational Assessment","volume":"41 1","pages":"267 - 282"},"PeriodicalIF":1.7,"publicationDate":"2022-12-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48486257","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yagmur Seven, R. Dedrick, Keri M. Madsen, T. Spencer, E. Kelley, Howard Goldstein
{"title":"Psychometric Properties of a Preschool Language, Literacy, and Behavior Screener","authors":"Yagmur Seven, R. Dedrick, Keri M. Madsen, T. Spencer, E. Kelley, Howard Goldstein","doi":"10.1177/07342829221143417","DOIUrl":"https://doi.org/10.1177/07342829221143417","url":null,"abstract":"This study investigated the psychometric properties of the Preschool Language, Literacy, and Behavior Screener (PLLB-S). We examined and tested the factor structure of the PLLB-S using exploratory and confirmatory factor analyses. We further conducted internal consistency, concurrent validity, and predictive validity analyses and evaluated teacher satisfaction using PLLB-S. Our factor analyses resulted in 22 items distributed among three subscales with high internal consistency: Oral language, emergent literacy, and behavior skills. The PLLB-S and its subscales correlated moderately to strongly with standardized measures. The emergent literacy of the PLLB-S was the only subscale that significantly predicted children’s later vocabulary knowledge. Preschool teachers reported high satisfaction with the content and purpose of the questionnaire. We concluded that this tool with sound psychometric properties can potentially help increase the feasibility and efficiency of implementing standardized assessments in MTSS frameworks in preschool classrooms.","PeriodicalId":51446,"journal":{"name":"Journal of Psychoeducational Assessment","volume":"41 1","pages":"283 - 297"},"PeriodicalIF":1.7,"publicationDate":"2022-12-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42364179","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Development and Internal Validity of the Student Opinion Scale: A Measure of Test-Taking Motivation","authors":"D. Pastor, Chris R. Patterson, S. Finney","doi":"10.1177/07342829221140957","DOIUrl":"https://doi.org/10.1177/07342829221140957","url":null,"abstract":"In low-stakes testing contexts, there are minimal personal consequences associated with examinee performance. Examples include assessments administered for research, program evaluation, test development, and international comparisons (e.g., Programme for International Student Assessment [PISA]). Because test-taking motivation can suffer in low-stakes conditions, the Student Opinion Scale (SOS) is commonly administered to measure test-taking effort and how personally important the examinee feels it is to do well on the test. Although popular, studies of the scale’s internal validity yield conflicting findings. The present study critically evaluates the creation of the SOS and considers its factor structure across six samples of college students differing in their college experience level and version of the SOS administered. Because findings only support the internal validity of the effort subscale, further study and development of the importance subscale is recommended.","PeriodicalId":51446,"journal":{"name":"Journal of Psychoeducational Assessment","volume":"41 1","pages":"209 - 225"},"PeriodicalIF":1.7,"publicationDate":"2022-11-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41831774","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jorge J. Varela, R. Melipillán, Amy L. Reschly, Ana Maria Squicciarini Navarro, Felipe Peña Quintanilla, Paola Sánchez Campos
{"title":"Cross-Cultural Validation of the Student Engagement Instrument for Chilean Students","authors":"Jorge J. Varela, R. Melipillán, Amy L. Reschly, Ana Maria Squicciarini Navarro, Felipe Peña Quintanilla, Paola Sánchez Campos","doi":"10.1177/07342829221141512","DOIUrl":"https://doi.org/10.1177/07342829221141512","url":null,"abstract":"Student engagement is associated with various aspects of students’ school experiences, including student achievement, high school completion, and post-secondary success. As measurement of student engagement has grown in countries around the world, few studies have been conducted in South America. This study examined a translated version of the Student Engagement Instrument, widely used in the U.S. and other countries, in a study of 2337 adolescents in Chile. Consistent with prior research, confirmatory factor analyses revealed a six-factor solution as the best fit for the data. However, fewer items were retained than on the studies of the SEI with students in the U.S. The Future Goals and Aspirations and Extrinsic motivation subscales were associated, in expected directions, with achievement 1 year later.","PeriodicalId":51446,"journal":{"name":"Journal of Psychoeducational Assessment","volume":"41 1","pages":"226 - 233"},"PeriodicalIF":1.7,"publicationDate":"2022-11-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44414403","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The Effects of Changing Negatively Worded Items to Positively Worded Items on the Reliability and the Factor Structure of Psychological Scales","authors":"H. Dodeen","doi":"10.1177/07342829221141934","DOIUrl":"https://doi.org/10.1177/07342829221141934","url":null,"abstract":"In survey measurement, acquiescence bias is a response effect that occurs when respondents agree to the item or the question in the scale regardless of its content. It is assumed that negative items force participants not to agree with some items. Using the mixture approach, however, is not without a substantial cost on both the structure and the scale psychometric properties. The effects of including negative items in scales is what this study tried to investigate. Therefore, the aim of the study is to empirically evaluate the effects of changing negative items to their equivalent positively worded items on the reliability and the factor structure of psychological scales. It is hypothesized that this approach improves the scale factors structures and reliability. Seven commonly used psychological scales that have both negatively and positively worded items have been selected. The scales were applied on seven different samples with a total number of 4192 participants from a public university in the United Arab Emirates. The results confirmed that changing negative items to their equivalent positively directed items systematically and significantly increased the reliability values as well as improved the factor structure of psychological scales.","PeriodicalId":51446,"journal":{"name":"Journal of Psychoeducational Assessment","volume":"41 1","pages":"298 - 310"},"PeriodicalIF":1.7,"publicationDate":"2022-11-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45346886","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"How Grit-S Items Work in Measuring Growth: Focusing on Item Discrimination and Sensitivity to Change","authors":"H. Lim, Sora Lee","doi":"10.1177/07342829221139344","DOIUrl":"https://doi.org/10.1177/07342829221139344","url":null,"abstract":"This study investigates how sensitively the current grit measure (Grit-S) is able to reflect the differences between individuals (i.e. item discrimination) and change within individuals (i.e. sensitivity to change of items) using longitudinal data. Data from the Korean Children and Youth Panel Study 2018 of elementary (fourth grade) and middle school (seventh grade) cohorts were analyzed. We compared a series of longitudinal factor models of change with an initial trait and a change in grit represented by latent traits. Results showed that the models hypothesizing two latent traits with respect to the two elements of grit (interest and effort) best fit the data. In the elementary school cohort, sensitivity to change was different from discriminating between-person differences at baseline. Meanwhile, in the middle school cohort, sensitivity to change was identical to discriminating between-person differences at baseline. The items having low discrimination and low sensitivity to change resulted in syntactical complexity and did not accurately reflect the trait that was intended to be measured. Despite the Grit-S being widely used for many years, it is suggested that researchers and educators should administer this measure at taking an individual’s change and school-level difference into account.","PeriodicalId":51446,"journal":{"name":"Journal of Psychoeducational Assessment","volume":"41 1","pages":"175 - 193"},"PeriodicalIF":1.7,"publicationDate":"2022-11-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47254195","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
David A. Klingbeil, Ethan R. Van Norman, David J. Osman, Kimberly Berry-Corie, Caroline K. Carberry, Jessica S. Kim
{"title":"Comparing AimswebPlus to the Benchmark Assessment System for Universal Screening in Upper Elementary Grades","authors":"David A. Klingbeil, Ethan R. Van Norman, David J. Osman, Kimberly Berry-Corie, Caroline K. Carberry, Jessica S. Kim","doi":"10.1177/07342829221139520","DOIUrl":"https://doi.org/10.1177/07342829221139520","url":null,"abstract":"Early identification of students needing additional support is a foundational component of Multi-Tiered Systems of Support (MTSS). Due to the resource-intensive nature of implementing MTSS, it is critical that universal screening procedures are maximally accurate and efficient. The purpose of this study was to compare the classification accuracy of aimswebPlus reading scores to the Benchmark Assessment System scores. We used data from a mid-size city in Texas to retrospectively compare the classification accuracy between fall aimswebPlus reading composites to the Benchmark Assessment System scores when predicting student performance on the statewide reading test. When classification decisions were made based on the vendor-recommended cut-scores, both measures were insufficiently sensitive for screening in MTSS. Following aimswebPlus’ recommended method for establishing local-cut scores improved the sensitivity of decisions, but the specificity values were well below minimally acceptable levels. Limitations, directions for future research, and implications for practice are discussed.","PeriodicalId":51446,"journal":{"name":"Journal of Psychoeducational Assessment","volume":"41 1","pages":"194 - 208"},"PeriodicalIF":1.7,"publicationDate":"2022-11-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42607340","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The Anti-Mattering Scale Versus the General Mattering Scale in Pathological Narcissism: How an Excessive Need to Matter Informs the Narcissism and Mattering Constructs","authors":"G. Flett, Taryn Nepon, Xena Scott","doi":"10.1177/07342829221136352","DOIUrl":"https://doi.org/10.1177/07342829221136352","url":null,"abstract":"The current study uniquely examines mattering and narcissism and reflects our contention that pathological narcissism involves an excessive need to matter and a hypersensitivity to being devalued and not mattering to other people. Specifically, we evaluated the proposed association between pathological narcissism and deficits in mattering by comparing the results obtained with the Anti-Mattering Scale and the General Mattering Scale. We also evaluated anti-mattering as a potential mediator of the link between narcissism and distress. A sample of 168 university students completed the Anti-Mattering Scale, the General Mattering Scale, the Pathological Narcissism Inventory, and a depression measure. Results confirmed that elevated scores on the Anti-Mattering Scale are associated with grandiose and vulnerable narcissism as well as depression. Mattering assessed by the General Mattering Scale had a weaker association with narcissism, thus highlighting the distinction between the Anti-Mattering Scale and the General Mattering Scale. Further analyses suggested that elevated Anti-Mattering Scale scores did indeed mediate the link between vulnerable narcissism and depression in keeping with anti-mattering as a factor that elicits the vulnerability of narcissists. Our findings attest to the uniqueness of the Anti-Mattering Scale and illustrate the need to consider the role of feelings of not mattering as a contributor to the self and identity issues and interpersonal sensitivity that contribute to pathological narcissism. This work also suggests the need to emphasize an excessive need to matter when assessing the self and when developing future measures of the need to matter.","PeriodicalId":51446,"journal":{"name":"Journal of Psychoeducational Assessment","volume":"41 1","pages":"619 - 633"},"PeriodicalIF":1.7,"publicationDate":"2022-10-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43356643","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}