评级方差的不确定性和小尺寸效应对监理评级可靠性的影响

IF 2.9 4区心理学 Q2 PSYCHOLOGY, APPLIED

Human Performance Pub Date : 2022-08-08 DOI:10.1080/08959285.2022.2111433

D. Jackson, G. Michaelides, Christopher Dewberry, Amanda Jones, S. Toms, Benjamin Schwencke, Wei-Ning Yang

{"title":"评级方差的不确定性和小尺寸效应对监理评级可靠性的影响","authors":"D. Jackson, G. Michaelides, Christopher Dewberry, Amanda Jones, S. Toms, Benjamin Schwencke, Wei-Ning Yang","doi":"10.1080/08959285.2022.2111433","DOIUrl":null,"url":null,"abstract":"ABSTRACT We modeled the effects commonly described as defining the measurement structure of supervisor performance ratings. In doing so, we contribute to different theoretical perspectives, including components of the multifactor and mediated models of performance ratings. Across two reanalyzed samples (Sample 1, N ratees = 392, N raters = 244; Sample 2, N ratees = 342, N raters = 397), we found a structure primarily reflective of general (>27% of variance explained) and rater-related (>49%) effects, with relatively small performance dimension effects (between 1% and 11%). We drew on findings from the assessment center literature to approximate the proportion of rater variance that might theoretically contribute to reliability in performance ratings. We found that even moderate contributions of rater-related variance to reliability resulted in a sizable impact on reliability estimates, drawing them closer to accepted criteria.","PeriodicalId":47825,"journal":{"name":"Human Performance","volume":null,"pages":null},"PeriodicalIF":2.9000,"publicationDate":"2022-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Uncertainty about Rater Variance and Small Dimension Effects Impact Reliability in Supervisor Ratings\",\"authors\":\"D. Jackson, G. Michaelides, Christopher Dewberry, Amanda Jones, S. Toms, Benjamin Schwencke, Wei-Ning Yang\",\"doi\":\"10.1080/08959285.2022.2111433\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"ABSTRACT We modeled the effects commonly described as defining the measurement structure of supervisor performance ratings. In doing so, we contribute to different theoretical perspectives, including components of the multifactor and mediated models of performance ratings. Across two reanalyzed samples (Sample 1, N ratees = 392, N raters = 244; Sample 2, N ratees = 342, N raters = 397), we found a structure primarily reflective of general (>27% of variance explained) and rater-related (>49%) effects, with relatively small performance dimension effects (between 1% and 11%). We drew on findings from the assessment center literature to approximate the proportion of rater variance that might theoretically contribute to reliability in performance ratings. We found that even moderate contributions of rater-related variance to reliability resulted in a sizable impact on reliability estimates, drawing them closer to accepted criteria.\",\"PeriodicalId\":47825,\"journal\":{\"name\":\"Human Performance\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":2.9000,\"publicationDate\":\"2022-08-08\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Human Performance\",\"FirstCategoryId\":\"102\",\"ListUrlMain\":\"https://doi.org/10.1080/08959285.2022.2111433\",\"RegionNum\":4,\"RegionCategory\":\"心理学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"PSYCHOLOGY, APPLIED\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Human Performance","FirstCategoryId":"102","ListUrlMain":"https://doi.org/10.1080/08959285.2022.2111433","RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"PSYCHOLOGY, APPLIED","Score":null,"Total":0}

引用次数: 0

摘要

摘要：我们对通常被描述为定义主管绩效评级的衡量结构的影响进行了建模。在这样做的过程中，我们对不同的理论观点做出了贡献，包括绩效评级的多因素和中介模型的组成部分。在两个重新分析的样本中（样本1，N个评分者=392，N个评价者=244；样本2，N个评级者=342，N个评等者=397），我们发现一个结构主要反映了一般（解释的方差的27%以上）和评分者相关（>49%）的影响，表现维度的影响相对较小（在1%到11%之间）。我们利用评估中心文献中的发现来近似评分者方差的比例，这在理论上可能有助于绩效评分的可靠性。我们发现，即使评分者相关方差对可靠性的适度贡献也会对可靠性估计产生相当大的影响，使其更接近公认的标准。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Uncertainty about Rater Variance and Small Dimension Effects Impact Reliability in Supervisor Ratings

ABSTRACT We modeled the effects commonly described as defining the measurement structure of supervisor performance ratings. In doing so, we contribute to different theoretical perspectives, including components of the multifactor and mediated models of performance ratings. Across two reanalyzed samples (Sample 1, N ratees = 392, N raters = 244; Sample 2, N ratees = 342, N raters = 397), we found a structure primarily reflective of general (>27% of variance explained) and rater-related (>49%) effects, with relatively small performance dimension effects (between 1% and 11%). We drew on findings from the assessment center literature to approximate the proportion of rater variance that might theoretically contribute to reliability in performance ratings. We found that even moderate contributions of rater-related variance to reliability resulted in a sizable impact on reliability estimates, drawing them closer to accepted criteria.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

Human Performance PSYCHOLOGY, APPLIED-

CiteScore

4.30

自引率

0.00%

发文量

期刊介绍： Human Performance publishes research investigating the nature and role of performance in the workplace and in organizational settings and offers a rich variety of information going beyond the study of traditional job behavior. Dedicated to presenting original research, theory, and measurement methods, the journal investigates individual, team, and firm level performance factors that influence work and organizational effectiveness. Human Performance is a respected forum for behavioral scientists interested in variables that motivate and promote high-level human performance, particularly in organizational and occupational settings. The journal seeks to identify and stimulate relevant research, communication, and theory concerning human capabilities and effectiveness. It serves as a valuable intellectual link between such disciplines as industrial-organizational psychology, individual differences, work physiology, organizational behavior, human resource management, and human factors.