Issues in Contrasting Conservative and Considered Estimation: A Reply to Bobko et al. (2024)

Abstract

Bobko et al. (2024) responded to Sackett et al.'s (2022) compilation of meta-analytic evidence for the validity of a wide variety of measures used as predictors of overall job performance, offering a set of alternative methodological choices which they term “considered estimation” to counter the Sackett et al. approach of “conservative estimation.” Here we offer a rebuttal to Bobko et al. A primary concern is that Bobko et al. apply the label “conservative estimation” to the full range of methodological choices made by Sackett et al. Yet, we clarify the narrow and specific meaning of “conservative estimation,” and note that that the bulk of Bobko et al.'s concerns are independent of the principle of conservative estimation. We also respond to Bobko et al.'s two key concerns, namely, comparing validity estimates when one is corrected for range restriction and one is not and comparing validity estimates for predictors reflecting psychological constructs and those reflecting measurement methods, and also briefly address a range of other critiques offered by Bobko et al.