Ideal solutions don’t necessarily inform reality

Abstract

We applaud the efforts of Sackett et al. (2023) to update research that is often uncritically cited and discussed in the field of I–O psychology, as well as their sensible recommendations and the overall thoughtfulness of their paper. However, like much published research in this area, we believe it falls short in its aim of informing selection practices in most organizations. Sackett et al.’s (2022) update to Schmidt and Hunter (1998) falls into the same trap as prior metaanalyses in that their statistically corrected results of aggregated studies designed for validating selection measures and subsequent recommendations seem to assume an ideal (perhaps even an imaginary) world. Specifically, as with nearly all academic articles and textbooks on selection, their results are framed in a context where organizations have copious amounts of financial resources, extended periods of time, access to technology, and readily available applicant pools with numerous applicants that vary substantially in terms of their job-relevant characteristics (e.g., personality, abilities, interests, etc.). However, HR practitioners are more frequently faced with limited resources, limited time to make hires, limited available technology (and experience using it), and severely limited applicant pools. Moreover, these limitations are likely to shift over time with changes in the ambient economic climate, the strategic priorities of organizational leadership, the success and reputation of their organization, and technological progress. So, when confronted with optimized models based on corrected estimates making recommendations about best practices in selection, we find ourselves asking not only how an HR practitioner is supposed to make use of this information but whether they should consider it at all. Although such articles are useful for providing hypothetical benchmarks when academics seek to inform practitioner choices, we should not only consider what selection would look like in ideal circumstances but also what is possible in a reality-constrained world. It is our position that I–O psychology would be better served if we study how selection works in practice and try to meet practitioners halfway in terms of making our recommendations reflect the limitations HR practitioners face.1 We freely admit that we don’t have ready-made solution to these real-world limitations and problems, but we hope that this comment can serve as a foundation for a more practice-oriented stream of research in I–O psychology. Therefore, the following represent an incomplete list of issues and considerations that I–O academics may want to address in future research concerning best practices in selection.