Getting from valid to useful: End user modifiability and human capital analytics implementation in selection

Abstract

A major problem in employee selection coalesces around convincing decision-makers (e.g., hiring managers) to use analytically derived models. Existing recommendations in the literature largely focus on convincing executives to adopt analytical models and then exert their top-down influence on lower-level hiring decisions. In contrast to these solutions, we explore end user modifiability (i.e., allowing decision-makers to modify a statistical model before use) as a bottom-up approach for increasing hiring managers' implementation of analytical recommendations. From a utility standpoint, we consider how incorporating end user modifiability into hiring decisions will result in a less statistically valid, but potentially more valuable, organizational selection process. We explore these ideas in two studies. In Study 1, we experimentally test whether model modification increases decision-maker reliance on a statistical model, as well as how much decision-makers need to modify a model in order to use it. In Study 2, we examine the extent that modifiability introduces implicit biases that might adversely affect marginalized groups. Results suggest that modifiability can increase decision-makers' perceived usefulness of a model and, importantly, that only a small amount of modifiability is needed to elicit this effect. Further, end user modifications were statistically insignificant predictors of hiring rates across race-based subgroups, though supplementary analyses suggest important cautionary nuance. Given that analytical models are rarely perfectly or wholly implemented, end user modifiability may offer a viable solution for organizations seeking to increase the implementation of algorithmic guidance in selection decisions, even if it deviates modestly from a statistical optimality.