Shaping the fairness journey: The roles of AI literacy, explanation, and interpersonal interaction in AI interviews

Abstract

Grounded in organizational justice theory, this two-study investigation provides a comprehensive examination of fairness perceptions across the entire AI interviews process. Through a three-stage experimental design (Study 1, N = 113; Study 2, N = 206), we explored how design and outcome factors influence procedural and distributive justice. We manipulated the AI’s explanation (With vs. Without) and level of interpersonal interaction (High vs. low) during the interview process, and the interview result (Pass vs. Fail) and decision agent (100% AI vs. 50% AI + 50% Human) in the post-decision stage. Results indicate that while candidate AI literacy, human-in-the-loop decision-making, and positive outcomes consistently improved fairness perceptions, the effects of system design were more complex. Design features intended to enhance the user experience, such as high AI interactivity and detailed explanations, improved aspects of procedural justice during the interview. Yet, these sometimes backfired by diminishing the perceived distributive justice of the final decision. This reveals a critical tension between a positive process experience and a fair outcome evaluation. These complex effects underscore the practical need for a holistic design approach that manages the entire candidate journey. Fair AI systems require not only improving candidate literacy but also carefully designing system explanations to manage applicant expectations effectively.