User judgment of an AI model is biased by its description: A study in a job interview training context

Abstract

The growth of artificial intelligence (AI) has introduced new AI-powered systems into many aspects of everyday life. Although many such systems may be embedded in contexts where long-term use is justified, there are also many cases where usage of such systems can be brief or limited to a single session. In those cases, initial information provided to the user about the AI model is important since users may not have enough engagement with the system to develop a mental model over time through use. Instead, users may simply rely on first impressions. However, little is known about how given information about the AI model in a system affects user judgment of the system. This work investigates this question within the context of job interview training. We conducted a controlled experiment where the description of the AI model within a simulated job interview training system was manipulated to describe the model as being either basic or more advanced. Participants in the condition where the AI model was described as more sophisticated and advanced reported significantly higher levels of agreement with the model outputs, more favorable ratings, and a greater willingness to use the system output.