You, Me, and the AI: The Role of Third-Party Human Teammates for Trust Formation Toward AI Teammates

Abstract

Artificial intelligence (AI)'s role in organizations has transitioned fundamentally during the “fourth industrial revolution” (Schwab 2017). Initially a mere tool for executing orders, AI is now implemented as an active teammate working along humans (Ulfert et al. 2024). In particular, organizations commonly install human-AI teams to obtain high performance and achieve their goals (Kanarik et al. 2023; Seeber et al. 2020).¹ With increasing AI capabilities, such human-AI teams already serve customers, perform tasks ranging from manufacturing to creative brainstorming, and even produce scientific outputs (Boiko et al. 2023; Evjemo et al. 2020; Ivanov and Webster 2019). As AI's role shifts from tool to teammate, it fundamentally alters employee-AI interactions, with novel forms of AI exhibiting greater autonomy, proactivity, and interdependence compared to traditional AI tools (Chugunova and Sele 2022). Yet, despite increasing adoption of AI in teams, employees often fail to effectively interact with AI (Dwivedi et al. 2021). For successful collaboration with AI, it is critical that employees perceive AI as trustworthy (i.e., view it as reliable and integer; cf. Whitener et al. 1998) and demonstrate trust behavior toward it (i.e., make themselves vulnerable to the AI's actions; cf. Mayer, Davis, and Schoorman 1995).²

Forming trust is crucial for shaping interactions with AI, influencing perceptions of AI usefulness and performance, and ultimately efficient usage (Choung, David, and Ross 2023; Lee and See 2004). Consequently, researchers are increasingly interested in understanding the factors that drive trust formation in AI within teams and organizational contexts (Glikson and Woolley 2020; Schelble et al. 2024). Emerging research on trust formation in AI suggests that its antecedents can be grouped into three categories, namely characteristics of (1) the human trustor, (2) the AI trustee, and (3) the context or environment (Hancock et al. 2011; Kaplan et al. 2023). Existing work has primarily concentrated on the first two categories, claiming that “factors specific to each of the human and automation are clearly central in trust development” (Schaefer et al. 2016, p. 385). Regarding human trustors, scholars mainly focused on their propensity to trust (Merritt and Ilgen 2008) and demographics including nationality, gender, and age (Hoff and Bashir 2015; Schaefer et al. 2016). For AI trustees, research has shown that characteristics such as its reliability and anthropomorphism (i.e., human-likeness, Epley, Waytz, and Cacioppo 2007) shape trust in AI (Kaplan et al. 2023), although the influence of antecedents may hinge on the modality of the AI (i.e., whether the AI is embodied like a humanoid social robot or disembodied like AI software or chatbots; Glikson and Woolley 2020; Lyons, Hamdan, and Vo 2023).

Interestingly, the third category of factors—broadly defined as context or environment (Kaplan et al. 2023; Schaefer et al. 2016)—has received much less attention with a few studies examining task-related elements (e.g., task type, complexity, risk; Hancock et al. 2021) or broader, team-level factors (e.g., team composition or shared mental models; Hancock et al. 2011; McNeese et al. 2021). However, despite the growing interest in human-AI trust formation, most studies have focused on factors within human-AI dyads. Therefore, little is known about the impact of relevant third parties on trust formation in human-AI teams.

We aim to provide a deeper understanding of human-AI trust formation by focusing on the direct social environment in which such human-AI interactions occur. Human-AI teams seldom entail exclusive interaction between a single human and an AI. Instead, in most organizations, human-AI teams involve at least one other human teammate (O'Neill et al. 2022). Acknowledging this direct presence of a relevant party is particularly important for understanding trust formation, as human-human trust research has shown that individuals learn who is trustworthy and when to trust based on observations of and interactions with relevant colleagues (De Cremer et al. 2018).

However, due to treating human-AI interactions as operating within a dyadic vacuum, the existing literature is unable to speak to how the direct social environment—another human teammate—influences when humans perceive and treat AI as trustworthy. To address this critical oversight, we propose a socio-cognitive lens toward trust formation in human-AI teams. More specifically, we integrate social cognitive theory (SCT; Bandura 1986) with emerging theory and research on human-AI teams (Georganta and Ulfert 2024; Wildman et al. 2024) to propose a human teammate's trustworthiness perceptions and trust behaviors toward an AI as critical antecedents of a focal employee's trust attitudes and actions toward an AI.³ SCT points toward a dynamic interplay between an employee's environment, cognitive factors, and behavior shaping an individual's choices. Importantly, this framework further suggests that an employee's own perceptions and behaviors may determine how they respond to potential environmental influences (Bandura 1977, 1986). Applied to the present study, we argue that employees will not blindly adopt AI-related attitudes and behaviors from their environment. Instead, we argue that the relationship between a human teammate's trustworthiness perceptions of AI and the employee's perceptions of AI trustworthiness critically hinges on an employee's perceptions of their human teammate's trustworthiness. In the same vein, we propose that an employee's trust in the human teammate strengthens the linkage between their teammate's trust in AI and the employee's trust in this technology. In line with Mayer, Davis, and Schoorman's (1995) seminal model, we therefore focus on trustworthiness perceptions and trust behaviors as two key components of trust formation. Past research has shown that both of these are associated with unique antecedents (Ben-Ner and Halldorsson 2010) and, importantly, both differentially affect various outcomes (Colquitt, Scott, and LePine 2007; Lyu and Ferrin 2018; Zapata, Olsen, and Martins 2013). While these two constructs are interrelated (Hardin 2002), the emergence of (AI) trustworthiness perceptions does not necessarily translate into trust (toward AI) behaviors in practice (cf. Dietz and Den Hartog 2006; Lewis and Marsh 2022), creating a need to separately investigate both of these central parts of the trust formation in AI process.

We test our assumptions across two main studies that examine different aspects of the trust formation process across various types of AI modalities, offering a comprehensive view of third-party effects in human-AI teams. Study 1, an experimental study comprising individuals with work experience, focuses on the “belief component” of the trust formation process (cf. Jones and Shah 2016; Mayer, Davis, and Schoorman 1995; McKnight, Cummings, and Chervany 1998). In particular, it examines AI trustworthiness perceptions in the context of disembodied AI (Correia et al. 2020). Building on and extending these insights, Study 2 shifts to the “action component” of the trust formation process (Jones and Shah 2016; Lewicki, Tomlinson, and Gillespie 2006; Mayer, Davis, and Schoorman 1995) and examines trust behaviors toward AI in an incentivized observational study involving two humans and an embodied AI (i.e., a real AI-powered service robot). This set of studies allows us to explore third-party effects in a holistic manner, extending findings across two key parts of the trust formation process and two different, yet increasingly prevalent (Harris-Watson et al. 2023) modalities of AI.

Through this approach, we make several contributions to the research on human-AI teams. First, our work advances the literature on antecedents of trust formation in AI (Schaefer et al. 2016). Emerging research has started to acknowledge the impact of triadic (or larger) human-AI teaming (Schelble et al. 2023) and the relevance of third parties operating outside of the human-AI dyad (Cameron et al. 2023) for trust formation in AI. We extend this literature by putting a spotlight on the role of the direct social environment as a driving force for trust formation in human-AI teams. Examining both human teammate's AI trustworthiness perceptions and trust behavior toward AI, enables us to disentangle the distinct and nuanced ways through which human teammates influence an employee's cognitive assessments of AI trustworthiness and their actual willingness to rely on AI in consequential situations. Thus, our investigation heeds recent calls to investigate in more depth the complex social nuances of human-AI trust dynamics (Dirks and De Jong 2022; Huang et al. 2021). Second, the present study extends a key principle of SCT (Bandura 1986) to the human-AI trust realm by showing that an employee's own cognitions and behaviors toward their human teammate act as critical boundary conditions that shape the employee's responses to environmental influences. Therefore, our hypothesized three-party framework (i.e., the employee trustor, the AI trustee, and the third-party human teammate) allows for a finer-grained, theory-driven understanding of when employees form trust towards this new technology.

Lastly, this research adds novel insights for practitioners and policy-makers aiming to better understand trust formation in AI. Recent global surveys highlight that—despite impressive performance developments—low trust toward AI remains common (Edelman 2021; Gillespie et al. 2023). Organizations and governments may draw upon our new insights to overcome such prevalent worries by acknowledging the role of human teammates in successful human-AI interactions.

The complex trust process encompasses several stages that entail different antecedents, boundary conditions, and (downstream) consequences of trust (Hancock et al. 2023; Mayer, Davis, and Schoorman 1995). Regarding relevant trust variables, the trust literature distinguishes between trust propensity (i.e., a dispositional willingness to rely on others; Gill et al. 2005), trustworthiness perceptions (i.e., perceptions of a trustee being capable and integer; Alarcon et al. 2024), and trust behaviors (i.e., displays of vulnerability to a trustee; Colquitt, Scott, and LePine 2007). Given that the rather stable, personality-like nature of trust propensity makes it less susceptible to change (Alarcon et al. 2018), the present research focuses on trustworthiness perceptions and trust behaviors as the more malleable parts of trust formation. These two constructs are related such that trustworthiness perceptions may often predict trust behaviors (Mayer, Davis, and Schoorman 1995). Nonetheless, both constitute separate constructs that are predicted by different antecedents and are uniquely associated with different outcomes (Lyu and Ferrin 2018; Thielmann and Hilbig 2015), in particular within the realm of human-AI trustworthiness and trust (Lewis and Marsh 2022). Moreover, research commonly distinguishes between cognitive and affective trustworthiness and trust (Legood et al. 2023; Vignovic and Thompson 2010). Cognitive trust (worthiness) deals with “reliability and dependability” (McAllister 1995, p. 25), and often precedes affective trust (worthiness), which is “grounded in reciprocal interpersonal care and concern” (McAllister 1995, p. 25). The present research focuses on cognitive trust (worthiness) as, particularly in the context of human-AI teams, cognitive trust (worthiness) predicts team performance, technology acceptance, and AI usage more strongly than its affective counterpart (Castelo, Bos, and Lehmann 2019; Glikson and Woolley 2020).⁴

We implemented a multi-study design intending to holistically investigate the effects of third-party human teammates on different parts of the trust formation in AI process (i.e., trustworthiness perceptions vs. trust behaviors), across different modalities (embodied AI vs. disembodied AI; Mallick et al. 2023), and across different tasks and populations. In particular, in Study 1, we used an experimental vignette approach amongst a diverse set of individuals with work experience to examine third-party effects on trustworthiness perceptions toward a disembodied AI (i.e., AI chatbot) and provide causal support for such influence (cf. Podsakoff and Podsakoff 2019). Together with these findings, we report results from one additional supplementary study and robustness checks to replicate Study 1's findings and provide deeper insights into the observed result patterns. In Study 2, we tested the effects on trust behaviors in an ecologically valid setting. In particular, three-party teams consisting of an embodied AI (i.e., real, physical AI-powered robot) and two human participants (undergraduate and graduate business students) worked together in an incentivized observational study that included monetary trust decisions. As explained later in more detail, we therefore aimed to provide generalizable findings that can extend our hypothesized third-party effects across contexts.

Intelligent machines have arrived in modern workplaces (Parker and Grote 2022) and human-AI teams have the potential to improve the performance of organizations and their workforces (McNeese et al. 2018). Building on SCT (Bandura 1986), this research illustrates the critical role of employees' social environment, namely other human teammates, in shaping an employee's trustworthiness perceptions and trust toward AI teammates. In particular, our findings showed a positive relationship between a human teammate's perceptions of an AI's trustworthiness and an employee's own trustworthiness perceptions of the AI (Study 1) and between a human teammate's trust in an AI and an employee's trust behaviors toward AI (Study 2). Importantly, we further found initial evidence that perceived trustworthiness of the human teammate strengthens these relationships. In particular, employees (a) adopted their human teammates' perceptions about AI's trustworthiness and unexpectedly, (b) emulated the human teammate's trust behaviors toward AI when they perceive their human teammates as at least moderately trustworthy. However, we did not find support for a moderating effect of an employee's trust behaviors toward their human teammate for these linkages.

With the increasing prevalence of AI as a teammate, it is imperative to better understand the nature of the complex social dynamics in human-AI teams. Our research paints a novel, more nuanced, and finer-grained picture of how and when a key third party (i.e., another human teammate) influences trust formation in such teams. Our findings show that employees adopt their human teammate's perceptions about the AI teammate's trustworthiness and match their human teammate's trust behaviors toward the AI teammate. Importantly, these effects primarily occur when employees perceive their human teammates as at least moderately trustworthy. Our investigation of third-party effects suggests that trust formation in human-AI teams is more complex than previously understood, involving a delicate interplay between direct human-AI interactions and social influences from other human team members. In sum, by highlighting the importance of third parties beyond the human-AI dyad, our findings provide a foundation for a better understanding of the increasingly common situation of teams that consist of human and AI team members.

The authors declare no conflicts of interest.

The data that support the findings of this study are openly available in Open Science Framework at https://osf.io/qcvwx.