Quid pro Quo: Information disclosure for AI feedback in Human-AI collaboration

Abstract

This paper explores the intersection of AI-driven teamwork facilitation and user privacy concerns in virtual collaboration. Specifically, we investigate the extent to which individuals in AI-supported teamwork environments are willing to disclose personal information to improve collaboration and performance. Leveraging a vignette-based study, we assess participants' intentions to disclose different different types of personal information, such as language and choice of words, talk times, body language or sentiments, during a professional virtual collaboration process. Thereby, conditions under which the information is disclosed vary, referring to whether the results of information disclosure are either shared individually or with the whole group and whether they are shared anonymously or non-anonymously. Drawing on privacy calculus theory, our investigation further incorporates crucial contextual factors—Trust in AI, Perceived Risks, and Perceived Benefits—to comprehensively examine their influence on individuals' intentions to disclose information. Our findings reveal notable differences in Intention to disclose across various Information types, with lower intentions observed particularly for emotion and attention level disclosures. Surprisingly, other manipulated factors, including the level of anonymity, do not show an effect in influencing disclosure intentions. Crucially, our study underscores the pivotal role of Trust in AI, emerging as a consistent predictor of Intention to disclose across all Information types. Furthermore, its impact on disclosure intentions is mediated by individuals’ perceptions of risks and benefits associated with disclosure. Our research contributes to the evolving field of privacy calculus theory by shedding light on the nuanced interplay between Trust, Perceived Risks, Benefits, and information disclosure to AI in teamwork scenarios. These insights bear implications for the effective deployment of AI in facilitating teamwork within the workplace, emphasizing the need for cultivating trust and understanding the specific sensitivities associated with different types of information.