Echoes of Bias: An Analysis of ChatGPT in Financial Planner–Client Dialogs

Abstract

This study examines how the ChatGPT Model 3.5, a large language model, exhibits implicit bias when generating financial planning communications with varying racial identifiers. Using a structured testing framework with 25 combinations of advisor–client racial identifiers, we analyzed AI-generated emails explaining investment diversification. Through content and discourse analysis informed by Critical Algorithm Studies, we found that while core financial advice remained consistent, subtle linguistic variations emerged based on racial identifiers. These variations manifested primarily as unconscious adjustments in tone, cultural references, and language choice rather than substantive differences in financial guidance. Drawing on recent research in AI bias, we introduce a novel 2 × 2 matrix categorizing AI biases along dimensions of explicitness and intentionality. Our findings suggest that even in professional contexts, AI systems may reflect societal patterns encoded in their training data, potentially influencing advisor–client communications. As financial planners increasingly adopt AI tools for client communications and administrative tasks, understanding these subtle biases becomes crucial for maintaining professional standards and fiduciary responsibilities. This research contributes to the growing literature on AI applications in financial planning while highlighting important considerations for practitioners using AI-powered tools in their practice.