Dual-model synergy for audit opinion prediction: A collaborative LLM agent framework approach

By combining the capabilities of Moonshot and DeepSeek-R1, which respectively evaluate risk scores based on MD&A text information and financial data, we exploit the complementary strengths of long-context and reasoning large language models (LLMs) to help evaluate material misstatement risks in audit opinions. Our results suggest that both MD&A-based and financial-based risk evaluations effectively distinguish qualified and unqualified audit opinions, but combining them yields the best performance. In addition to the interpretative analysis text output, the combined LLM evaluation consistently outperforms logistic regression prediction that incorporates all indicators, and it achieves comparable performance compared to the sophisticated machine learning methods like gradient boosting regression and random forests.

Further analysis reveals that LLMs excel in high-risk scenarios: in firms with 1) high financial constraints, 2) low internal controls, 3) low audit quality, 4) low readability, or 5) negative tone of MD&A texts. A topic model analysis has shown clear difference in the MD&A emphasis for firms the qualified and unqualified opinions given by our framework. These findings shed light on the potential role of the collaborative LLM agent framework as a tool to help auditors and investors detect financial fraud.