Assessing the efficacy of ChatGPT in addressing Chinese financial conundrums: An in-depth comparative analysis of human and AI-generated responses

Abstract

ChatGPT, the latest iteration of OpenAI's natural language generation model, has found applications in a wide range of tasks such as question answering, text summarization, machine translation, classification, code generation, and dialogue A.I. Its potential in the financial industry has garnered significant attention. This paper aims to bridge the gap between chatGPT and human services in the financial domain, while also exploring the opportunities and challenges it presents in this industry. To comprehensively evaluate the processing capabilities of chatGPT in the financial field, we collected a dataset of n = 7165 financial questions and analyzed the perplexity value, emotion value, accuracy, professionalism, and real-time performance of both human-generated and chatGPT-generated content using machine learning algorithms and evaluation tests. The experimental results indicate that chatGPT exhibits higher levels of professionalism and accuracy compared to manual services, leading to improved efficiency, cost reduction, and enhanced customer satisfaction, thereby boosting the competitiveness and profitability of financial institutions. However, challenges such as a lack of emotional value in its responses, potential bias from one-sided training data, information errors, and the risk of job displacement need to be addressed. These findings provide theoretical and data-driven support for the future implementation of chatGPT in financial innovation and development.