CMedRAGBot: A Chinese Medical Chatbot Based on Graph RAG and Large Language Models.

Abstract

In the domain of Chinese clinical medical question-answering (QA), traditional Large Language Models (LLMs) encounter challenges such as hallucinations and difficulties in updating knowledge for knowledge-intensive tasks. To address these issues, this research presents a Chinese clinical medical QA model that integrates Retrieval-Augmented Generation (RAG) and a medical knowledge graph, named CMedRAGBot. First, a Chinese medical knowledge graph encompassing multiple entity types-including diseases, medications, and symptoms-is constructed. Based on this knowledge graph, a Named Entity Recognition (NER) model built on a Chinese-RoBERTa and BiGRU architecture is designed, with data augmentation strategies employed to enhance its generalization capability. In addition, prompt engineering techniques are used to implement intent recognition for user queries, mapping them to predefined intent categories. Finally, the aforementioned modules are integrated to form a complete Chinese clinical medical QA system. In the experimental evaluation, CMedRAGBot is deployed on five state-of-the-art LLMs (including ChatGPT-4o, ChatGPT-o1, DeepSeek-R1, Llama-3.3-70B-Instruct, and Gemini 2.0 Flash) and tested using specialized question banks derived from the Chinese Clinical Medical Qualification Examinations and Residency Standardization Training Examinations from 2000 to 2023. The results indicate that the integration of CMedRAGBot significantly improves the test accuracy of all models, with increases of up to approximately 10%. Furthermore, ablation experiments reveal that data augmentation enhances NER model's F1 score from 95.27% to 97.55%, while the inclusion of an intent recognition module markedly improves the model's ability to understand complex queries, thereby further boosting answer accuracy. Source code of the research is available at https://github.com/zhdongfang/CMedRAGBot .