ChatExosome: An Artificial Intelligence (AI) Agent Based on Deep Learning of Exosomes Spectroscopy for Hepatocellular Carcinoma (HCC) Diagnosis

Large language models (LLMs) hold significant promise in the field of medical diagnosis. There are still many challenges in the direct diagnosis of hepatocellular carcinoma (HCC). α-Fetoprotein (AFP) is a commonly used tumor marker for liver cancer. However, relying on AFP can result in missed diagnoses of HCC. We developed an artificial intelligence (AI) agent centered on LLMs, named ChatExosome, which created an interactive and convenient system for clinical spectroscopic analysis and diagnosis. ChatExosome consists of two main components: the first is the deep learning of the Raman fingerprinting of exosomes derived from HCC. Based on a patch-based 1D self-attention mechanism and downsampling, the feature fusion transformer (FFT) was designed to process the Raman spectra of exosomes. It achieved accuracies of 95.8% for cell-derived exosomes and 94.1% for 165 clinical samples, respectively. The second component is the interactive chat agent based on LLM. The retrieval-augmented generation (RAG) method was utilized to enhance the knowledge related to exosomes. Overall, LLM serves as the core of this interactive system, which is capable of identifying users’ intentions and invoking the appropriate plugins to process the Raman data of exosomes. This is the first AI agent focusing on exosome spectroscopy and diagnosis, enhancing the interpretability of classification results, enabling physicians to leverage cutting-edge medical research and artificial intelligence techniques to optimize medical decision-making processes, and it shows great potential in intelligent diagnosis.