Can LLMs revolutionize text mining in chemistry? A comparative study with domain-specific tools

Abstract

The exponential growth of chemical literature necessitates advanced tools for efficient data extraction and utilization. This study investigates the performance of Large Language Models (LLMs) in Chemical Named Entity Recognition (CNER), comparing them against traditional domain-specific tools. We fine-tuned the LLaMA-2 model using the NLM-Chem corpus and integrated a Retrieval-Augmented Generation (RAG) pipeline to enhance performance. The results revealed that fine-tuned LLaMA-2 models, particularly those incorporating RAG, achieved an F1 score of 0.82, surpassing the score of traditional CNER tools. Furthermore, LLMs demonstrated superior generalizability across different datasets. The study also explores the dependency of LLMs size for CNER tasks. A practical case study highlighting the application of these models in chemical entity extraction from pharmaceutical literature, achieving high accuracy in identifying drug and their interactions. These findings establish LLMs as a robust and adaptable alternative to traditional CNER tools, paving the way for transformative applications in chemoinformatics.