ZeroTuneBio NER: A three-stage framework for zero-shot and zero-tuning biomedical entity extraction using large language models and prompt engineering

Objective

This study aims to (1) enhance the performance of large language models (LLMs) in biomedical entity extraction, (2) investigate zero-shot named entity recognition (NER) capabilities without fine-tuning, and (3) compare the proposed framework with existing models and human annotation methods. Additionally, we analyze discrepancies between human and LLM-generated annotations to refine manual labeling processes for specialized datasets.

Materials and Methods

We propose ZeroTuneBio NER, a three-stage NER framework integrating chain-of-thought reasoning and prompt engineering. Evaluated on three public datasets (disease, chemistry, and gene), the method requires no task-specific examples or LLM fine-tuning, addressing challenges in complex concept interpretation.

Results

ZeroTuneBio NER excels in tasks without strict matching, achieving an average F1-score improvement of 0.28 over direct LLM queries and a partial-matching F1-score of ∼88 %. It rivals the performance of a fine-tuned LLaMA model trained on 11,240 examples and surpasses BioBERT trained on 22,480 examples when strict-matching errors are excluded. Notably, LLMs significantly optimize manual annotation, accelerating speed and reducing costs.

Conclusion

ZeroTuneBio NER demonstrates that LLMs can perform high-quality NER without fine-tuning, reducing reliance on manual annotation. The framework broadens LLM applications in biomedical NER, while our analysis highlights its scalability and future research directions.