Evaluating RAG and Non-RAG Pipelines for Concept Discovery in Environmental Health Ontologies.

Abstract

The expansion of biomedical ontologies with relevant, high utility concepts remains a significant challenge in biomedical knowledge representation, particularly for rapidly evolving fields like Environmental Determinants of Health (EnDOH). In this work, we evaluate the effectiveness of using LLMs in support of ontology expansion, comparing Retrieval-Augmented Generation (RAG) with non-RAG concept extraction from the medical literature. Candidate concepts were generated across 15 targeted topics using category-specific prompts. The quality of candidate concepts was assessed through semantic similarity to existing EnDOH concepts and sub-hierarchies. This design enables both a comparative analysis of RAG versus non-RAG concept extraction approaches and the identification of topic-level concept alignment with the ontology. Our results quantify the comparative strengths and weaknesses of RAG vs non-RAG concept extraction and offer a replicable methodology for effectively extracting potentially useful candidate concepts from the literature for the purpose of inclusion in biomedical ontologies.