From Guidelines to Code: Formalizing STOPP/START Criteria Using LLMs and RAG for Clinical Decision Support.

Abstract

STOPP/START v3 is a set of criteria for optimizing therapy for elderly patients with polypharmacy. Implementing these criteria in prescribing software requires to formalize them, which is a difficult task. This project aimed to automate the formalization of these criteria using large language models (LLMs), specifically leveraging Retrieval-Augmented Generation (RAG) for enhanced accuracy. We employed DeepSeek and GPT-4o-mini for entity extraction, code mapping to ICD-10, LOINC, and ATC, and the generation of executable Python code. A preliminary evaluation conducted on a subset of rules yielded a notably high F1-score (0.90, 0.92, 1 for drug, disease and observation entity mapping respectively and perfect results for medical entity extraction and code logic consistency). These results confirm the model's effectiveness in accurately transforming complex clinical rules into executable code. In conclusion, we successfully automated the creation of executable code from medical guidelines, proving that LLMs, supported by RAG, can be effective for automating clinical decision support tasks and formalizing medical rules.