RGPO: Ranking-Guided Preference Optimization for Reliable Clinical Reasoning

Abstract

Medical question answering (QA) in healthcare systems requires advanced reasoning that integrates domain knowledge with clinical workflows. However, existing large language models (LLMs) often generate reasoning chains that lack factual accuracy and clinical reliability. We propose ranking-guided preference optimization (RGPO) for Reliable Clinical Reasoning, a framework that combines reinforcement learning with preference driven reasoning refinement to enhance clinical chain-of-thought performance in LLM-based healthcare systems. RGPO employs task-adaptive reasoning templates aligned with clinical protocols and a probabilistic evaluation mechanism that integrates with healthcare information systems, including electronic health records (EHRs) and clinical documentation, while automatically identifying and correcting low-quality reasoning chains. Unlike traditional pairwise preference methods, RGPO introduces groupwise ranking optimization based on the Bradley–Terry model with Kullback–Leibler (KL)-divergence regularization for stable clinical deployment. The framework addresses key challenges, including multimodal data integration, clinical validation, and real-world interoperability. Experiments on PubMedQA, MedQA-USMLE, and real-world validation at Far Eastern Memorial Hospital (FEMH) demonstrate consistent improvements over strong baselines. Our 2B-parameter model outperforms larger 7B–20B models, including medical-specialized variants, while maintaining computational efficiency for healthcare deployment. These results demonstrate that combining preference optimization with quality-driven refinement provides a scalable, clinically validated approach to building reliable medical LLMs for practical healthcare applications.