Imogen E Leaning, Andrea Costanzo, Raj Jagesar, Loran Knol, Sarah Tjeerdsma, Anna Tyborowska, Nessa Ikani, Lianne M Reus, Pieter Jelle Visser, Martien Jh Kas, Christian F Beckmann, Henricus G Ruhé, Andre F Marquand
{"title":"Did you miss me? Making the most of digital phenotyping data by imputing missingness with point process models: observational study.","authors":"Imogen E Leaning, Andrea Costanzo, Raj Jagesar, Loran Knol, Sarah Tjeerdsma, Anna Tyborowska, Nessa Ikani, Lianne M Reus, Pieter Jelle Visser, Martien Jh Kas, Christian F Beckmann, Henricus G Ruhé, Andre F Marquand","doi":"10.1136/bmjhci-2026-102079","DOIUrl":"10.1136/bmjhci-2026-102079","url":null,"abstract":"<p><strong>Objectives: </strong>Smartphone-based digital phenotyping can provide low-burden behavioural measures for mental disorder monitoring. However, progress in making inferences from these data is challenged by the common occurrence of missing data. We propose a method to impute missingness using non-homogeneous Poisson point process models (PPPMs), where activities (overall phone, social media, communication app usage, outgoing/incoming calls) are modelled as 'points'.</p><p><strong>Methods: </strong>We evaluate personalised PPPMs for imputation and investigate their influence on downstream analysis. In a ground truth evaluation (in and out-of-sample), we evaluate time-varying covariates ('hour of the day', 'day of the week'; encoded using one-hot encoding and sine-cosine transformation) to model behavioural patterns in participants from SMARD (depression; n=26). We train a hidden Markov model (HMM) on data simulated by the PPPMs and compare this to a ground truth HMM. We then perform a replication of a prior HMM analysis in PRISM (Alzheimer's disease, schizophrenia, healthy controls; n=65) and Hersenonderzoek studies (Alzheimer's disease, memory complaints, healthy controls; n=283).</p><p><strong>Results: </strong>In the ground truth evaluation, 'hour' was consistently significant in in-sample likelihood ratio tests and 'day' was less commonly significant. PPPMs including one-hot encoded hour generally provided the highest out-of-sample likelihood. Using this PPPM variant, HMM properties were preserved, and prior findings were replicated.</p><p><strong>Discussion: </strong>Personalised PPPMs provide behavioural simulations that can be used for temporal imputation. These models capture average patterns and could be extended to include further temporal components.</p><p><strong>Conclusion: </strong>Non-homogeneous PPPMs are a promising imputation tool that may contribute to improved utility of digital phenotyping by providing realistic temporal imputations.</p>","PeriodicalId":9050,"journal":{"name":"BMJ Health & Care Informatics","volume":"33 1","pages":""},"PeriodicalIF":6.2,"publicationDate":"2026-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13636317/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149001223","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Leveraging time-series electronic health records with large language models for chronic kidney disease diagnosis in primary care.","authors":"Jingyi Wu, Yuewen Zheng, Zhijun He, Qianlin Zuo, Weidong Zhang, Pengfei Li, Zhang Luxia","doi":"10.1136/bmjhci-2025-101830","DOIUrl":"10.1136/bmjhci-2025-101830","url":null,"abstract":"<p><strong>Objectives: </strong>Chronic kidney disease (CKD) presents a growing public health challenge in China, exacerbated by low patient awareness and limited nephrology resources. This study evaluated the potential of large language models (LLMs) to support CKD diagnosis in primary care using time-series electronic health record (EHR) data.</p><p><strong>Methods: </strong>Longitudinal data were extracted from the EHR system of Weinan, China. Among 29 963 adults meeting inclusion criteria, 2300 participants were randomly sampled to generate clinical vignettes. CKD status was determined using diagnosis data, and individuals with early kidney injury indicated by laboratory abnormalities who lack a formal diagnosis were also identified. Using the DeepSeek LLM with prompt engineering, we generated binary CKD diagnoses and probability scores based on EHR data. Diagnostic performance was evaluated using accuracy, sensitivity, specificity, F1 score, area under the curve (AUC) and the detection rate for early kidney injury and compared with traditional machine learning (ML) models.</p><p><strong>Results: </strong>Using 1-month EHR data, the LLM achieved an accuracy of 87.0%, AUC of 0.921, F1 score of 0.579, sensitivity of 89.1%, specificity of 86.8%, and detection rate for early kidney injury of 42.2%. With 1-year data, sensitivity increased to 92.2% and early detection to 52.6%.</p><p><strong>Discussion: </strong>The LLM outperformed ML models across most metrics, achieving diagnostic performance comparable to clinicians and further enhancing detection for early kidney injury.</p><p><strong>Conclusions: </strong>LLMs applied to time-series EHR data may serve as a clinical decision support tool to improve CKD diagnosis in primary care, with particular value in resource-limited settings.</p>","PeriodicalId":9050,"journal":{"name":"BMJ Health & Care Informatics","volume":"33 1","pages":""},"PeriodicalIF":6.2,"publicationDate":"2026-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13599875/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149001012","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Katherine Griffin, Jessica M Ryan, Jan Sorensen, Laura Labbe, Qistina Emily Mohammad Feisal, Dara O Kavanagh, Walter Eppich, Deborah A McNamara
{"title":"Applications of the Early Warning Score as an outcome measure: a systematic review of methodologies.","authors":"Katherine Griffin, Jessica M Ryan, Jan Sorensen, Laura Labbe, Qistina Emily Mohammad Feisal, Dara O Kavanagh, Walter Eppich, Deborah A McNamara","doi":"10.1136/bmjhci-2026-102227","DOIUrl":"10.1136/bmjhci-2026-102227","url":null,"abstract":"<p><strong>Objectives: </strong>To evaluate how Early Warning Scores (EWS) are used and analysed as research outcome measures in interventional and comparative studies and to assess the consistency and methodological rigour of these approaches.</p><p><strong>Methods: </strong>We conducted a systematic review in accordance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (INPLASY 202540054), searching OVID Medline (MEDLINE, Embase, CENTRAL) (1946 to 6 January 2026) for interventional or comparative studies using an EWS as a research outcome measure. Three reviewers independently screened articles, extracted data and performed a narrative synthesis of methodological approaches regarding the use and analysis of EWS as a research outcome.</p><p><strong>Results: </strong>EWS were most commonly derived from intermittent ward observations (92/127, 72.0%) rather than continuous monitoring (2/127, 1.6%). Outcomes were typically defined as clinically interpretable summary measures, including index-time values, maximum scores within defined windows or threshold-based categories. Temporal analysis was limited, with most studies not incorporating time explicitly (91/127, 71.7%). Where time was considered (32/127, 25.2%), approaches focused on short-horizon prediction or fixed time point comparisons rather than modelling longitudinal trajectories; explicit assessment of change in EWS was uncommon (14/127, 11.0%). Reporting prioritised discrimination (98/127, 77.2%) over calibration (38/127, 29.9%), with variable consideration of repeated measures, missing data and validation.</p><p><strong>Conclusions: </strong>EWS are predominantly used as simple, clinically interpretable severity endpoints rather than as longitudinal measures of physiological change. Inconsistent handling of temporality and limited methodological standardisation restrict comparability and interpretation despite the availability of repeated time-stamped data. Development of reporting and analytic guidance is needed to support more robust use of EWS as research outcomes.</p>","PeriodicalId":9050,"journal":{"name":"BMJ Health & Care Informatics","volume":"33 1","pages":""},"PeriodicalIF":6.2,"publicationDate":"2026-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13630057/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149001234","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Implementation of electronic medication management systems in hospital settings: a theory-informed systematic review protocol of mechanisms influencing patient safety, operational efficiency and staff experience.","authors":"Chisom Victor Ukaegbu, Deborah Kamsi Asiegbu","doi":"10.1136/bmjhci-2026-102280","DOIUrl":"10.1136/bmjhci-2026-102280","url":null,"abstract":"<p><strong>Introduction: </strong>Electronic medication management systems (EMMS) are central to hospital digital transformation and are intended to improve patient safety, operational efficiency and staff experience, but evidence of their impact remains fragmented, inconsistent and context-dependent. Consequently, there is an urgent need to understand how local organisational structures and complex sociotechnical factors drive successful long-term integration.</p><p><strong>Methods and analysis: </strong>This systematic review will search MEDLINE, EMBASE, PsycINFO, CINAHL, Scopus, ABI/INFORM Collection and Business Source Premier for empirical hospital studies published from 2010 to 2026 that centrally apply a theoretical or conceptual framework. Study quality will be appraised using the Mixed Methods Appraisal Tool (MMAT) Version 2018, and a theory-informed thematic synthesis will identify implementation mechanisms and organise them into higher-order domains. This protocol has been registered with PROSPERO (CRD420261283974).</p><p><strong>Ethics and dissemination: </strong>Ethics approval is not required because only published data will be used. Findings will be disseminated through peer-reviewed publication and doctoral outputs.</p><p><strong>Prospero registration number: </strong>CRD420261283974.</p>","PeriodicalId":9050,"journal":{"name":"BMJ Health & Care Informatics","volume":"33 1","pages":""},"PeriodicalIF":6.2,"publicationDate":"2026-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13536060/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148872997","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Caroline Merel Anouck Mussert, Simone H Reitsma, Marjon H Cnossen, Evelien Shannon Van Hoorn
{"title":"How to develop care pathways: a practical overview of methodological approaches.","authors":"Caroline Merel Anouck Mussert, Simone H Reitsma, Marjon H Cnossen, Evelien Shannon Van Hoorn","doi":"10.1136/bmjhci-2026-102340","DOIUrl":"10.1136/bmjhci-2026-102340","url":null,"abstract":"<p><p>Care pathways are essential to enhance the organisation of care, optimise resource use and improve patient outcomes, thereby supporting value-based healthcare. As care pathways are increasingly required, there is a need for clear methodological guidance on the identification and visualisation of care pathways. This perspective article provides an overview of methodological approaches to support the identification and visualisation of these care pathways. Two possible methods are described, including co-creation and process mining as well as different visualisation techniques that can be applied for care pathway presentation. Moreover, practical guidance as well as advantages and limitations of each identification method are discussed to further support decision-making.Whether to implement co-creation or process mining, or use a combined approach, depends on several factors, such as the intended purpose and end users of the care pathway, time availability, level of expertise at hand, the structure and quality of the data, as well as stakeholder needs and availability. These considerations will vary for different clinical conditions and care processes involved. With anticipated workforce shortages, rising healthcare costs and increasing prevalence of hybrid and virtual care, delivering efficient, patient-centred and personalised care has become imperative. Consequently, the development of care pathways is essential to eliminate unnecessary interventions or those that do not meaningfully contribute to patient outcomes.</p>","PeriodicalId":9050,"journal":{"name":"BMJ Health & Care Informatics","volume":"33 1","pages":""},"PeriodicalIF":6.2,"publicationDate":"2026-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13536112/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148872953","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Bridging the black box: artificial intelligence in acute care needs greater interpretability and precision.","authors":"Yao-Chin Wang","doi":"10.1136/bmjhci-2026-102271","DOIUrl":"10.1136/bmjhci-2026-102271","url":null,"abstract":"","PeriodicalId":9050,"journal":{"name":"BMJ Health & Care Informatics","volume":"33 1","pages":""},"PeriodicalIF":6.2,"publicationDate":"2026-08-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13504477/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148787983","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Hannah Jeffery, Aaditya Prakash Sinha, Belul Shifa, Georgina Bitsakou, Mark Harries, Elinor Sawyer, Sarah Pinder, Wen Ng, Ali Sever, Sultana Hasso, Maria Javed, Priya Devadas, Qurratulain Chougle, Neyaf Almajali, Mohamed Attia, Mariam Malik, Mangesh A Thorat, Vivekanand Patkar, Arnie Purushotham
{"title":"Development and evaluation of AI-based clinical decision support system for streamlining breast multidisciplinary team meetings.","authors":"Hannah Jeffery, Aaditya Prakash Sinha, Belul Shifa, Georgina Bitsakou, Mark Harries, Elinor Sawyer, Sarah Pinder, Wen Ng, Ali Sever, Sultana Hasso, Maria Javed, Priya Devadas, Qurratulain Chougle, Neyaf Almajali, Mohamed Attia, Mariam Malik, Mangesh A Thorat, Vivekanand Patkar, Arnie Purushotham","doi":"10.1136/bmjhci-2026-102045","DOIUrl":"10.1136/bmjhci-2026-102045","url":null,"abstract":"<p><strong>Objectives: </strong>Multidisciplinary team (MDT) meetings are key to delivering cancer care. Increasing caseload and limited resources make them less effective and unsustainable. The aim of this quality improvement project was to assess novel artificial intelligence-based clinical decision support (CDS) technology to develop and validate standard of care (SoC) to streamline the breast MDT meetings in a tertiary cancer centre.</p><p><strong>Methods: </strong>A clinical governance group of the MDT approved international guidelines used to develop SoC. Deontics CDS was assessed for its suitability to apply the SoC pathway for benign and malignant breast disease with the exclusion of metastatic and recurrent cancer.</p><p><strong>Results: </strong>Patients discussed over the preceding 16 months were added to the platform in cohorts of 50 women: two consisting of 50 women each diagnosed with benign disease (benign A and B: n=100) and three consisting of 50 women each diagnosed with malignant disease (cancer A, B and C: n=150). Concordance between the blinded MDT decision outcomes and SoC recommendations was analysed. This stepwise approach identified knowledge gaps in SoC and refined the CDS. Concordance improved from 82% to 100% in benign and from 94% to 100% in malignant cases.</p><p><strong>Discussion: </strong>A sequential process of validating the SoC with data derived from the development of evidence-based SoC protocols based on international guidelines resulted in a final 100% concordance rate between the platform and MDT recommendations for both benign and malignant disease.</p><p><strong>Conclusions: </strong>CDS technology could be a milestone in using SoC to deliver a sustainable clinical decision pathway.</p>","PeriodicalId":9050,"journal":{"name":"BMJ Health & Care Informatics","volume":"33 1","pages":""},"PeriodicalIF":6.2,"publicationDate":"2026-08-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13504842/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148788047","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Sara Santos, Lara Bruno-Boucher, Hallie Benjamin, Bianca Arenas-Rodriguez, Joanna Soscia, Kimberly Colapinto, Susan Miranda, Kate Nelson, Julia Orkin, Peter J Gill, Alene Toulany, Martin Yi, Lillian Sung, Eyal Cohen
{"title":"CMC-ID: identifying children and youth with medical complexity using an electronic health record.","authors":"Sara Santos, Lara Bruno-Boucher, Hallie Benjamin, Bianca Arenas-Rodriguez, Joanna Soscia, Kimberly Colapinto, Susan Miranda, Kate Nelson, Julia Orkin, Peter J Gill, Alene Toulany, Martin Yi, Lillian Sung, Eyal Cohen","doi":"10.1136/bmjhci-2026-102268","DOIUrl":"10.1136/bmjhci-2026-102268","url":null,"abstract":"<p><strong>Objective: </strong>We aimed to develop and evaluate the performance of CMC-ID, an algorithm designed to identify children and youth with medical complexity (CMC) from electronic health record (EHR) data for recruitment to a transition to adult care programme, and then tested it in younger patients.</p><p><strong>Methods: </strong>CMC-ID was developed iteratively at a paediatric tertiary care hospital (SickKids) in Toronto, Canada, using a data repository derived from an Epic System EHR. The algorithm captured standard clinical criteria for CMC used throughout Ontario, Canada, including (1) Complexity, (2) Chronicity, (3) Fragility and (4) Technology dependence. We refined the CMC-ID iteratively to maximise positive predictive value (PPV). In phase I, we retrospectively identified youth aged 17 years to <18 years and conducted chart reviews to confirm CMC status, and then repeated the search in a younger cohort (1 year to <17 years) in phase II.</p><p><strong>Results: </strong>Among 280 unique patients identified in phase I, the last of six iterations of the algorithm identified CMC with a PPV of 85.5% (95% CI 73.3% to 93.5%). When applied to a younger cohort in phase II, 947 CMC were identified, and the algorithm performed similarly with a PPV of 89.1% (95% CI 86.9% to 91.0%).</p><p><strong>Discussion: </strong>CMC-ID identified youth for recruitment to an intervention focused on transition to adult care with high PPV, and the algorithm performed similarly when applied to younger patients.</p><p><strong>Conclusion: </strong>CMC-ID is a pragmatic, high-alert index to support recruitment to clinical programmes and other interventions aimed at improving CMC outcomes.</p>","PeriodicalId":9050,"journal":{"name":"BMJ Health & Care Informatics","volume":"33 1","pages":""},"PeriodicalIF":6.2,"publicationDate":"2026-08-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13504859/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148788129","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Artificial intelligence in lumbar radiography: bridging deep learning and clinical practice in low-resource environments.","authors":"Yu-Li Wang, Shuwei Huang, Kuei-Chen Lee, Chao-Min Cheng","doi":"10.1136/bmjhci-2025-101693","DOIUrl":"10.1136/bmjhci-2025-101693","url":null,"abstract":"<p><strong>Background: </strong>Artificial intelligence (AI) has increasingly been applied to medical imaging, yet its role in lumbar spine radiography, particularly in low-resource settings, remains underexplored.</p><p><strong>Objective: </strong>To evaluate recent developments in AI-based approaches for lumbar spine radiography and their clinical applicability in resource-constrained environments.</p><p><strong>Methods: </strong>A narrative review was conducted focusing on deep learning models applied to lumbar radiographic analysis. Studies published between 2022 and 2024 were identified through structured screening of PubMed and Google Scholar.</p><p><strong>Results: </strong>Deep learning models, including convolutional neural networks, U-Net, ResNet and generative adversarial networks, have demonstrated improved performance in segmentation, classification and curvature analysis. Lightweight architectures show potential for deployment in resource-limited settings.</p><p><strong>Conclusion: </strong>AI-based lumbar imaging has the potential to enhance diagnostic accuracy and workflow efficiency in low-resource environments. However, challenges related to validation, interpretability and clinical integration remain, highlighting the need for further large-scale and real-world studies.</p>","PeriodicalId":9050,"journal":{"name":"BMJ Health & Care Informatics","volume":"33 1","pages":""},"PeriodicalIF":6.2,"publicationDate":"2026-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13475137/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148711131","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Benchmarking large language models for question answering on German clinical practice guidelines.","authors":"Johannes Schwietering, Gregor Lichtner","doi":"10.1136/bmjhci-2026-102207","DOIUrl":"10.1136/bmjhci-2026-102207","url":null,"abstract":"<p><strong>Objectives: </strong>Clinical practice guidelines are a cornerstone of evidence-based medicine, yet their implementation in routine care remains inconsistent. Large language models (LLMs), particularly with Retrieval-Augmented Generation (RAG), have shown strong performance in medical question answering, but their ability to use knowledge from German-language guidelines has not been systematically evaluated due to a lack of a dedicated benchmark. We therefore developed such a benchmark and evaluated guideline-based question answering with different LLMs and retriever configurations.</p><p><strong>Methods: </strong>We developed cpgQA-DE, an expert-validated benchmark dataset of 200 multiple-choice questions derived from 10 current German clinical practice guidelines across five specialties. All questions were reviewed for correctness, relevance and complexity. The dataset includes case-based and knowledge-based questions with metadata on guideline source, specialty, relevance and difficulty. We evaluated LLM performance using a RAG-based pipeline built on a corpus of German guidelines, comparing multiple model-retriever combinations.</p><p><strong>Results: </strong>RAG integration substantially improved accuracy across all tested models and for most retrievers. The best-performing configuration, GPT-5 combined with the multilingual-e5-large retriever, achieved an accuracy of 95%. Notably, the open-weight model gpt-oss-120b reached 90% accuracy when used with RAG.</p><p><strong>Conclusions: </strong>cpgQA-DE enables systematic and reproducible offline evaluation of guideline-aware question answering systems in the German healthcare context. The observed performance gains with RAG support the use of LLMs augmented with quality-assured external knowledge. Such systems may help bridge the evidence-practice gap by providing guideline-based recommendations at the point of care, while strong performance of open-weight models suggests potential for on-premises deployment in privacy-sensitive clinical environments.</p>","PeriodicalId":9050,"journal":{"name":"BMJ Health & Care Informatics","volume":"33 1","pages":""},"PeriodicalIF":6.2,"publicationDate":"2026-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13475501/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148711248","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}