NPJ Digital Medicine最新文献

MEDA-PKD: a dynamic, real-time analytics platform for precision care in autosomal dominant polycystic kidney disease.

IF 18 1区医学

NPJ Digital Medicine Pub Date : 2026-10-20 DOI: 10.1038/s41746-026-03267-0

Franziska Grundmann, Sita Arjune, Samer Alkarkoukly, Malte P Bartram, Bodo B Beck, Thomas Benzing, Volker Boehm, Katrin Bohl, Volker Burst, Martin Busch, Figen Cakiroglu, Sadrija Cukoski, Peter Gerke, Nils Große-Hokamp, Jan Halbritter, Ludwig Matrisch, Thorsten Persigehl, Friederike Petzold, Johannes Schmidt, Roland Schmitt, Polina Todorova, Christoph Wanner, Philipp Antczak, Roman-Ulrich Müller

{"title":"MEDA-PKD: a dynamic, real-time analytics platform for precision care in autosomal dominant polycystic kidney disease.","authors":"Franziska Grundmann, Sita Arjune, Samer Alkarkoukly, Malte P Bartram, Bodo B Beck, Thomas Benzing, Volker Boehm, Katrin Bohl, Volker Burst, Martin Busch, Figen Cakiroglu, Sadrija Cukoski, Peter Gerke, Nils Große-Hokamp, Jan Halbritter, Ludwig Matrisch, Thorsten Persigehl, Friederike Petzold, Johannes Schmidt, Roland Schmitt, Polina Todorova, Christoph Wanner, Philipp Antczak, Roman-Ulrich Müller","doi":"10.1038/s41746-026-03267-0","DOIUrl":"https://doi.org/10.1038/s41746-026-03267-0","url":null,"abstract":"<p><p>The use of medical real-world data is constrained by fragmented data infrastructures and the absence of tools integrating longitudinal clinical with molecular data at scale. Autosomal dominant polycystic kidney disease (ADPKD), the most common monogenic cause of kidney failure, exemplifies this gap given its phenotypic complexity and the need for individualized treatment decisions. We present MEDA-PKD (MEdical DAta translation service of Polycystic Kidney Disease), an interactive, near-real-time analytics platform that integrates clinical routine with research data from the German AD(H)PKD Tolvaptan Registry (NCT02497521, DRKS00008910) into a harmonized database ( https://doi.org/10.18716/adpkd_registry ). MEDA-PKD provides automated near-real-time analysis and dynamic visualization, replacing traditional static approaches to clinical cohorts. As of March 2026, MEDA-PKD visualized data from 1735 patients (mean follow-up 796 days). eGFR slope analysis of data from patients initiating tolvaptan therapy (332 patients) demonstrates the platform's capacity to evaluate real-world therapeutic impact in real-time. Other tabs include family history, medication, lifestyle, quality of life and a wide array of lab values (20,894 entries). Besides, MEDA-PKD integrates proteomics data and allows for secure individual patient-level data access. MEDA-PKD represents a paradigm shift by transforming static cohort data into a living, actionable knowledge resource in ADPKD and will serve as a template for other diseases.</p>","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":"9 1","pages":""},"PeriodicalIF":18.0,"publicationDate":"2026-10-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149228381","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A generalizable deep learning model for predicting opioid use disorder using language modeling for cross-EHR data harmonization 使用跨ehr数据协调的语言建模预测阿片类药物使用障碍的可推广深度学习模型

IF 15.2 1区医学

NPJ Digital Medicine Pub Date : 2026-10-19 DOI: 10.1038/s41746-026-03392-w

Chun-Nan Hsu, Rohith Vutukuru, Brian H. Park, Seshadri C. Mudumbai, Lucas Teixeira, Ji-Qing Chen, Soraya Mehdipour, Gregory Polston, Carla Marienfeld, Rodney A. Gabriel

{"title":"A generalizable deep learning model for predicting opioid use disorder using language modeling for cross-EHR data harmonization","authors":"Chun-Nan Hsu, Rohith Vutukuru, Brian H. Park, Seshadri C. Mudumbai, Lucas Teixeira, Ji-Qing Chen, Soraya Mehdipour, Gregory Polston, Carla Marienfeld, Rodney A. Gabriel","doi":"10.1038/s41746-026-03392-w","DOIUrl":"https://doi.org/10.1038/s41746-026-03392-w","url":null,"abstract":"The opioid use disorder (OUD) epidemic in the United States has been a major public health crisis. We developed OUD-REY (OUD Risk Evaluator at 1 Year) - a deep learning model that uses longitudinal electronic health record (EHR) data from the All of Us research program to predict risk of OUD at one year for patients with an opioid prescription. On internal validation, OUD-REY achieved an AUROC of 91.2% and AUPRC of 77.0%. We leveraged a pretrained language model to accommodate thousands of unseen, out-of-vocabulary clinical features in a patient’s longitudinal EHR history. We externally validated our model on a new dataset released by All of Us that was temporally separated from our development set and on a dataset containing ~6 million patients from medical centers of the Veterans Affairs (VA) nationwide over a span of 20 years. On external validation, our model had an AUROC and AUPRC of 86.4% and 23.1% (OUD prevalence of 2.9%), respectively, on our temporally separated All of Us validation set. It had an AUROC and AUPRC of 85.5% and 19.5% (OUD prevalence of 3.5%), respectively, on the VA dataset. Our fairness auditing showed that the model was generally fair for patients with different genders, race, and ethnic backgrounds. This model may potentially improve prevention and care of OUD.","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":"54 1","pages":""},"PeriodicalIF":15.2,"publicationDate":"2026-10-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149218130","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Reinforcement learning for repetitive transcranial magnetic stimulation protocol selection in treatment-resistant depression 反复经颅磁刺激治疗难治性抑郁症方案选择的强化学习

IF 15.2 1区医学

NPJ Digital Medicine Pub Date : 2026-10-19 DOI: 10.1038/s41746-026-03417-4

Jason B. Gibbons, Nicolas Leenaerts, Shan H. Siddiqi, Susan A. Murphy, Loreen Straub, David B. Hathaway, Jie Yang, Alejandro G. Szmulewicz, Meekang Sung, Rajendra Aldis, Jacob C. Jameson, Richard Wyss, Rachel Wood, Matthew Ruble, Cason Elliott, Denise H. Mulcahy, Stanley Lyndon, Philip S. Wang

{"title":"Reinforcement learning for repetitive transcranial magnetic stimulation protocol selection in treatment-resistant depression","authors":"Jason B. Gibbons, Nicolas Leenaerts, Shan H. Siddiqi, Susan A. Murphy, Loreen Straub, David B. Hathaway, Jie Yang, Alejandro G. Szmulewicz, Meekang Sung, Rajendra Aldis, Jacob C. Jameson, Richard Wyss, Rachel Wood, Matthew Ruble, Cason Elliott, Denise H. Mulcahy, Stanley Lyndon, Philip S. Wang","doi":"10.1038/s41746-026-03417-4","DOIUrl":"https://doi.org/10.1038/s41746-026-03417-4","url":null,"abstract":"Repetitive transcranial magnetic stimulation (rTMS), including intermittent theta-burst stimulation (iTBS), is an established treatment for treatment-resistant depression, but clinicians lack evidence-based guidance for session-level choices among stimulation type, intensity, and laterality. In this study, we investigated whether a reinforcement learning (RL) algorithm trained on routine clinical data could recommend session-level rTMS protocols associated with greater symptom improvement using electronic health record data from 1609 adults treated with rTMS between January 2023 and June 2025 across 19 clinic groups. Among 370 patients from held-out clinics, treatment decisions matching algorithm recommendations were associated with 0.788 points greater next-session Patient Health Questionnaire-9 improvement than treatments that differed (95% CI: 0.634–0.940). Following every recommendation was projected to produce 3.22 points of additional improvement (95% CI: 2.72–3.75). These findings suggest RL can find treatment patterns associated with greater next-session improvement and potentially meaningful improvement accumulated across a treatment episode.","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":"13 1","pages":""},"PeriodicalIF":15.2,"publicationDate":"2026-10-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149218128","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Extending synthetic medical education: generating OSCE cases with LLMs 扩大综合医学教育：利用法学硕士产生欧安组织案例

IF 15.2 1区医学

NPJ Digital Medicine Pub Date : 2026-10-19 DOI: 10.1038/s41746-026-03421-8

Kaiz P. Esmail, Arya S. Rao, Richard S. Lee, Ali Ebada, David C. Whitehead, Caroline Meehan, Andrew Cheung, Marc D. Succi

{"title":"Extending synthetic medical education: generating OSCE cases with LLMs","authors":"Kaiz P. Esmail, Arya S. Rao, Richard S. Lee, Ali Ebada, David C. Whitehead, Caroline Meehan, Andrew Cheung, Marc D. Succi","doi":"10.1038/s41746-026-03421-8","DOIUrl":"https://doi.org/10.1038/s41746-026-03421-8","url":null,"abstract":"Objective Structured Clinical Examinations (OSCEs) are central to competency-based medical education, yet high-quality case development is resource-intensive. Large language models (LLMs) offer a scalable solution, but comprehensive OSCE case generation, including components such as candidate instructions, standardized patient scripts, and examiner marking schemes, is underexplored. We developed an LLM-based tool to generate complete OSCE stations. Three board-certified internists evaluated 24 generated cases across eight internal medicine diagnoses and three learner levels using a 12-item Likert instrument. Ratings were high across all domains; communication behaviors (4.64/5), EPA alignment (4.62/5), and overall value (4.61/5) performed best; guideline concordance scored lowest (4.24/5) with the greatest rater disagreement. As a feasibility study, these findings establish that whole, schema-validated OSCE stations can be generated from a single structured call at a quality experts rate as acceptable, while also identifying guideline concordance and difficult calibration as the domains most requiring faculty oversight -- providing reproducible infrastructure for, but not yet evidence of, effective EPA-aligned OSCE development.","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":"42 1","pages":""},"PeriodicalIF":15.2,"publicationDate":"2026-10-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149218137","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

AI-supported first consultation preparation in oncology: a real-world clinical usability study 人工智能支持的肿瘤学首次会诊准备：现实世界的临床可用性研究

IF 15.2 1区医学

NPJ Digital Medicine Pub Date : 2026-10-17 DOI: 10.1038/s41746-026-03396-6

Josephine W. van Koevorden,Rachael Mertooetomo,David Neal,Bilge Çelik,Veerle Struben,Nanne Aben,Niels van den Hork,Vincent van der Noort,Marene Kroon,Luc Karssemakers,Ludi Smeele,Michel W. Wouters,Richard Dirven

{"title":"AI-supported first consultation preparation in oncology: a real-world clinical usability study","authors":"Josephine W. van Koevorden,Rachael Mertooetomo,David Neal,Bilge Çelik,Veerle Struben,Nanne Aben,Niels van den Hork,Vincent van der Noort,Marene Kroon,Luc Karssemakers,Ludi Smeele,Michel W. Wouters,Richard Dirven","doi":"10.1038/s41746-026-03396-6","DOIUrl":"https://doi.org/10.1038/s41746-026-03396-6","url":null,"abstract":"Abstract Large Language Models (LLMs) may support clinicians by facilitating information management, but their usability in real-world oncology practice remains understudied. This study evaluated the workflow performance of Artificial Intelligence-based First Consultation Preparation (AI-FCP), generating structured summaries from unstructured clinical documentation, co-created with end users. In a prospective single‑group pre‑post design, nine clinicians completed a person-level baseline, followed by 33 consultations with AI-FCP. Case numbers varied by clinic scheduling and enrollment timing, potentially introducing survivorship bias. Usability was evaluated using validated questionnaires and study-specific instruments. Post AI-FCP, overall perceived workload (NASA-TLX) was 28.9% lower than baseline (Hodges-Lehmann estimate difference −12.3; 95% CI –21.7 to –3.3), with self-reported average time savings of 3.26 min (SD = 2.65) per case. Preparation succeeded in 84.9% of consultations; 63.6% of summaries were complete. These exploratory findings suggest clinicians perceived lower workload and reported improved workflow efficiency with AI-FCP. Further research is needed to determine generalizability.","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":"14 1","pages":""},"PeriodicalIF":15.2,"publicationDate":"2026-10-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149208638","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Large language models for depression assessment from brief daily diaries 从简短的日常日记中评估抑郁症的大型语言模型

IF 15.2 1区医学

NPJ Digital Medicine Pub Date : 2026-10-17 DOI: 10.1038/s41746-026-03388-6

Whitney R. Ringwald,Aman Taxali,Mike Angstadt,Colin E. Vize,Chandra Sripada,Aidan G. C. Wright

{"title":"Large language models for depression assessment from brief daily diaries","authors":"Whitney R. Ringwald,Aman Taxali,Mike Angstadt,Colin E. Vize,Chandra Sripada,Aidan G. C. Wright","doi":"10.1038/s41746-026-03388-6","DOIUrl":"https://doi.org/10.1038/s41746-026-03388-6","url":null,"abstract":"Abstract Closely tracking changes in depression is important for clinical research and practice. However, intensive longitudinal depression assessment relies on self-report surveys that have major limitations. Scoring depression from natural language with large language models (LLMs) could resolve these shortcomings. This study tested the validity of depression rated by LLMs from daily video diaries. We studied 108 participants selected for mental health treatment status (45% in treatment). Participants completed a clinical interview and 14-21 days of depression surveys and video diaries. Daily depression scores were estimated by LLMs from diary transcripts. LLM ratings converged with self-report depression within-person ( r = .45;95% CrI[.40, .49]) and when averaged across days for an overall depression score ( r = .61;95% CrI[.43,.75]). Averaged LLM ratings correlated with depression symptom severity ascertained by clinical interview ( r = .53;95% CrI[.35, .68]). LLM-rated depression and self-report had similar profiles of associations with daily functioning variables (profile r = .98 within-person; r = .94 between-person). Multivariable regression results showed LLM ratings predicted daily functioning variables (partial $$beta s$$ β s =|.12 | - | .28 | ) and interview-rated depression (partial $$beta$$ β =.33;95% CrI[.07, .58]) over and above self-reports, suggesting LLMs capture clinically-relevant information that self-reports miss. Findings establish a strong foundation for a depression assessment approach that addresses shortcomings of self-reports while satisfying the unique demands of intensive longitudinal monitoring.","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":"36 1","pages":""},"PeriodicalIF":15.2,"publicationDate":"2026-10-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149208643","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

GWDRO-Lung: Gromov–Wasserstein distributionally robust learning for calibrated pulmonary nodule malignancy prediction across imaging cohorts gwro - lung: Gromov-Wasserstein分布稳健性学习在影像学队列中校准肺结节恶性预测

IF 15.2 1区医学

NPJ Digital Medicine Pub Date : 2026-10-17 DOI: 10.1038/s41746-026-03323-9

Yubo Yan,An Yan,Wen Li,Jianlong Bu,Jinfeng Ning

引用次数: 0

Postoperative CT habitat fusion model predicts prognosis in non-small cell lung cancer after neoadjuvant immunochemotherapy 术后CT栖息地融合模型预测非小细胞肺癌新辅助免疫化疗后预后

IF 15.2 1区医学

NPJ Digital Medicine Pub Date : 2026-10-16 DOI: 10.1038/s41746-026-03415-6

Shiyang Huang,Chaokang Han,Zhenwei Shi,Xiaonan Cui,Yue Yu,Chunhao Fang,Qian Zhou,Chen Lin,Mingyue Zou,Dingpin Huang,Jinyi Jiang,Jiajie Hu,Pingping Wu,Zaiyi Liu,Dongsheng Yue,Yilong Huang,Xianglan Zhou,Hongjie Hu,Liting Shi,Lei Shi

{"title":"Postoperative CT habitat fusion model predicts prognosis in non-small cell lung cancer after neoadjuvant immunochemotherapy","authors":"Shiyang Huang,Chaokang Han,Zhenwei Shi,Xiaonan Cui,Yue Yu,Chunhao Fang,Qian Zhou,Chen Lin,Mingyue Zou,Dingpin Huang,Jinyi Jiang,Jiajie Hu,Pingping Wu,Zaiyi Liu,Dongsheng Yue,Yilong Huang,Xianglan Zhou,Hongjie Hu,Liting Shi,Lei Shi","doi":"10.1038/s41746-026-03415-6","DOIUrl":"https://doi.org/10.1038/s41746-026-03415-6","url":null,"abstract":"Abstract Neoadjuvant immunochemotherapy (NAIC) has been widely adopted for non-small cell lung cancer (NSCLC); however, reliable tools for postoperative disease-free survival (DFS) risk stratification remain lacking. We developed and validated a habitat radiomics–clinicopathological fusion model, termed the Neoadjuvant Lung Joint Clustering (NeoLung-JC) model, for postoperative DFS stratification and prediction in patients with NSCLC who underwent surgery after NAIC. The model integrates dual-sequence CT-derived tumor habitat heterogeneity scores with key clinicopathological factors. The habitat heterogeneity scores were extracted from the tumor core and peripheral regions using a joint clustering approach. In a multicenter cohort of 508 patients, the NeoLung-JC model showed favorable and consistent predictive performance, achieving a concordance index of 0.750 (95% confidence interval [CI]: 0.680–0.807) in the external validation cohort. The model significantly stratified patients into high-, medium-, and low-risk DFS groups, with clearly separated Kaplan–Meier survival curves across all cohorts ( P < 0.0001). Its prognostic performance exceeded that of conventional clinicopathological indicators, including clinical TNM (cTNM) stage, post-treatment pathological TNM (ypTNM) stage, and pathological response. By integrating dual-sequence CT-derived tumor spatial heterogeneity with clinical risk characteristics, the NeoLung-JC model may provide a useful supplementary approach for individualized postoperative prognostic assessment in patients with NSCLC undergoing surgery after NAIC.","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":"46 1","pages":""},"PeriodicalIF":15.2,"publicationDate":"2026-10-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149208644","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Effectiveness of ecological momentary interventions for patients with cancer: a meta-analysis of randomized controlled trials 生态短暂干预对癌症患者的有效性：随机对照试验的荟萃分析

IF 15.2 1区医学

NPJ Digital Medicine Pub Date : 2026-10-16 DOI: 10.1038/s41746-026-03400-z

Liyang Duan,Shu Zhang,Linna Li,Xiaolin Hu

{"title":"Effectiveness of ecological momentary interventions for patients with cancer: a meta-analysis of randomized controlled trials","authors":"Liyang Duan,Shu Zhang,Linna Li,Xiaolin Hu","doi":"10.1038/s41746-026-03400-z","DOIUrl":"https://doi.org/10.1038/s41746-026-03400-z","url":null,"abstract":"Abstract Symptom burden and impaired quality of life (QoL) are major challenges for patients with cancer. Ecological Momentary Interventions (EMIs) offer real-time, context-sensitive, and personalized support, enabling more precise management of patient needs. To explore the effectiveness of EMIs in improving symptoms and QoL in patients with cancer, a systematic review and meta-analysis was conducted. Following PRISMA 2020, a comprehensive search was conducted in nine databases from inception to December 8, 2025. The RoB 2 tool and GRADE were used to assess risk of bias and certainty of evidence, respectively. The R (version 4.4.1) and the “meta” package were used for statistical analysis. A total of 17 studies involving 4,160 randomized participants were included. The results suggested that EMIs may improve depression in patients with cancer, but were not associated with significant improvements in fatigue, pain severity, pain interference, sleep disturbance, anxiety, or QoL. Given the limited certainty and robustness of the available evidence, further adequately powered, allocation-concealed randomized controlled trials with blinded outcome assessment, where feasible, are needed to strengthen the evidence base for EMIs in oncology care.","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":"13 1","pages":""},"PeriodicalIF":15.2,"publicationDate":"2026-10-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149208642","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A multi-stage validation framework for trustworthy large-scale clinical information extraction using large language models 基于大型语言模型的可信赖大规模临床信息提取的多阶段验证框架

IF 15.2 1区医学

NPJ Digital Medicine Pub Date : 2026-10-16 DOI: 10.1038/s41746-026-03341-7

Maria Mahbub,Gregory M. Dams,Josh Arnold,Caitlin Rizy,Sudarshan Srinivasan,Elliot M. Fielstein,Minu A. Aghevli,Kamonica L. Craig,Elizabeth M. Oliva,Joseph Erdos,Jodie Trafton,Ioana Danciu

{"title":"A multi-stage validation framework for trustworthy large-scale clinical information extraction using large language models","authors":"Maria Mahbub,Gregory M. Dams,Josh Arnold,Caitlin Rizy,Sudarshan Srinivasan,Elliot M. Fielstein,Minu A. Aghevli,Kamonica L. Craig,Elizabeth M. Oliva,Joseph Erdos,Jodie Trafton,Ioana Danciu","doi":"10.1038/s41746-026-03341-7","DOIUrl":"https://doi.org/10.1038/s41746-026-03341-7","url":null,"abstract":"Abstract Large language models (LLMs) show promise for extracting clinically meaningful information from unstructured health records, yet real-world deployment is constrained by limited scalable, trustworthy validation approaches. Conventional evaluation relies on annotation-intensive reference standards and incomplete structured data, limiting population-scale use. We propose a multi-stage validation framework for LLM-based clinical information extraction that enables rigorous assessment under weak supervision by integrating complementary validation strategies. The framework combines prompt calibration, rule-based filtering, semantic grounding, judge-LLM evaluation, expert review, and predictive validity analysis to quantify uncertainty without exhaustive manual annotation. We applied this framework to extract substance use disorder (SUD) diagnoses across 11 substance categories from 919,783 clinical notes. Rule-based filtering and semantic grounding removed 14.59% of LLM-positive extractions that were unsupported or structurally implausible. For high-uncertainty cases, the judge LLM showed substantial agreement with subject matter expert. Using judge evaluations as references, the primary LLM achieved an F1 score of 0.80. LLM-extracted SUD diagnoses predicted subsequent engagement in SUD specialty care more accurately than structured-data baselines. By unifying complementary validation strategies, this work presents a generalizable human-in-the-loop framework for establishing the trustworthiness of LLM-based clinical information extraction at scale while minimizing the need for exhaustive manual annotation.","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":"97 1","pages":""},"PeriodicalIF":15.2,"publicationDate":"2026-10-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149208640","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0