Natural Language Processing in Clinical Quality Measures of Diagnostic Performance: Learnings from Three Case Reports

IF 3 Q2 HEALTH CARE SCIENCES & SERVICES

Joint Commission journal on quality and patient safety Pub Date : 2026-04-01 Epub Date: 2026-01-26 DOI:10.1016/j.jcjq.2026.01.005

Abigail Evans PhD (is Senior Social Scientist, Health Research and Analytics, Battelle Memorial Institute, Columbus, Ohio.), Meridith Eastman PhD, MSPH (is Senior Research Scientist, Health Research and Analytics, Battelle Memorial Institute.), Patricia C. Dykes PhD, MA, RN (is Research Program Director, Center for Patient Safety Research and Practice, Brigham and Women’s Hospital, Boston, and Professor of Medicine, Harvard Medical School.), Barbara E. Jones MD, MSCI (is Associate Professor and Pulmonologist, Division of Pulmonary and Critical Care Medicine, University of Utah School of Medicine.), John Sather MD (is Associate Professor and Director of Patient Safety and Quality, Department of Emergency Medicine, Yale University School of Medicine.), Jeffrey Geppert EdM, JD (is Senior Research Leader, Health Research and Analytics, Battelle Memorial Institute. Please address correspondence to Abigail Evans)

{"title":"Natural Language Processing in Clinical Quality Measures of Diagnostic Performance: Learnings from Three Case Reports","authors":"Abigail Evans PhD (is Senior Social Scientist, Health Research and Analytics, Battelle Memorial Institute, Columbus, Ohio.), Meridith Eastman PhD, MSPH (is Senior Research Scientist, Health Research and Analytics, Battelle Memorial Institute.), Patricia C. Dykes PhD, MA, RN (is Research Program Director, Center for Patient Safety Research and Practice, Brigham and Women’s Hospital, Boston, and Professor of Medicine, Harvard Medical School.), Barbara E. Jones MD, MSCI (is Associate Professor and Pulmonologist, Division of Pulmonary and Critical Care Medicine, University of Utah School of Medicine.), John Sather MD (is Associate Professor and Director of Patient Safety and Quality, Department of Emergency Medicine, Yale University School of Medicine.), Jeffrey Geppert EdM, JD (is Senior Research Leader, Health Research and Analytics, Battelle Memorial Institute. Please address correspondence to Abigail Evans)","doi":"10.1016/j.jcjq.2026.01.005","DOIUrl":null,"url":null,"abstract":"<div><h3>Background</h3><div>Clinical quality measures (CQMs) have historically relied on structured data elements in electronic health records; however, unstructured fields more reliably capture information needed to assess quality of care. Unstructured fields such as clinical notes and imaging results are particularly important for assessing diagnostic performance. Natural language processing (NLP) can efficiently capture nuanced information from unstructured fields to assess healthcare quality. This article describes three CQM projects that incorporated NLP in ways that are scalable for healthcare systemwide implementation.</div></div><div><h3>Methods</h3><div>The projects leveraged NLP to measure diagnostic performance related to venous thromboembolism, pneumonia, and pulmonary embolism, each of which is characterized by nonspecific symptoms and patient safety risk.</div></div><div><h3>Results</h3><div>NLP was used to extract patient symptoms, procedures, and findings from clinical notes in outpatient settings, emergency department visits, radiology reports, and discharge summaries. One NLP algorithm developed in the US Department of Veterans Affairs system performed comparably in an academic medical system. Other measures were successfully implemented in a data registry. Two of these CQMs were endorsed by the Centers for Medicare & Medicaid Services consensus-based entity. The third CQM set will be available as registry measures.</div></div><div><h3>Conclusion</h3><div>These projects demonstrate that CQMs leveraging NLP can achieve the high levels of reliability and validity necessary for national-level implementation. This will support efforts to assess diagnostic performance, which is often challenging to assess using structured data.</div></div>","PeriodicalId":14835,"journal":{"name":"Joint Commission journal on quality and patient safety","volume":"52 4","pages":"Pages 154-161"},"PeriodicalIF":3.0000,"publicationDate":"2026-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Joint Commission journal on quality and patient safety","FirstCategoryId":"1085","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1553725026000097","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2026/1/26 0:00:00","PubModel":"Epub","JCR":"Q2","JCRName":"HEALTH CARE SCIENCES & SERVICES","Score":null,"Total":0}

引用次数: 0

Abstract

Background

Clinical quality measures (CQMs) have historically relied on structured data elements in electronic health records; however, unstructured fields more reliably capture information needed to assess quality of care. Unstructured fields such as clinical notes and imaging results are particularly important for assessing diagnostic performance. Natural language processing (NLP) can efficiently capture nuanced information from unstructured fields to assess healthcare quality. This article describes three CQM projects that incorporated NLP in ways that are scalable for healthcare systemwide implementation.

Methods

The projects leveraged NLP to measure diagnostic performance related to venous thromboembolism, pneumonia, and pulmonary embolism, each of which is characterized by nonspecific symptoms and patient safety risk.

Results

NLP was used to extract patient symptoms, procedures, and findings from clinical notes in outpatient settings, emergency department visits, radiology reports, and discharge summaries. One NLP algorithm developed in the US Department of Veterans Affairs system performed comparably in an academic medical system. Other measures were successfully implemented in a data registry. Two of these CQMs were endorsed by the Centers for Medicare & Medicaid Services consensus-based entity. The third CQM set will be available as registry measures.

Conclusion

These projects demonstrate that CQMs leveraging NLP can achieve the high levels of reliability and validity necessary for national-level implementation. This will support efforts to assess diagnostic performance, which is often challenging to assess using structured data.

查看原文本刊更多论文

自然语言处理在临床质量测量诊断性能：从三个案例报告学习。

背景：临床质量测量（CQMs）历来依赖于电子健康记录中的结构化数据元素；然而，非结构化字段更可靠地获取评估护理质量所需的信息。诸如临床记录和成像结果等非结构化字段对于评估诊断性能尤其重要。自然语言处理（NLP）可以有效地从非结构化字段中捕获细微信息，以评估医疗保健质量。本文描述了三个CQM项目，它们以可扩展的方式将NLP集成到医疗保健系统范围的实现中。方法：该项目利用NLP来衡量与静脉血栓栓塞、肺炎和肺栓塞相关的诊断表现，每一种都具有非特异性症状和患者安全风险。结果：NLP用于提取患者的症状、程序和发现，从门诊设置的临床记录、急诊科访问、放射学报告和出院摘要。在美国退伍军人事务部系统中开发的一种NLP算法在学术医疗系统中表现相当。在数据注册中心中成功实现了其他措施。其中两个CQMs得到了医疗保险和医疗补助服务中心的认可。第三个CQM集将作为注册表度量提供。结论：这些项目表明，利用NLP的CQMs可以达到国家层面实施所需的高信度和效度水平。这将有助于评估诊断性能，这通常是使用结构化数据进行评估的挑战。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊