Applied Clinical Informatics最新文献

{"title":"Usage of and Satisfaction with Artificial Intelligence-Generated Draft Replies to Patient Portal Messages.","authors":"Aidan P Crowley, Asaf Hanish, Jason Lubken, Michael Becker, Carole Rosen, Jeffrey Moon, Susan H Regli","doi":"10.1055/a-2893-7363","DOIUrl":"10.1055/a-2893-7363","url":null,"abstract":"<p><strong>Objectives: </strong>This study aimed to describe the usage and perceptions of physicians and nurses in primary and specialty care on the acceptability, usability, and quality of artificial intelligence (AI)-generated draft replies to patient portal messages.</p><p><strong>Methods: </strong>We conducted a pilot study of 80 users across seven clinics within a large academic health system in Pennsylvania. Epic's Augmented Response Technology (ART) used a large language model to present clinicians with draft replies to patient messages. To measure usage, we calculated the frequency with which replies were initiated with ART-generated drafts, as well as the overlap between drafts and replies sent to patients. To measure acceptability, usability, and quality, we developed and disseminated a 15-item survey.</p><p><strong>Results: </strong>Voluntary pilot participants included 80 users (36 physicians, 25 nurses [registered nurses/licensed practice nurses], 10 nurse practitioners, 9 medical assistants). The mean rate of starting with an ART draft was 20.2%, and 40% of replies exhibited little to no editing of the draft. Of the 52 survey respondents, 66% (<i>n</i> = 33) agreed that AI-generated drafts were useful, and 46% (<i>n</i> = 23) agreed that ART improved the quality of replies to patient messages. Across survey questions, a higher percentage of nurses reported favorable opinions of the technology than did physicians. In free-response questions, the most common themes were that the message content was helpful (28.3%, <i>n</i> = 13) and that it reduced perceived effort or time (23.9%, <i>n</i> = 11), but that drafts occasionally contained incorrect or inappropriate content (34.9%, <i>n</i> = 15) or too often suggested an appointment (27.9%, <i>n</i> = 12).</p><p><strong>Conclusion: </strong>In this voluntary pilot study, AI-generated draft replies to patient portal messages were generally rated as acceptable and useful, with nurses and primary care clinicians more often reporting acceptance of the technology than physicians or specialty care clinicians. Scaling efforts should continue to monitor acceptability and effects on cognitive burden, in addition to safety.</p>","PeriodicalId":48956,"journal":{"name":"Applied Clinical Informatics","volume":"17 3","pages":"510-517"},"PeriodicalIF":2.7,"publicationDate":"2026-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13309259/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148340129","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Optimizing Clinical Decision Support for Antibiotic Prescribing in Pediatric Acute Respiratory Tract Infections: A Usability Study.","authors":"Janet Lee, Dina Abdel-Rahman, Emma Kelly, Praveena Korakuti, Erin Tagai","doi":"10.1055/a-2906-3186","DOIUrl":"10.1055/a-2906-3186","url":null,"abstract":"<p><strong>Background: </strong>Inappropriate antibiotic prescribing contributes to antimicrobial resistance. Antimicrobial stewardship (AMS) programs increasingly rely on electronic health record (EHR)-based clinical decision support (CDS) alerts to guide evidence-based care, yet poorly designed alerts can increase cognitive burden and contribute to burnout. Usability testing offers a structured approach to improving alert design and adoption.</p><p><strong>Objectives: </strong>This study aimed to evaluate and iteratively refine interruptive CDS alerts developed to support appropriate antibiotic prescribing for pediatric acute respiratory tract infections (ARTIs) within an urban academic health system as part of a larger implementation project.</p><p><strong>Methods: </strong>Twelve providers participated in four waves of virtual usability testing using scripted clinical scenarios reflecting common workflows. Participants interacted with three clinical alerts and one design-focused scenario. Quantitative outcomes included task effectiveness, efficiency, satisfaction, and perceived difficulty. Qualitative feedback was gathered via think-aloud techniques and synthesized using a \"rainbow spreadsheet.\" Iterative modifications were implemented between waves until thematic saturation was achieved.</p><p><strong>Results: </strong>Task completion across scenarios was high (83.3%). Effectiveness and efficiency improved over waves, and satisfaction increased when alerts embedded actionable tasks such as ordering tests, adding diagnoses, or switching antibiotics. Participants emphasized the importance of clear, concise text and intuitive defaults. Including links to evidence-based guidelines improved trust. Alert refinements reduced cognitive load and improved workflow alignment.</p><p><strong>Conclusion: </strong>Rapid, iterative usability testing substantially improved the clarity and workflow integration of interruptive CDS alerts. Incorporating usability evaluation into CDS stewardship efforts can enhance adoption, reduce alert fatigue, and strengthen implementation of evidence-based practices.</p>","PeriodicalId":48956,"journal":{"name":"Applied Clinical Informatics","volume":" ","pages":"615-624"},"PeriodicalIF":2.7,"publicationDate":"2026-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13391217/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148382808","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Digital Identity as Patient Safety Infrastructure.","authors":"Henry Bair","doi":"10.1055/a-2880-9387","DOIUrl":"10.1055/a-2880-9387","url":null,"abstract":"","PeriodicalId":48956,"journal":{"name":"Applied Clinical Informatics","volume":"17 3","pages":"412-414"},"PeriodicalIF":2.7,"publicationDate":"2026-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13287922/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148176963","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"LOINC-Related Clinical Terminology Management in an Academic Medical Center Using Human in the Loop Machine Learning.","authors":"Thomas R Campion, Tru V Tran, Adam D Cheriff, J Travis Gossey, Curtis L Cole","doi":"10.1055/a-2880-6252","DOIUrl":"10.1055/a-2880-6252","url":null,"abstract":"<p><strong>Objective: </strong>Many institutions have reported poor mapping of laboratory results to Logical Observation Identifiers Names and Codes (LOINC). A human-in-the-loop machine learning approach for clinical terminology management may help.</p><p><strong>Methods: </strong>Since 2002, Weill Cornell Medicine has maintained TruData, an application between reference laboratories and the electronic health record (EHR) system that predicts LOINC for unknown results using Bayesian networks, among other techniques. Staff have used TruData to adjudicate LOINC predictions and manage EHR clinical content.</p><p><strong>Results: </strong>From 2002 to 2025, TruData identified 244,000 unique laboratory result concepts and enabled LOINC determination for 193,000 (79%). As of 2025, the EHR system contained 50,000 unique laboratory result components, 42,000 (84%) of which had a LOINC assignment. The EHR team used TruData to maintain results and procedures, making available in the EHR the subset of TruData content observed clinically. Reasons for result components lacking LOINC included LOINC not yet available (i.e., new result) and components not warranting LOINC (e.g., \"Comment,\" \"Reflex Information\"). This case report is among the first to describe human-in-the-loop machine learning for clinical terminology management in use for more than 20 years at an academic medical center.</p><p><strong>Conclusion: </strong>Practitioners elsewhere may find human-in-the-loop machine learning and TruData valuable for clinical terminology management.</p>","PeriodicalId":48956,"journal":{"name":"Applied Clinical Informatics","volume":" ","pages":"443-449"},"PeriodicalIF":2.7,"publicationDate":"2026-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13286108/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148007237","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Beyond the Model: Practical Insights from Monitoring Predictive Models across Diverse Clinical Workflows.","authors":"John J Hanna, Christopher R Dennis, Andrew O Johnson, Lauren N Cooper, Christopher S Evans, Christoph Ulrich Lehmann, Richard J Medford","doi":"10.1055/a-2876-1059","DOIUrl":"10.1055/a-2876-1059","url":null,"abstract":"<p><strong>Objectives: </strong>To support the artificial intelligence (AI) lifecycle in an integrated academic health system, we implemented a modular monitoring system to oversee electronic health record (EHR) vendor-provided clinical predictive models. This case study describes the lessons learned using this modular system to support sustainable oversight of the deployed predictive models across their lifecycle.</p><p><strong>Methods: </strong>We developed a modular monitoring system using automated data pipelines refreshed daily to support longitudinal oversight of four clinical predictive models developed by our EHR vendor (Epic Systems). An interactive monitoring application was designed to bridge the technical-operational gap for stakeholders involved in AI lifecycle decisions. Interactive reports include automated and assisted threshold-independent and threshold-dependent performance measures, flag rates, and problem-specific metrics crafted to support operational decision-making.</p><p><strong>Results: </strong>Through the lenses of people, process, and technology, we describe lessons learned from the real-world implementation of a monitoring system. Effective oversight required clearly defined dyadic ownership (technical and operational) and embedded monitoring activities within existing AI governance decision-making processes across all AI lifecycle stages. Monitoring extended beyond performance drift to include alert burden, workflow alignment, and problem-level signals. Modularity enabled rapid adaptation as models and workflows evolved. Silent alert simulations proved valuable for selecting preimplementation thresholds that aligned with clinical capacity and workflows but were limited in predicting postimplementation performance when the models faced real-world human-alert interactions.</p><p><strong>Conclusion: </strong>Implementing a modular, governance-aligned monitoring system enabled sustained oversight of vendor-provided clinical predictive models and shifted monitoring beyond traditional technical performance metrics toward user-centric and operationally meaningful measures. These findings highlight the importance of integrated monitoring infrastructure as a core component of responsible AI governance in clinical settings.</p>","PeriodicalId":48956,"journal":{"name":"Applied Clinical Informatics","volume":" ","pages":"384-391"},"PeriodicalIF":2.7,"publicationDate":"2026-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13287939/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147989674","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Effect of Speech Recognition Software on Provider Documentation Characteristics Within an Electronic Health Record System.","authors":"McClane Howland, Zoe Lockhart, Sunit P Jariwala","doi":"10.1055/a-2882-9924","DOIUrl":"10.1055/a-2882-9924","url":null,"abstract":"<p><strong>Objectives: </strong>This study evaluates the effect of speech recognition software Dragon Medical One (DMO) on physician documentation burden and EHR workflows using Epic Signal usage metrics.</p><p><strong>Methods: </strong>We conducted a longitudinal cohort study of physicians at a single academic medical center between December 2019 and November 2023. We compared 291 DMO adopters with 2,828 non-adopters using a staggered-adoption difference-in-differences design with two-way fixed effects and standard errors clustered at the physician level. We analyzed 23 Epic Signal metrics spanning documentation methods, time metrics, and workflow efficiency. We applied Benjamini-Hochberg correction for multiple comparisons and validated parallel trends assumptions through event study analysis. Heterogeneous effects were assessed through interaction models with demeaned baseline physician characteristics.</p><p><strong>Results: </strong>Among 3,119 physicians (291 adopters, 2,828 non-adopters), DMO adoption shifted documentation composition from manual typing to voice recognition (+2.67 percentage points; q < 0.001) with corresponding decreases in manual (-1.98 percentage points; q = 0.002) and SmartTool-based documentation (-1.89 percentage points; q = 0.004). Time in notes per appointment decreased by 0.50 minutes (q = 0.004) and time outside scheduled hours decreased by 2.74 minutes per day (q = 0.009). Of 23 outcomes 6 survived false discovery rate correction at q < 0.05. Event study analyses confirmed parallel pre-treatment trends for 22 of 23 outcomes. Physicians with higher baseline documentation time experienced larger reductions in time in notes per day and time in system per day following DMO adoption (interaction q < 0.01).</p><p><strong>Conclusion: </strong>Adoption of speech recognition software was associated with a shift from manual to voice-based documentation, reduced time in notes per appointment, and reduced time outside scheduled hours. Effects were most pronounced among physicians with the highest baseline documentation burden, suggesting that those with the greatest room for improvement benefit most from this technology.</p>","PeriodicalId":48956,"journal":{"name":"Applied Clinical Informatics","volume":" ","pages":"415-422"},"PeriodicalIF":2.7,"publicationDate":"2026-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13287925/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148036794","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Health Information Exchange-Enabled Investigation of Health-Related Social Needs Referral Process Retention and Fulfillment.","authors":"Paulina S Sockolow, Yasemin Algur, Daniel Vader, Edgar Y Chou","doi":"10.1055/a-2917-6605","DOIUrl":"10.1055/a-2917-6605","url":null,"abstract":"<p><strong>Background: </strong>Health-related social needs (HRSN) significantly influence health outcomes, yet healthcare organizations face persistent challenges tracking referrals to community-based organizations and interpreting referral outcomes across fragmented clinical and social care systems. Prior studies report low referral fulfillment rates, but much of this evidence derives from single organizations, manual data collection, or incentivized documentation workflows, limiting insight into how information infrastructure shapes what is observable at scale.</p><p><strong>Objective: </strong>This study aimed to evaluate how HRSN referral initiation, documented outcomes, and time-to-fulfillment are captured when electronic health record (EHR) data and closed-loop HRSN platform data are integrated through a statewide health information exchange, and to identify informatics strengths and limitations affecting cross-system observability of referral processes.</p><p><strong>Methods: </strong>We conducted a retrospective observational study of 1,628 adult patients from a federally qualified health center and three health systems. Linked EHR and HRSN platform data were used to examine referral initiation, documented fulfillment, and time to fulfillment across resource types.</p><p><strong>Results: </strong>Patients received an average of two referrals, with approximately 80% addressing basic needs. Fewer than 3% were documented as fulfilled, reflecting constrained documentation workflows rather than confirmed absence of service delivery. Short-term needs were more likely to transition to documented outcome states and had shorter documented time-to-fulfillment, whereas long-term needs-particularly housing-exhibited wide variability and documentation patterns consistent with workflow-driven closure behavior.</p><p><strong>Conclusion: </strong>Instead of examining referral fulfillment as a performance metric, the findings suggest how socio-technical infrastructure, workflow design, and documentation incentives affect what becomes observable in secondary use of HRSN data.</p>","PeriodicalId":48956,"journal":{"name":"Applied Clinical Informatics","volume":"17 3","pages":"680-690"},"PeriodicalIF":2.7,"publicationDate":"2026-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13427388/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148649671","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Electronic Strategies for Tailored Exercise to Prevent Falls: Evaluating Implementation in Primary Care.","authors":"Jenna Reisler, Patricia Dykes, Nancy K Latham, Biai Digbeu, Efstathia Polychronopoulou, Michael Sainlaire, Tien Thai, Mackenzie Kiesman, Erin Hommel","doi":"10.1055/a-2896-8054","DOIUrl":"10.1055/a-2896-8054","url":null,"abstract":"<p><strong>Background: </strong>Falls are the leading cause of injury and injury-related death among older adults. Clinical decision support (CDS) may improve clinician referral and patient access to fall prevention exercise.</p><p><strong>Objectives: </strong>We evaluated implementation of a CDS intervention linking high-risk older adults to gait and balance exercises and identified patient and clinician factors associated with success.</p><p><strong>Methods: </strong>Using the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework, we evaluated implementation year 1 after a 3-month wash-in period at clinics in an academic health system. Community-dwelling adults aged ≥ 65 seen in person by a physician or advanced practice provider were eligible. We implemented standardized fall risk screening, a CDS alert for positive screens, linked electronic order sets for gait and balance exercise, and preconfigured documentation text. We assessed reach, adoption, implementation, and maintenance; the primary outcome was connection to exercise via order sets or confirmed ongoing gait and balance exercise. Hierarchical logistic regression evaluated associations between patient and clinician characteristics and odds of this outcome.</p><p><strong>Results: </strong>The CDS alert reached 5,458 of 5,891 eligible patients (92.6%); 1,664 (30.5%) were connected to exercise. Male patient sex (odds ratio [OR]: 0.82; 95% confidence interval [CI]: 0.70-0.95) was associated with lower odds of connection, while multiple recent falls or a fall with injury increased odds (OR: 1.24; 95% CI: 1.02-1.49) compared with self-reported fear of falling. Variation in adoption by clinician was substantial (intraclass correlation coefficient approximately, 30%). No significant differences were found by patient age, race/ethnicity, or insurance, nor for clinician department or qualification.</p><p><strong>Conclusion: </strong>CDS can identify older adults at high fall risk and prompt referral to gait and balance exercise. However, adoption was modest and varied by clinician, with patient sex and recent fall history influencing uptake. Targeted strategies to improve clinician adoption and address patient biases may enhance implementation.</p>","PeriodicalId":48956,"journal":{"name":"Applied Clinical Informatics","volume":" ","pages":"642-651"},"PeriodicalIF":2.7,"publicationDate":"2026-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13400137/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148309247","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Algorithmic Identification of the Daily Primary Frontline Clinician during Inpatient Medicine Encounters Using EHR Audit Log Data.","authors":"Laura R Baratta, Joanne Wang, Bailey W Osweiler, Daphne Lew, Elise Eiden, Thomas Kannampallil, Sunny S Lou","doi":"10.1055/a-2903-0478","DOIUrl":"10.1055/a-2903-0478","url":null,"abstract":"<p><strong>Background: </strong>Interprofessional teams are central to high-quality patient care. However, identifying the daily frontline clinician who performs direct care-related tasks for a patient requires labor-intensive methodologies. Although electronic health record (EHR) audit logs offer a scalable alternative, their use for identifying frontline clinicians is underdeveloped.</p><p><strong>Objectives: </strong>This study aimed to develop and validate an algorithm utilizing EHR audit logs to identify the daily frontline clinician per patient-day of an inpatient medicine encounter.</p><p><strong>Methods: </strong>This was a cross-sectional cohort study of adult inpatient medicine encounters across 12 hospitals in one health system using a shared EHR. Admissions between February 1, 2023, and April 30, 2023, with a length of stay of at least 3 days and without an intensive care unit (ICU) admission were included. Four algorithm iterations were designed to identify the attending physician, resident, or advanced practice provider (APP) serving as the daily frontline clinician. The performance of each algorithm was compared with manual chart review on 1,401 patient-days from 246 randomly sampled patient encounters. Algorithm accuracy versus chart review was compared using McNemar's test with Bonferroni-adjusted <i>p</i>-values.</p><p><strong>Results: </strong>The best-performing algorithm correctly identified the daily frontline clinician on 91% of patient-days (1,268/1,401), outperforming algorithm 1, which selected the clinician who performed the most actions (78% accuracy, 1,098/1,401, <i>p</i> < 0.001). Algorithm errors were attributable to misidentified specialties and ambiguity on days with transitions of care or shared responsibilities between clinicians. The best-performing algorithm was applied to the entire cohort (5,801 encounters and 34,001 patient-days), where it identified attending physicians, resident physicians, and APPs as the daily frontline clinicians for 26,750 (79%), 3,106 (9%), and 4,145 (12%) of patient-days, respectively. Encounters had a median of 1 (interquartile range [IQR]: 0-2) handoff between frontline clinicians.</p><p><strong>Conclusion: </strong>We developed a scalable, audit log-based algorithm to determine the daily frontline clinician with high accuracy compared with manual chart review.</p>","PeriodicalId":48956,"journal":{"name":"Applied Clinical Informatics","volume":"17 3","pages":"540-548"},"PeriodicalIF":2.7,"publicationDate":"2026-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13337303/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148398882","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Optimizing GPT-5 for Operation-Procedure-Code-Extraction from Operative Reports in Meningioma Surgery: Feasibility and Comparison of Context-Enhancements.","authors":"Sebastian Lehmann, Florian Wilhelmy, Frederic V Schwaebe, Nikolaus von Dercks, Erdem Güresir, Johannes S Wach","doi":"10.1055/a-2913-8450","DOIUrl":"10.1055/a-2913-8450","url":null,"abstract":"<p><strong>Objective: </strong>In our recent study we showed that GPT's ability to perform OPS-code extraction from operational reports was equivalent to coding neurosurgeons. In this study we aim to evaluate the effect of context enhancement on GPT-5's code extraction abilities.</p><p><strong>Methods: </strong>We provided OpenAIs GPT-5 with 100 operational reports (OR) of patients that underwent meningioma surgery. A chat prompt was generated instructing GPT to generate the correct OPS coding. Five groups were formed and provided with different context-enhancing modalities: 1. No additional context (GPT-5s), 2. the current OPS-catalogue (GPT-5o), 3. specified rules on Meningioma coding (GPT-5r), 4. code-triggering sample phrases based on 50 additional ORs (GPT-5e) and 5. all enhancements combined (GPT-5c). We analyzed code-extraction abilities, mistakes and hallucinations.</p><p><strong>Results: </strong>Non context enhanced GPT-5s showed the lowest rate of correct coding (44%) and highest number of hallucinations (105) and total mistakes (132). GPT-5o showed identical accuracy to GPT-5s (44%, <i>p</i> = 1.0), but fewer hallucinations (51, <i>p</i> = 0.002) and mistakes (78, <i>p</i> = 0.008). All other models were significantly superior to GPT-5s and GPT-5o in accuracy (GPT-5r 79%, GPT-5e 70%, GPT-5c 86%, <i>p</i> < 0.001), hallucinations (GPT-5r 8, GPT-5e 2, GPT-5c 6, <i>p</i> < 0.001) and mistakes (GPT-5r 21, GPT-5e 35, GPT-5c 14, <i>p</i> < 0.001). Highest coding accuracy was achieved by GPT-5c (GPT-5c vs GPT-5r, <i>p</i> = 0.143, GPT-5c vs GPT5e <i>p</i> = 0.008). GPT-5r and GPT-5e performed equally regarding accuracy, GPT-5r produced fewer mistakes (<i>p</i> = 0.034), while GPT-5e hallucinated less (<i>p</i> = 0.057).</p><p><strong>Conclusion: </strong>We show, that specific context enhancement significantly improves GPT-5s ability to perform OPS-code extraction from operational reports, while significantly lowering mistake- and hallucination rates.</p>","PeriodicalId":48956,"journal":{"name":"Applied Clinical Informatics","volume":"17 3","pages":"625-632"},"PeriodicalIF":2.7,"publicationDate":"2026-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13395532/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148580934","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}