Journal of the American Medical Informatics Association最新文献

{"title":"A SNAPpy use of large language models: using large language models to classify treatment plans in pediatric acute otitis media.","authors":"Jessica J Pourian, Ben Michaels, Anh Vo, A Jay Holmgren, Augusto Garcia-Agundez, Valerie Flaherman","doi":"10.1093/jamia/ocaf170","DOIUrl":"https://doi.org/10.1093/jamia/ocaf170","url":null,"abstract":"<p><strong>Background and significance: </strong>Acute otitis media (AOM) is a leading cause of pediatric antibiotic overuse. Safety Net Antibiotic Prescriptions (SNAPs) are recommended for antibiotic stewardship but are difficult to identify due to lack of structured documentation.</p><p><strong>Objective: </strong>This study validates the accuracy of Versa, a GPT-4o based HIPAA-compliant large language model (LLM), to classify AOM treatment plans from physician notes.</p><p><strong>Methods: </strong>A retrospective cross-sectional study analyzed pediatric AOM encounters. Multiple prompting strategies were used to classify treatment plans and validated against a representative sample of manual reviews by 2 pediatricians. A locally fine-tuned model, Clinical-Longformer was also trained and tested against Versa and human review.</p><p><strong>Results: </strong>In total, 5707 encounters were included; 374 reviewed manually. Zero-shot accuracy was 97.8%; few-shot accuracy was 85%. Clinical-Longformer achieved 93.3% accuracy.</p><p><strong>Conclusion: </strong>Versa effectively identifies AOM treatment plans, providing a cost-efficient quality improvement tracking tool for prescription practice patterns in pediatric antibiotic stewardship efforts.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":""},"PeriodicalIF":4.6,"publicationDate":"2025-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145259774","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Should we synthesize more than we need: impact of synthetic data generation for high-dimensional cross-sectional medical data.","authors":"Lisa Pilgram, Samer El Kababji, Dan Liu, Khaled El Emam","doi":"10.1093/jamia/ocaf169","DOIUrl":"https://doi.org/10.1093/jamia/ocaf169","url":null,"abstract":"<p><strong>Objective: </strong>In medical research and education, generative artificial intelligence/machine learning (AI/ML) models to synthesize artificial medical data can enable the sharing of high-quality data while preserving the privacy of patients. Given that such data is often high-dimensional, a relevant consideration is whether to synthesize the entire dataset when only a task-relevant subset is needed. This study evaluates how the number of variables in training impacts fidelity, utility, and privacy of the synthetic data (SD).</p><p><strong>Material and methods: </strong>We used 12 cross-sectional medical datasets, defined a downstream task with corresponding core variables, and derived 6354 variants by adding adjunct variables to the core. SD was generated using 7 different generative models and evaluated for fidelity, downstream utility, and privacy. Mixed-effect models were used to assess the effect of adjunct variables on the respective evaluation metric, accounting for the medical dataset as a random component.</p><p><strong>Results: </strong>Fidelity was unaffected by the number of adjunct variables in 5/7 SDG models. Similarly, downstream utility remained stable in 6/7 (predictive task) and 5/7 (inferential task) SDG models. Where significant effects were observed, they were minimal, resulting, for example, in a 0.05 decrease in Area under the Receiver Operating Characteristic curve (AUROC) when adding 120 variables. Privacy was not impacted by the number of adjunct variables.</p><p><strong>Discussion: </strong>Our findings show that fidelity, utility, and privacy are preserved when generating a more comprehensive medical dataset than the task-relevant subset.</p><p><strong>Conclusion: </strong>Our findings support a cost-effective, utility, and privacy-preserving way of implementing SDG into medical research and education.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":""},"PeriodicalIF":4.6,"publicationDate":"2025-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145259724","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Zero-shot medical event prediction using a generative pretrained transformer on electronic health records.","authors":"Ekaterina Redekop, Zichen Wang, Rushikesh Kulkarni, Mara Pleasure, Aaron Chin, Hamid Reza Hassanzadeh, Brian L Hill, Melika Emami, William F Speier, Corey W Arnold","doi":"10.1093/jamia/ocaf160","DOIUrl":"https://doi.org/10.1093/jamia/ocaf160","url":null,"abstract":"<p><strong>Objectives: </strong>Longitudinal data in electronic health records (EHRs) represent an individual's clinical history through a sequence of codified concepts, including diagnoses, procedures, medications, and laboratory tests. Generative pretrained transformers (GPT) can leverage this data to predict future events. While fine-tuning of these models can enhance task-specific performance, it becomes costly when applied to many clinical prediction tasks. In contrast, a pretrained foundation model can be used in zero-shot forecasting setting, offering a scalable alternative to fine-tuning separate models for each outcome.</p><p><strong>Materials and methods: </strong>This study presents the first comprehensive analysis of zero-shot forecasting with GPT-based foundational models in EHRs, introducing a novel pipeline that formulates medical concept prediction as a generative modeling task. Unlike supervised approaches requiring extensive labeled data, our method enables the model to forecast the next medical event purely from a pretraining knowledge. We evaluate performance across multiple time horizons and clinical categories, demonstrating model's ability to capture latent temporal dependencies and complex patient trajectories without task supervision.</p><p><strong>Results: </strong>The model's performance in predicting the next medical concept was evaluated using precision and recall metrics, achieving an average top-1 precision of 0.614 and recall of 0.524. For 12 major diagnostic conditions, the model demonstrated strong zero-shot performance, achieving high true positive rates while maintaining low false positives.</p><p><strong>Discussion: </strong>We demonstrate the power of a foundational EHR GPT model in capturing diverse phenotypes and enabling robust, zero-shot forecasting of clinical outcomes. This capability highlights both its versatility across conditions like liver cancer and SLE, and its limitations in more ambiguous settings such as depression, while also revealing meaningful latent clinical structure.</p><p><strong>Conclusion: </strong>This capability enhances the versatility of predictive healthcare models and reduces the need for task-specific training, enabling more scalable applications in clinical settings.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":""},"PeriodicalIF":4.6,"publicationDate":"2025-10-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145253468","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"What patients want from healthcare chatbots: insights from a mixed-methods study.","authors":"Natalia S Dellavalle, Jessica R Ellis, Annie A Moore, Marlee Akerson, Matt Andazola, Eric G Campbell, Matthew DeCamp","doi":"10.1093/jamia/ocaf164","DOIUrl":"https://doi.org/10.1093/jamia/ocaf164","url":null,"abstract":"<p><strong>Objectives: </strong>To understand whether patients prefer chatbots for certain tasks in healthcare, and their motivations for doing so, recognizing that chatbots are already assisting patients with various healthcare tasks.</p><p><strong>Materials and methods: </strong>We conducted a mixed-methods study with patient-users of a healthcare system multi-task chatbot integrated in an electronic health record. We purposively oversampled by race or ethnicity to survey 617/3089 (response rate, 20.0%) chatbot users using de novo and validated survey items. We conducted semi-structured interviews with 46 patient-users and 2 chatbot developers between November 2022 and May 2024. We used modified grounded theory to analyze interviews, descriptive statistics and Chi-square tests to compare survey results, and mixed-methods techniques to integrate findings.</p><p><strong>Results: </strong>Patient-users preferred chatbots for administrative tasks to save providers' time, because of the chatbot availability, and to avoid unpleasant interactions. Some preferred to discuss sensitive tasks (such as mental health or gender-affirming care) with chatbots due to more privacy or anonymity and less embarrassment or judgment. Developer interviews corroborated this finding. Avoiding bias and using a preferred means of communication applied to all tasks. In surveys, patient-users were less likely to worry about being judged based on chatbot interactions (153/608, 25.2%) compared to interactions with a doctor (219/606, 36.1%) (P < .001). Patient-users preferred human clinicians for diagnostic tasks.</p><p><strong>Discussion: </strong>Patient-users appear to simultaneously prefer chatbots for simple tasks or sensitive ones, with diverse motivations. Whether chatbots best meet patient needs while balancing ethical tensions regarding access, privacy, judgment, and bias is unclear.</p><p><strong>Conclusion: </strong>Future chatbot design must accommodate different and diverse patient preferences.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":""},"PeriodicalIF":4.6,"publicationDate":"2025-10-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145240434","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Real-world evaluation of user engagement with an artificial intelligence-powered clinical trial application in oncology.","authors":"Tony K W Hung, Jun J Mao, Alan L Ho, Eric J Sherman, Mark Robson, Jae Park, Eytan M Stein, Gilad J Kuperman, David G Pfister","doi":"10.1093/jamia/ocaf129","DOIUrl":"10.1093/jamia/ocaf129","url":null,"abstract":"<p><strong>Objectives: </strong>This quality improvement study implemented and prospectively examined user engagement with an artificial intelligence (AI)-powered clinical trial knowledge management application at an NCI-designated comprehensive cancer center.</p><p><strong>Materials and methods: </strong>We prospectively auto-captured user engagement measures from July 1, 2022 to February 29, 2024. Measurement included: (1) event: an app interaction; (2) session: group of events within single setting; (3) engaged session: session longer than 10 s; (4) engagement time; (5) app downloads; (6) active user; and (7) stickiness: monthly active users per normalized total downloads. We analyzed the measures using time series and linear regression.</p><p><strong>Results: </strong>During a 20-month evaluation, the application supported 138 clinical trials, recorded 136 632 user interactions, including 2754 engaged sessions with an average engagement time of 6 min 31 s. Of 243 downloads, 228 (94%) users remained active, with an estimated stickiness score of 3.12 (SD 0.91), indicating sustained provider engagement.</p><p><strong>Discussion: </strong>This study provided insights into the feasibility and potential for integrating an AI-powered clinical trial knowledge management application into oncology workflows, with sustained engagement among providers over a 20-month period. High rates of active users and session stickiness suggest that such application offered meaningful utility in real-world clinical settings, underscoring the need for future studies to assess optimal integration strategies and impact on clinical trial accrual.</p><p><strong>Conclusion: </strong>This study addresses an important gap in the literature regarding the real-world integration of AI technologies in oncology care and offers valuable insights for future research and clinical practice.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"1562-1569"},"PeriodicalIF":4.6,"publicationDate":"2025-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12451935/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144734985","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Machine learning based prediction of medication adherence in heart failure using large electronic health record cohort with linkages to pharmacy-fill and neighborhood-level data.","authors":"Samrachana Adhikari, Tyrel Stokes, Xiyue Li, Yunan Zhao, Cassidy Fitchett, Nathalia Ladino, Steven Lawrence, Min Qian, Young S Cho, Carine Hamo, John A Dodson, Rumi Chunara, Ian M Kronish, Amrita Mukhopadhyay, Saul B Blecker","doi":"10.1093/jamia/ocaf162","DOIUrl":"https://doi.org/10.1093/jamia/ocaf162","url":null,"abstract":"<p><strong>Objective: </strong>While timely interventions can improve medication adherence, it is challenging to identify which patients are at risk of nonadherence at point-of-care. We aim to develop and validate flexible machine learning (ML) models to predict a continuous measure of adherence to guideline-directed medication therapies (GDMTs) for heart failure (HF).</p><p><strong>Materials and methods: </strong>We utilized a large electronic health record (EHR) cohort of 34,697 HF patients seen at NYU Langone Health with an active prescription for ≥1 GDMT between April 01, 2021 and October 31, 2022. The outcome was adherence to GDMT measured as proportion of days covered (PDC) at 6 months following a clinical encounter. Over 120 predictors included patient-, therapy-, healthcare-, and neighborhood-level factors guided by the World Health Organization's model of barriers to adherence. We compared performance of several ML models and their ensemble (superlearner) for predicting PDC with traditional regression model (OLS) using mean absolute error (MAE) averaged across 10-fold cross-validation, % increase in MAE relative to superlearner, and predictive-difference across deciles of predicted PDC.</p><p><strong>Results: </strong>Superlearner, a flexible nonparametric prediction approach, demonstrated superior prediction performance. Superlearner and quantile random forest had the lowest MAE (mean [95% CI] = 18.9% [18.7%-19.1%] for both), followed by MAEs for quantile neural network (19.5% [19.3%-19.7%]) and kernel support vector regression (19.8% [19.6%-20.0%]). Gradient boosted trees and OLS were the 2 worst performing models with 17% and 14% higher MAEs, respectively, relative to superlearner. Superlearner demonstrated improved predictive difference.</p><p><strong>Conclusion: </strong>This development phase study suggests potential of linked EHR-pharmacy data and ML to identify HF patients who will benefit from medication adherence interventions.</p><p><strong>Discussion: </strong>Fairness evaluation and external validation are needed prior to clinical integration.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":""},"PeriodicalIF":4.6,"publicationDate":"2025-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145201938","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Supporting public transit research in healthcare settings: testing a free, fast, and secure method for routing public transit from patient address to the point of care.","authors":"Sinan L Aktay, Ozan A Aktay, Samia Menon, Shuo Jim Huang, Rozalina G McCoy","doi":"10.1093/jamia/ocaf161","DOIUrl":"https://doi.org/10.1093/jamia/ocaf161","url":null,"abstract":"<p><strong>Objectives: </strong>Gaps in transportation, particularly public transit, are a significant barrier to accessible, high-quality healthcare. Health systems, payors, and regulatory bodies recognize the need to identify and address these gaps. However, clinical research examining public transportation accessibility and its impacts on healthcare utilization, outcomes, and costs remains limited. Existing tools used for studying public transit are generally non-HIPAA compliant, expensive, proprietary, and/or difficult to use. A tool addressing these concerns is needed to enable the incorporation of transportation variables into research and clinical care settings.</p><p><strong>Materials and methods: </strong>We developed and implemented a novel framework for building a public transit routing system that is comprised of free, publicly available data and offline software to maintain HIPAA compliance. The system consists of a transit router and a geocoder for converting addresses into coordinates.</p><p><strong>Results: </strong>A total of 463 879 out of 505 379 (∼91.8%) of Baltimore, Maryland, addresses were successfully routed to University of Maryland Medical Center in 24 hours of compute time. A significant portion of journeys consisted of walking (36% of median trip time) or using a transit vehicle (57.2%). Testing the router with varying random-access memory levels showed a plateau in routing speed between 12 and 20 GB. The geocoding approach is >90% consistent with a widely used but non-HIPAA compliant geocoder.</p><p><strong>Discussion: </strong>The methodology and step-by-step guidance shared in this study can allow researchers, public health professionals, non-for-profit agencies, and other stakeholders to efficiently, effectively, and safely incorporate public transportation information into their work.</p><p><strong>Conclusion: </strong>Public transportation routing using freely available data and software is possible in a HIPAA-compliant manner.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":""},"PeriodicalIF":4.6,"publicationDate":"2025-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145201900","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Enhancing Structured Team Communication in Acute Care Settings with Ambient AI Scribes.","authors":"Laleh Jalilian, Paul Lukac, Meghan Lane-Fall","doi":"10.1093/jamia/ocaf166","DOIUrl":"https://doi.org/10.1093/jamia/ocaf166","url":null,"abstract":"<p><strong>Objective: </strong>This perspective explores how ambient artificial intelligence (AI) scribes could support documentation and quality improvement (QI) of structured, team-based provider-to-provider communication in acute care settings.</p><p><strong>Background: </strong>In acute care settings, team-based discussions such as multidisciplinary rounds and handoffs are essential to the delivery of safe care. These discussions rely on standardized frameworks (eg, IPASS, checklists) to ensure consistent information transfer and shared understanding. Despite their importance, these verbal discussions are often incompletely documented or left undocumented in the electronic health record, leading to gaps in clinical narrative, difficulty in QI evaluation, and lost opportunities for organizational learning.</p><p><strong>Approach: </strong>We outline how ambient AI scribes could enhance documentation of team-based communication in daily rounding and handoff discussions. We examine key sociotechnical challenges, including workflow integration, multiprovider consent, surveillance concerns, and vendor collaboration. We describe our experience with proof-of-concept demonstrations as an early feasibility signal.</p><p><strong>Results: </strong>Ambient AI scribes are a promising tool for capturing structured team communication. Their use should be explored for its potential to improve documentation, support clinician well-being, and enable data-driven approaches to QI and communication fidelity assessments. Effective implementation requires workflow adaptations incorporating scribe output verification, transparent governance, and trust-building efforts to ensure clinician acceptance.</p><p><strong>Discussion: </strong>Ambient AI scribes represent a novel frontier in documentation of structured team discussions in acute care settings, with the potential to strengthen communication reliability and systems learning of these vital conversations. Future research should evaluate their impact on patient safety, workforce well-being, and patient outcomes in acute care settings.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":""},"PeriodicalIF":4.6,"publicationDate":"2025-09-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145151490","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Towards responsible artificial intelligence in healthcare-getting real about real-world data and evidence.","authors":"Eileen Koski, Amar Das, Pei-Yun Sabrina Hsueh, Anthony Solomonides, Amanda L Joseph, Gyana Srivastava, Carl Erwin Johnson, Joseph Kannry, Bilikis Oladimeji, Amy Price, Steven Labkoff, Gnana Bharathy, Baihan Lin, Douglas Fridsma, Lee A Fleisher, Monica Lopez-Gonzalez, Reva Singh, Mark G Weiner, Robert Stolper, Russell Baris, Suzanne Sincavage, Tristan Naumann, Tayler Williams, Tien Thi Thuy Bui, Yuri Quintana","doi":"10.1093/jamia/ocaf133","DOIUrl":"https://doi.org/10.1093/jamia/ocaf133","url":null,"abstract":"<p><strong>Background: </strong>The use of real-world data (RWD) in artificial intelligence (AI) applications for healthcare offers unique opportunities but also poses complex challenges related to interpretability, transparency, safety, efficacy, bias, equity, privacy, ethics, accountability, and stakeholder engagement.</p><p><strong>Methods: </strong>A multi-stakeholder expert panel comprising healthcare professionals, AI developers, policymakers, and other stakeholders was assembled. Their task was to identify critical issues and formulate consensus recommendations, focusing on the responsible use of RWD in healthcare AI. The panel's work involved an in-person conference and workshop and extensive deliberations over several months.</p><p><strong>Results: </strong>The panel's findings revealed several critical challenges, including the necessity for data literacy and documentation, the identification and mitigation of bias, privacy and ethics considerations, and the absence of an accountability structure for stakeholder management. To address these, the panel proposed a series of recommendations, such as the adoption of metadata standards for RWD sources, the development of transparency frameworks and instructional labels likened to \"nutrition labels\" for AI applications, the provision of cross-disciplinary training materials, the implementation of bias detection and mitigation strategies, and the establishment of ongoing monitoring and update processes.</p><p><strong>Conclusion: </strong>Guidelines and resources focused on the responsible use of RWD in healthcare AI are essential for developing safe, effective, equitable, and trustworthy applications. The proposed recommendations provide a foundation for a comprehensive framework addressing the entire lifecycle of healthcare AI, emphasizing the importance of documentation, training, transparency, accountability, and multi-stakeholder engagement.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":""},"PeriodicalIF":4.6,"publicationDate":"2025-09-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145151534","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Development and application of desiderata for automated clinical ordering.","authors":"Sameh N Saleh, Kevin B Johnson","doi":"10.1093/jamia/ocaf152","DOIUrl":"https://doi.org/10.1093/jamia/ocaf152","url":null,"abstract":"<p><strong>Introduction: </strong>Automation of clinical orders in electronic health records (EHRs) has the potential to reduce clinician burden and enhance patient safety. However, determining which orders are appropriate for automation requires a structured framework to ensure clinical validity, transparency, and safety.</p><p><strong>Objective: </strong>To develop and validate a framework of desiderata for assessing the appropriateness of automating clinical orders in EHRs and to demonstrate its operational value in a live health system dataset.</p><p><strong>Materials and methods: </strong>The study comprised 4 phases to move from concept generation to real-world demonstration. First, we conducted focus group analyses using ground theory to identify themes and developed desiderata informed by these themes and existing literature. We validated the desiderata by surveying clinicians at a single institution, presenting 10 use cases to and assessing perceived appropriateness, cognitive support, and patient safety using a 4-point Likert scale. Survey results were compared to a priori appropriateness designations using t-tests. To evaluate operational impact, we analyzed one year of order-based alerts and orders (1.4 million firings alert and 44.1 million orders, respectively) using filtering rules and association rule mining to identify candidate orders for automation and their impact.</p><p><strong>Results: </strong>We identified 8 desiderata for automated order appropriateness: logical consistency, data provenance, order transparency, context permanence, monitoring plans, trigger consistency, care team empowerment, and system accountability. Use cases deemed appropriate based on these criteria received significantly higher scores for appropriateness (3.13 ± 0.84 vs 2.30 ± 0.99), cognitive support (3.08 ± 0.82 vs 2.25 ± 0.94), and patient safety (3.08 ± 0.86 vs 2.21 ± 0.98) (all P < .001) compared to those considered inappropriate. Operational analysis revealed an alert firing 19 109 times annually, with a 96% signed order rate, where automation could save an estimated 26.5 provider hours per year. Additionally, an association rule with 16 628 occurrences (68.4% confidence) suggested automation could save 15.8 hours annually and yield 8000 additional appropriate orders.</p><p><strong>Discussion: </strong>The desiderata align with clinician perceptions and provide a structured approach for evaluating automated orders. Our findings highlight the potential for automation of certain clinical orders to improve cognitive support while maintaining patient safety.</p><p><strong>Conclusion: </strong>Healthcare systems should use these desiderata, coupled with data mining techniques, to systematically identify and govern appropriate automated orders. Further research is needed to validate operational scalability.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":""},"PeriodicalIF":4.6,"publicationDate":"2025-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145126405","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}