Journal of the American Medical Informatics Association最新文献

{"title":"Expectations of healthcare AI and the role of trust: understanding patient views on how AI will impact cost, access, and patient-provider relationships.","authors":"Paige Nong, Molin Ji","doi":"10.1093/jamia/ocaf031","DOIUrl":"10.1093/jamia/ocaf031","url":null,"abstract":"<p><strong>Objectives: </strong>Although efforts to effectively govern AI continue to develop, relatively little work has been done to systematically measure and include patient perspectives or expectations of AI in governance. This analysis is designed to understand patient expectations of healthcare AI.</p><p><strong>Materials and methods: </strong>Cross-sectional nationally representative survey of US adults fielded from June to July of 2023. A total of 2039 participants completed the survey and cross-sectional population weights were applied to produce national estimates.</p><p><strong>Results: </strong>Among US adults, 19.55% expect AI to improve their relationship with their doctor, while 19.4% expect it to increase affordability and 30.28% expect it will improve their access to care. Trust in providers and the healthcare system are positively associated with expectations of AI when controlling for demographic factors, general attitudes toward technology, and other healthcare-related variables.</p><p><strong>Discussion: </strong>US adults generally have low expectations of benefit from AI in healthcare, but those with higher trust in their providers and health systems are more likely to expect to benefit from AI.</p><p><strong>Conclusion: </strong>Trust and provider relationships should be key considerations for health systems as they create their AI governance processes and communicate with patients about AI tools. Evidence of patient benefit should be prioritized to preserve or promote trust.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"795-799"},"PeriodicalIF":4.7,"publicationDate":"2025-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12012342/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143558514","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A call for the informatics community to define priority practice and research areas at the intersection of climate and health: report from 2023 mini-summit.","authors":"Titus Schleyer, Manijeh Berenji, Monica Deck, Hana Chung, Joshua Choi, Theresa A Cullen, Timothy Burdick, Amanda Zaleski, Kelly Jean Thomas Craig, Oluseyi Fayanju, Muhammad Muinul Islam","doi":"10.1093/jamia/ocae292","DOIUrl":"10.1093/jamia/ocae292","url":null,"abstract":"<p><strong>Objective: </strong>Although biomedical informatics has multiple roles to play in addressing the climate crisis, collaborative action and research agendas have yet to be developed. As a first step, AMIA's new Climate, Health, and Informatics Working Group held a mini-summit entitled Climate and health: How can informatics help? during the AMIA 2023 Fall Symposium to define an initial set of areas of interest and begin mobilizing informaticians to confront the urgent challenges of climate change.</p><p><strong>Materials and methods: </strong>The AMIA Climate, Health, and Informatics Working Group (at the time, an AMIA Discussion Forum), the International Medical Informatics Association (IMIA), the International Academy of Health Sciences Informatics (IAHSI), and the Regenstrief Institute hosted a mini-summit entitled Climate and health: How can informatics help? on November 11, 2023, during the AMIA 2023 Annual Symposium (New Orleans, LA, USA). Using an affinity diagramming approach, the mini-summit organizers posed 2 questions to ∼50 attendees (40 in-person, 10 virtual).</p><p><strong>Results: </strong>Participants expressed a broad array of viewpoints on actions that can be undertaken now and areas needing research to support future actions. Areas of current action ranged from enhanced education to expanded telemedicine to assessment of community vulnerability. Areas of research ranged from emergency preparedness to climate-specific clinical coding to risk prediction models.</p><p><strong>Discussion: </strong>The mini-summit was intended as a first step in helping the informatics community at large set application and research priorities for climate, health, and informatics.</p><p><strong>Conclusion: </strong>The working group will use these perspectives as it seeks further input, and begins to establish priorities for climate-related biomedical informatics actions and research.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"971-979"},"PeriodicalIF":4.7,"publicationDate":"2025-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12012334/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143626625","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"High-performance automated abstract screening with large language model ensembles.","authors":"Rohan Sanghera, Arun James Thirunavukarasu, Marc El Khoury, Jessica O'Logbon, Yuqing Chen, Archie Watt, Mustafa Mahmood, Hamid Butt, George Nishimura, Andrew A S Soltan","doi":"10.1093/jamia/ocaf050","DOIUrl":"10.1093/jamia/ocaf050","url":null,"abstract":"<p><strong>Objective: </strong>screening is a labor-intensive component of systematic review involving repetitive application of inclusion and exclusion criteria on a large volume of studies. We aimed to validate large language models (LLMs) used to automate abstract screening.</p><p><strong>Materials and methods: </strong>LLMs (GPT-3.5 Turbo, GPT-4 Turbo, GPT-4o, Llama 3 70B, Gemini 1.5 Pro, and Claude Sonnet 3.5) were trialed across 23 Cochrane Library systematic reviews to evaluate their accuracy in zero-shot binary classification for abstract screening. Initial evaluation on a balanced development dataset (n = 800) identified optimal prompting strategies, and the best performing LLM-prompt combinations were then validated on a comprehensive dataset of replicated search results (n = 119 695).</p><p><strong>Results: </strong>On the development dataset, LLMs exhibited superior performance to human researchers in terms of sensitivity (LLMmax = 1.000, humanmax = 0.775), precision (LLMmax = 0.927, humanmax = 0.911), and balanced accuracy (LLMmax = 0.904, humanmax = 0.865). When evaluated on the comprehensive dataset, the best performing LLM-prompt combinations exhibited consistent sensitivity (range 0.756-1.000) but diminished precision (range 0.004-0.096) due to class imbalance. In addition, 66 LLM-human and LLM-LLM ensembles exhibited perfect sensitivity with a maximal precision of 0.458 with the development dataset, decreasing to 0.1450 over the comprehensive dataset; but conferring workload reductions ranging between 37.55% and 99.11%.</p><p><strong>Discussion: </strong>Automated abstract screening can reduce the screening workload in systematic review while maintaining quality. Performance variation between reviews highlights the importance of domain-specific validation before autonomous deployment. LLM-human ensembles can achieve similar benefits while maintaining human oversight over all records.</p><p><strong>Conclusion: </strong>LLMs may reduce the human labor cost of systematic review with maintained or improved accuracy, thereby increasing the efficiency and quality of evidence synthesis.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"893-904"},"PeriodicalIF":4.7,"publicationDate":"2025-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12012331/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143677361","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Associations of perceived discrimination with health outcomes and health disparities in the All of Us cohort.","authors":"Vincent Lam, Sonali Gupta, I King Jordan, Leonardo Mariño-Ramírez","doi":"10.1093/jamia/ocaf040","DOIUrl":"10.1093/jamia/ocaf040","url":null,"abstract":"<p><strong>Objectives: </strong>The goal of this study was to investigate the association of perceived discrimination with health outcomes and disparities.</p><p><strong>Materials and methods: </strong>The study cohort consists of 60 180 participants from the 4 largest self-identified race and ethnicity (SIRE) groups in the All of Us Research Program participant body: Asian (1291), Black (4726), Hispanic (5336), and White (48 827). A perceived discrimination index (PDI) was derived from participant responses to the \"Social Determinants of Health\" survey, and the All of Us Researcher Workbench was used to analyze associations and mediation effects of PDI and SIRE with 1755 diseases.</p><p><strong>Results: </strong>The Black SIRE group has the greatest median PDI, followed by the Asian, Hispanic, and White groups. The Black SIRE group shows the greatest number of diseases with elevated risk relative to the White reference group, followed by the Hispanic and Asian groups. Perceived discrimination index was found to be positively and significantly associated with 489 out of 1755 (27.86%) diseases. \"Mental Disorders\" is the disease category with the greatest proportion of diseases positively and significantly associated with PDI: 59 out of 72 (81.94%) diseases. Mediation analysis showed that PDI mediates 69 out of 351 (19.66%) Black-White disease disparities.</p><p><strong>Discussion: </strong>Perceived discrimination is significantly associated with risk for numerous diseases and mediates Black-White disease disparities in the All of Us participant cohort.</p><p><strong>Conclusion: </strong>This work highlights the role of discrimination as an important social determinant of health and provides a means by which it can be quantified and modeled on the All of Us platform.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"823-834"},"PeriodicalIF":4.7,"publicationDate":"2025-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12012377/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143617674","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Principles and implementation strategies for equitable and representative academic partnerships in global health informatics research.","authors":"Elizabeth Campbell, Oliver J Bear Don't Walk, Hamish Fraser, Judy Gichoya, Kavishwar B Wagholikar, Andrew S Kanter, Felix Holl, Sansanee Craig","doi":"10.1093/jamia/ocaf015","DOIUrl":"10.1093/jamia/ocaf015","url":null,"abstract":"<p><strong>Objective: </strong>Developing equitable, sustainable informatics solutions is key to scalability and long-term success for projects in the global health informatics (GHI) domain. This paper presents key strategies for incorporating principles of health equity in the GHI project lifecycle.</p><p><strong>Materials and methods: </strong>The American Medical Informatics Association (AMIA) GHI Working Group organized a collaborative workshop at the 2023 AMIA Annual Symposium that included the presentation of five case studies of how principles of health equity have been incorporated into projects situated in low-and-middle-income countries and with Indigenous communities in the U.S. and best practices for operationalizing these principles into other informatics projects.</p><p><strong>Results: </strong>We present five principles: (1) Inclusion and Participation in Ethical, Sustainable Collaborations; (2) Engaging Community-Based Participatory Research Approaches; (3) Stakeholder Engagement; (4) Scalability and Sustainability; (5) Representation in Knowledge Creation, along with strategies that informatics researchers may use to incorporate these principles into their work.</p><p><strong>Discussion: </strong>Presented case studies and subsequent focus groups yielded key concepts and strategies to promote health equity that may be operationalized across GHI projects.</p><p><strong>Conclusion: </strong>Equitable, sustainable, and scalable GHI projects require intentional integration of community and stakeholder perspectives in project development, implementation, and knowledge creation processes.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"958-963"},"PeriodicalIF":4.7,"publicationDate":"2025-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12012363/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143411280","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Development and validation of a multi-stage self-supervised learning model for optical coherence tomography image classification.","authors":"Sungho Shim, Min-Soo Kim, Che Gyem Yae, Yong Koo Kang, Jae Rock Do, Hong Kyun Kim, Hyun-Lim Yang","doi":"10.1093/jamia/ocaf021","DOIUrl":"10.1093/jamia/ocaf021","url":null,"abstract":"<p><strong>Objective: </strong>This study aimed to develop a novel multi-stage self-supervised learning model tailored for the accurate classification of optical coherence tomography (OCT) images in ophthalmology reducing reliance on costly labeled datasets while maintaining high diagnostic accuracy.</p><p><strong>Materials and methods: </strong>A private dataset of 2719 OCT images from 493 patients was employed, along with 3 public datasets comprising 84 484 images from 4686 patients, 3231 images from 45 patients, and 572 images. Extensive internal, external, and clinical validation were performed to assess model performance. Grad-CAM was employed for qualitative analysis to interpret the model's decisions by highlighting relevant areas. Subsampling analyses evaluated the model's robustness with varying labeled data availability.</p><p><strong>Results: </strong>The proposed model outperformed conventional supervised or self-supervised learning-based models, achieving state-of-the-art results across 3 public datasets. In a clinical validation, the model exhibited up to 17.50% higher accuracy and 17.53% higher macro F-1 score than a supervised learning-based model under limited training data.</p><p><strong>Discussion: </strong>The model's robustness in OCT image classification underscores the potential of the multi-stage self-supervised learning to address challenges associated with limited labeled data. The availability of source codes and pre-trained models promotes the use of this model in a variety of clinical settings, facilitating broader adoption.</p><p><strong>Conclusion: </strong>This model offers a promising solution for advancing OCT image classification, achieving high accuracy while reducing the cost of extensive expert annotation and potentially streamlining clinical workflows, thereby supporting more efficient patient management.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"800-810"},"PeriodicalIF":4.7,"publicationDate":"2025-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12012341/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143558511","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Optimizing the efficiency and effectiveness of data quality assurance in a multicenter clinical dataset.","authors":"Anne Fu, Trong Shen, Surain B Roberts, Weihan Liu, Shruthi Vaidyanathan, Kayley-Jasmin Marchena-Romero, Yuen Yu Phyllis Lam, Kieran Shah, Denise Y F Mak, Fahad Razak, Amol A Verma","doi":"10.1093/jamia/ocaf042","DOIUrl":"10.1093/jamia/ocaf042","url":null,"abstract":"<p><strong>Objectives: </strong>Electronic health records (EHRs) data are increasingly used for research and analysis, but there is little empirical evidence to inform how automated and manual assessments can be combined to efficiently assess data quality in large EHR repositories.</p><p><strong>Materials and methods: </strong>The GEMINI database collected data from 462 226 patient admissions across 32 hospitals from 2021 to 2023. We report data quality issues identified through semi-automated and manual data quality assessments completed during the data collection phase. We conducted a simulation experiment to evaluate the relationship between the number of records reviewed manually, the detection of true data errors (true positives) and the number of manual chart abstraction errors (false positives) that required unnecessary investigation.</p><p><strong>Results: </strong>The semi-automated data quality assessments identified 79 data quality issues requiring correction, of which 14 had a large impact, affecting at least 50% of records in the data. After resolving issues identified through semi-automated assessments, manual validation of 2676 patient encounters at 19 hospitals identified 4 new meaningful data errors (3 in transfusion data and 1 in physician identifiers), distributed across 4 hospitals. There were 365 manual chart abstraction errors, which required investigation by data analysts to identify as \"false positives.\" These errors increased linearly with the number of charts reviewed manually. Simulation results demonstrate that all 3 transfusion data errors were identified with 95% sensitivity after manual review of 5 records, whereas 18 records were needed for the physician's table.</p><p><strong>Discussion and conclusion: </strong>The GEMINI approach represents a scalable framework for data quality assessment and improvement in multisite EHR research databases. Manual data review is important but can be minimized to optimize the trade-off between true and false identification of data quality errors.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"835-844"},"PeriodicalIF":4.7,"publicationDate":"2025-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12012372/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143626627","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"MedBot vs RealDoc: efficacy of large language modeling in physician-patient communication for rare diseases.","authors":"Magdalena T Weber, Richard Noll, Alexandra Marchl, Carlo Facchinello, Achim Grünewaldt, Christian Hügel, Khader Musleh, Thomas O F Wagner, Holger Storf, Jannik Schaaf","doi":"10.1093/jamia/ocaf034","DOIUrl":"10.1093/jamia/ocaf034","url":null,"abstract":"<p><strong>Objectives: </strong>This study assesses the abilities of 2 large language models (LLMs), GPT-4 and BioMistral 7B, in responding to patient queries, particularly concerning rare diseases, and compares their performance with that of physicians.</p><p><strong>Materials and methods: </strong>A total of 103 patient queries and corresponding physician answers were extracted from EXABO, a question-answering forum dedicated to rare respiratory diseases. The responses provided by physicians and generated by LLMs were ranked on a Likert scale by a panel of 4 experts based on 4 key quality criteria for health communication: correctness, comprehensibility, relevance, and empathy.</p><p><strong>Results: </strong>The performance of generative pretrained transformer 4 (GPT-4) was significantly better than the performance of the physicians and BioMistral 7B. While the overall ranking considers GPT-4's responses to be mostly correct, comprehensive, relevant, and emphatic, the responses provided by BioMistral 7B were only partially correct and empathetic. The responses given by physicians rank in between. The experts concur that an LLM could lighten the load for physicians, rigorous validation is considered essential to guarantee dependability and efficacy.</p><p><strong>Discussion: </strong>Open-source models such as BioMistral 7B offer the advantage of privacy by running locally in health-care settings. GPT-4, on the other hand, demonstrates proficiency in communication and knowledge depth. However, challenges persist, including the management of response variability, the balancing of comprehensibility with medical accuracy, and the assurance of consistent performance across different languages.</p><p><strong>Conclusion: </strong>The performance of GPT-4 underscores the potential of LLMs in facilitating physician-patient communication. However, it is imperative that these systems are handled with care, as erroneous responses have the potential to cause harm without the requisite validation procedures.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"775-783"},"PeriodicalIF":4.7,"publicationDate":"2025-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12012358/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143505806","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Interoperability of health-related social needs data at US hospitals.","authors":"Sahil Sandhu, Michael Liu, Laura M Gottlieb, A Jay Holmgren, Lisa S Rotenstein, Matthew S Pantell","doi":"10.1093/jamia/ocaf049","DOIUrl":"10.1093/jamia/ocaf049","url":null,"abstract":"<p><strong>Objective: </strong>To measure hospital engagement in interoperable exchange of health-related social needs (HRSN) data.</p><p><strong>Materials and methods: </strong>This study combined national data from the 2022 American Hospital Association (AHA) Annual Survey, AHA IT Supplement, and the Centers for Medicare and Medicaid Services Impact File. Multivariable logistic regression was used to identify hospital characteristics associated with receiving HRSN data from external organizations.</p><p><strong>Results: </strong>Of 2502 hospitals, 61.4% reported electronically receiving HRSN data from external sources, most commonly from health information exchange organizations. Hospitals participating in accountable care organizations or patient-centered medical homes and hospitals using Epic or Cerner electronic health records (EHRs) were more likely to receive external HRSN data. In contrast, for-profit hospitals and public hospitals were less likely to participate in HRSN data exchange.</p><p><strong>Discussion: </strong>Hospital ownership, participation in value-based care models, and EHR vendor capabilities are important drivers in advancing HRSN data exchange.</p><p><strong>Conclusion: </strong>Additional policy and technological support may be needed to enhance HRSN data interoperability.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"914-919"},"PeriodicalIF":4.7,"publicationDate":"2025-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12012371/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143674776","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"The value of simulation testing for the evaluation of ambient digital scribes: a case report.","authors":"Joshua M Biro, Jessica L Handley, James Mickler, Sahithi Reddy, Varsha Kottamasu, Raj M Ratwani, Nathan K Cobb","doi":"10.1093/jamia/ocaf052","DOIUrl":"10.1093/jamia/ocaf052","url":null,"abstract":"<p><strong>Objectives: </strong>The objective of this work is to demonstrate the value of simulation testing for rapidly evaluating artificial intelligence (AI) products.</p><p><strong>Materials and methods: </strong>Researcher-physician teams simulated the use of 2 Ambient Digital Scribe (ADS) products by reading scripts of outpatient encounters while using both products, yielding a total of 44 draft notes. Time to edit, perceived amount of effort and editing, and errors in the AI-generated draft notes were analyzed.</p><p><strong>Results: </strong>Ambient Digital Scribe Product A draft notes took significantly longer to edit, had fewer omissions, and more additions and irrelevant or misplaced text errors than ADS Product B. Ambient Digital Scribe Product A was rated as performing better for most encounters.</p><p><strong>Discussion: </strong>Artificial intelligence-enabled products are being rapidly developed and implemented into practice, outpacing safety concerns. Simulation testing can efficiently identify safety issues.</p><p><strong>Conclusion: </strong>Simulation testing is a crucial first step to take when evaluating AI-enabled technologies.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"928-931"},"PeriodicalIF":4.7,"publicationDate":"2025-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12012335/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143674778","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}