{"title":"From Cloud to Pocket: Frontier Edge Foundation Models and the Future of Clinical Informatics.","authors":"Ravi Shankar","doi":"10.1055/a-2969-1166","DOIUrl":"10.1055/a-2969-1166","url":null,"abstract":"<p><strong>Abstract: </strong>BACKGROUND: Open foundation models designed for on-device and edge deployment, exemplified by the Qwen 3.5 small series (March 2, 2026) and Gemma 4 (April 2, 2026), challenge the default of routing all clinical natural language processing through commercial cloud application programming interfaces (APIs).</p><p><strong>Abstract: </strong>OBJECTIVES: This study aimed to examine the implications of these models for clinical informatics, for clinical informaticians, healthcare chief information officers, governance committees and NLP researchers.</p><p><strong>Abstract: </strong>METHODS: Narrative (non-systematic) perspective. We summarized the technical advances of both releases and situated them within the 2024 to 2026 peer-reviewed evidence on locally deployed clinical large language models. We searched PubMed, Scopus and Google Scholar (January 2024 to June 2026) and added citation tracking.</p><p><strong>Abstract: </strong>RESULTS: Both releases narrow the gap between local and frontier cloud capability for structured extraction and template-based documentation. The existing evidence comes from 3 to 70 billion parameter models on institutional servers. Clinical benchmarks at 0.8 to 4 billion parameters are absent. Hallucination, quantization, endpoint security, liability and unvalidated multimodal and multilingual claims remain open concerns. Frontier cloud models retain clear advantages for complex clinical reasoning.</p><p><strong>Abstract: </strong>CONCLUSION: For a defined subset of tasks, cloud-by-default may no longer be the most defensible choice. The field needs reporting standards, governance and liability frameworks, task-appropriate routing, and edge-aware benchmarks.</p>","PeriodicalId":48956,"journal":{"name":"Applied Clinical Informatics","volume":" ","pages":"882-888"},"PeriodicalIF":2.7,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13649340/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149082541","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Carrie Toni Chan, Karthik Balakrishnan, Aaron Scheffler, Yulin Hswen, Jan Yeager, Mark J Pletcher
{"title":"Human-Centered Design and Usability Evaluation of an EHR-Integrated Clinical Decision Support Prototype for Pediatric Surgical Site Infection Risk.","authors":"Carrie Toni Chan, Karthik Balakrishnan, Aaron Scheffler, Yulin Hswen, Jan Yeager, Mark J Pletcher","doi":"10.1055/a-2969-1041","DOIUrl":"10.1055/a-2969-1041","url":null,"abstract":"<p><strong>Abstract: </strong>OBJECTIVES: This study aimed to design and evaluate the usability of an electronic health record (EHR)-integrated clinical decision support prototype supporting pediatric surgical site infection (SSI) risk estimation and preoperative optimization using a multiphase human-centered design approach.</p><p><strong>Abstract: </strong>METHODS: We conducted a three-phase study at a quaternary pediatric academic health system involving clinicians from anesthesia and seven surgical specialties. Phase 1 used contextual inquiry to examine preoperative information gathering, infection risk assessment, and coordination workflows to define user needs. Phase 2 used structured mock-up design review sessions to refine interface design and establish prototype requirements. Phase 3 involved formative usability testing of an interactive prototype using a think-aloud approach, followed by the Healthcare Systems Usability Scale (HSUS). Qualitative data were analyzed using thematic analysis. HSUS scores were analyzed descriptively.</p><p><strong>Abstract: </strong>RESULTS: Nine clinicians participated in each phase (10 total participants). Contextual inquiry identified challenges including manual chart review burden, task coordination outside the EHR, and largely qualitative risk assessment practices. Design review translated these needs into a centralized preoperative workspace integrating patient summaries, contextualized SSI risk with contributing factors and benchmarks, and role-based task tracking. During usability testing, clinicians reported reduced chart review burden and better visibility of high-priority information, while identifying opportunities to improve visual prioritization, transparency of risk logic and recommendations, and mechanisms to correct or flag inaccurate content. HSUS results indicated good to excellent usability, with an overall usability score of 80%. Workflow integration received the highest ratings (90%), followed by patient safety and decision effectiveness (86%).</p><p><strong>Abstract: </strong>CONCLUSION: This study demonstrates that translating predictive models into clinical practice requires integration within real-world workflows, transparent presentation of information, and design aligned with the cognitive and operational realities of perioperative care. These findings provide a foundation for iterative refinement, implementation, and prospective evaluation of EHR-integrated pediatric SSI decision support.</p>","PeriodicalId":48956,"journal":{"name":"Applied Clinical Informatics","volume":" ","pages":"899-911"},"PeriodicalIF":2.7,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13657793/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149082552","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Oliver T Nguyen, Majid Afshar, Michael A Jaeb, Mary Ryan Baumann, Felice Resnik, Anne Gravel Sullivan, Graham Wills, Jason Dambach, Leigh A Mrotek, Mariah Quinn, Kirsten Abramson, Peter Kleinschmidt, Thomas B Brazelton, Margaret A Leaf, Heidi Twedt, Brian W Patterson, Frank Liao, Stacy Rasmussen, Elizabeth S Burnside, Douglas A Wiegmann, Joel E Gordon
{"title":"Ambient Artificial Intelligence Scribes in Ambulatory Care: Patient Acceptability and Trust.","authors":"Oliver T Nguyen, Majid Afshar, Michael A Jaeb, Mary Ryan Baumann, Felice Resnik, Anne Gravel Sullivan, Graham Wills, Jason Dambach, Leigh A Mrotek, Mariah Quinn, Kirsten Abramson, Peter Kleinschmidt, Thomas B Brazelton, Margaret A Leaf, Heidi Twedt, Brian W Patterson, Frank Liao, Stacy Rasmussen, Elizabeth S Burnside, Douglas A Wiegmann, Joel E Gordon","doi":"10.1055/a-2936-9935","DOIUrl":"10.1055/a-2936-9935","url":null,"abstract":"<p><strong>Abstract: </strong>BACKGROUND: Ambient artificial intelligence (AI) scribes may improve clinician-level outcomes (e.g., documentation burden) and patient experience. However, patients must agree to allow clinicians to use ambient AI scribes for these benefits to materialize. What motivates patients to agree to these tools' use and what factors improve their acceptability remain underexplored. It is also unclear whether patients' current trust in AI tools influences which factors affect ambient AI scribe acceptability.</p><p><strong>Abstract: </strong>OBJECTIVES: This secondary, convergent mixed-methods analysis had a quantitative aim of describing patients' trust in AI tools, a qualitative aim of exploring correlates of ambient AI scribe acceptability, and an integrated aim of exploring whether factors associated with acceptability varied across patients with differing levels of general trust in AI tools.</p><p><strong>Abstract: </strong>METHODS: Survey and interview data originally collected to assess the patient experience after implementing ambient AI scribes were re-analyzed (<i>n</i> = 20). For survey data, we descriptively summarized the trust in AI items and categorized patients into low, moderate, and strong trust groups. For interview data, we used deductive (guided by the Theoretical Framework of Acceptability) and inductive coding to draw themes pertaining to acceptability. For data integration, we qualitatively examined differences in acceptability themes by level of trust in AI.</p><p><strong>Abstract: </strong>RESULTS: In our sample, patients' trust in AI ranged from low to high, and most patients (60%) had moderate trust in AI. Ambient AI scribe acceptability was generally influenced by minimal ethicality concerns, supportive affective attitudes, minimal patient burden, substantial perceived or realized benefits, and strong intervention coherence. There were minimal differences in acceptability themes by level of trust in AI. Patients recommended developing robust patient education initiatives and, to allow patients to make informed decisions, providing advance notice that these tools will be used.</p><p><strong>Abstract: </strong>CONCLUSION: These findings highlight patients' suggested strategies for improving ambient AI scribe acceptability. Our findings also generate hypotheses for future research examining the explanatory processes that underlie patients' perceived acceptability of ambient AI scribes.</p>","PeriodicalId":48956,"journal":{"name":"Applied Clinical Informatics","volume":"17 4","pages":"701-710"},"PeriodicalIF":2.7,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13518063/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148833816","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yael Redelman-Sidi, Charlie White, Kenneth Seier, Allison Lipitz-Snyderman, Jennifer Prey Dawson, Gilad Kuperman, Fernanda C G Polubriaginof, Susan Chimonas
{"title":"Disparities in Cancer Patients' Portal Usage during Hospitalizations.","authors":"Yael Redelman-Sidi, Charlie White, Kenneth Seier, Allison Lipitz-Snyderman, Jennifer Prey Dawson, Gilad Kuperman, Fernanda C G Polubriaginof, Susan Chimonas","doi":"10.1055/a-2961-1034","DOIUrl":"10.1055/a-2961-1034","url":null,"abstract":"<p><strong>Abstract: </strong>BACKGROUND: Patient portals are crucial for successful transition after hospitalization, especially for cancer patients with complex treatment plans.</p><p><strong>Abstract: </strong>OBJECTIVES: This study aimed to identify inequities in hospitalized cancer patients' portal use.</p><p><strong>Abstract: </strong>METHODS: This retrospective analysis assessed portal usage at a high-volume cancer hospital in 2022 and 2023. We assessed whether patients ever logged in, and their login rates, during hospitalizations and intensive care unit (ICU) admissions. Multivariate logistic and Poisson regression models evaluated relationships between demographic and care characteristics and inpatient portal usage.</p><p><strong>Abstract: </strong>RESULTS: Of 28,386 patients, most were female (15,107, 53%), non-Hispanic (24,177, 91%), and White (20,643, 77%); the median age was 65 (interquartile range [IQR]: 54.75) and the median length of stay was 4 days (IQR: 2-8). A total of 65% (18,588) logged in at least once during hospitalization. Patients who were Black (odds ratio [OR] = 0.68, 95% confidence interval [CI]: 0.62, 0.74, <i>p</i> < 0.001; incidence rate ratio [IRR] = 0.68, 95% CI: 0.62, 0.73, <i>p</i> < 0.001), single (OR = 0.80, 95% CI: 0.75, 0.86, <i>p</i> < 0.001; IRR = 0.88, 95% CI: 0.83, 0.93, <i>p</i> < 0.001), and with Medicaid insurance (OR = 0.79, 95% CI: 0.71, 0.87, <i>p</i> < 0.001; IRR = 0.86, 95% CI: 0.8, 0.94, <i>p</i> < 0.001) were, respectively, 32, 20, and 21% less likely to use the portal at least once, and their login rates were 32, 12, and 14% less, compared with White, partnered, and commercially insured patients. Of the 1782 patients with ICU admissions, 971 (55%) logged in at least once during hospitalization. Black patients were 41% (OR = 0.59, 95% CI: 0.43, 0.80, <i>p</i> < 0.001) less likely to use the portal at least once and their login rates were 41% less (IRR = 0.66, 95% CI: 0.51, 0.85, <i>p</i> = 0.001) than White patients.</p><p><strong>Abstract: </strong>CONCLUSION: We uncovered disparities in hospitalized cancer patients' portal usage by race, insurance, and partnership status. Future work should explore the underlying reasons and leverage these insights so that portal onboarding processes and interface designs address the needs of all patients.</p>","PeriodicalId":48956,"journal":{"name":"Applied Clinical Informatics","volume":" ","pages":"844-853"},"PeriodicalIF":2.7,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13630482/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149018408","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yifan Zhang, Sarolta H Trinh, Tom Phelan, Thomas F Byrd, Roshan Tourani, Vipin Kumar, Pedro J Caraballo, Genevieve B Melton-Meaux, Gyorgy J Simon
{"title":"Real-World Effects of a Sepsis Early Detection Model Integrated into Clinical Workflow: A Quasi-Experimental Study.","authors":"Yifan Zhang, Sarolta H Trinh, Tom Phelan, Thomas F Byrd, Roshan Tourani, Vipin Kumar, Pedro J Caraballo, Genevieve B Melton-Meaux, Gyorgy J Simon","doi":"10.1055/a-2962-8091","DOIUrl":"10.1055/a-2962-8091","url":null,"abstract":"<p><strong>Abstract: </strong>BACKGROUND: Sepsis is a life-threatening condition in which delayed recognition and treatment are associated with increased mortality. While predictive models such as Epic's Early Detection of Sepsis Model (ESM) were developed to support early intervention, their real-world effects after integration into clinical workflows remain difficult to evaluate.</p><p><strong>Abstract: </strong>OBJECTIVES: This study aimed to evaluate the real-world effects of ESM integrated into clinical workflow on clinical outcomes, antibiotic use, and harm-benefit trade-offs.</p><p><strong>Abstract: </strong>METHODS: We conducted a quasi-experimental study in a single health care system using encounter-level data from inpatient settings. Inpatient mortality, prolonged hospitalization, antibiotic use, and sepsis prevalence (defined by electronic health record [EHR]) were compared between the preacquisition period (June 3, 2023-June 24, 2024) and the online period (August 21, 2024-December 26, 2024) when the model became visible to clinicians. We also applied a counterfactual framework using models trained on preacquisition data to estimate expected outcomes without ESM and to quantify harms related to overtreatment and delayed treatment.</p><p><strong>Abstract: </strong>RESULTS: Among 89,469 encounters, 75,215 occurred during the preacquisition period and 14,254 during the online period. In unadjusted analyses, inpatient mortality, prolonged hospitalization, antibiotic use, and EHR-defined sepsis proportion all decreased during the online period (all <i>p</i> ≤ 0.006). In the counterfactual analyses, observed outcomes were lower than expected without ESM for mortality (1.21 vs. 1.94%; <i>p</i> < 0.001), prolonged hospitalization (5.56 vs. 8.14%; <i>p</i> < 0.001), and antibiotic use (43.52 vs. 47.74%; <i>p</i> < 0.001). False positive harm (37.72 vs. 42.31%; <i>p</i> < 0.001) was also lower than expected.</p><p><strong>Abstract: </strong>CONCLUSION: Integration of ESM into clinical workflow was associated with improved patient outcomes, reduced antibiotic use, and decreased harm from overtreatment, without evidence of increased harm from delayed treatment, supporting a positive net clinical benefit and the safety and effectiveness of ESM under Software as a Medical Device principles.</p>","PeriodicalId":48956,"journal":{"name":"Applied Clinical Informatics","volume":" ","pages":"854-862"},"PeriodicalIF":2.7,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13645477/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149065546","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Nida Afzal, Amy D Nguyen, Maeve Ryan, Annie Y S Lau
{"title":"User Experiences of Assistive Living Technologies for Fall Prevention, Detection, and Monitoring in Older Adults: A Thematic Synthesis.","authors":"Nida Afzal, Amy D Nguyen, Maeve Ryan, Annie Y S Lau","doi":"10.1055/a-2969-2451","DOIUrl":"10.1055/a-2969-2451","url":null,"abstract":"<p><strong>Abstract: </strong>BACKGROUND: Falls are a leading cause of morbidity, functional decline, and premature care entry among older adults. Assistive living technologies (ALTs) are increasingly used for fall prevention, detection, and risk monitoring, but their value depends on how older adults, caregivers, and care staff experience them in practice.</p><p><strong>Abstract: </strong>OBJECTIVES: The objective of this study is to synthesize user-reported experiences of ALTs used for fall prevention, detection, or monitoring across private homes, retirement villages (RVs), and residential aged care homes (RACHs), examining perceived value, implementation friction, adaptation over time, and contextual differences.</p><p><strong>Abstract: </strong>METHODS: A thematic synthesis was conducted within a scoping review. Searches of Embase, MEDLINE, CINAHL, and Scopus covered 2010 to June 2025. Studies reporting user experience data on real-world ALT deployment in nonhospital settings were included. Line-by-line codes were developed into descriptive subthemes and three analytical themes: negotiated value, implementation friction, and adaptation as relational work. Findings were compared by setting, technology, and evidence source.</p><p><strong>Abstract: </strong>RESULTS: Twenty studies published between 2012 and 2024 were included. Eighteen reported at least one implementation challenge; the remaining two did not provide evidence challenges were absent. Perceived value was negotiated by setting: autonomy and self-management dominated in private homes, whereas RACH evidence was staff or family mediated and tied to situational awareness and workflow. Implementation friction stemmed from unreliable alerts, device burden, privacy concerns, infrastructure, and organizational integration. Twelve studies described some changes during deployment, but reporting was heterogeneous and did not support a universal familiarization timeframe. The single RV study offered no user experience evidence.</p><p><strong>Abstract: </strong>CONCLUSION: ALT value is produced through alignment among technology, user roles, care context, and available resources. Sustained use requires user-centered design, workflow integration, infrastructure and financial planning, and support tailored rather than assumed. Future research should prioritize RVs, direct resident perspectives in RACHs, cost and socioeconomic reporting, and prospective cross-setting comparisons.</p>","PeriodicalId":48956,"journal":{"name":"Applied Clinical Informatics","volume":"17 4","pages":"933-944"},"PeriodicalIF":2.7,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13657792/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149171494","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Do Large Language Models Link to Incorrect Citations? A PubMed and DOI Benchmark.","authors":"Goktug Ozkan","doi":"10.1055/a-2972-4974","DOIUrl":"10.1055/a-2972-4974","url":null,"abstract":"<p><strong>Abstract: </strong>BACKGROUND: Large language models can return citations whose PubMed Identifier (PMID), digital object identifier (DOI), title, and journal appear plausible but identify different papers. Checking one valid identifier may therefore fail to detect an incorrect citation bundle.</p><p><strong>Abstract: </strong>OBJECTIVE: This study aimed to measure citation metadata incoherence in responses from five large language model snapshots to an internal medicine prompt benchmark.</p><p><strong>Abstract: </strong>METHODS: Five models answered 30 synthetic prompts each, producing 150 model-by-prompt responses. Twenty-six prompts per model requested citations, and four were designed to elicit abstention. A corrected parser extracted 300 citation units. We checked reported PMIDs against PubMed metadata, queried reported DOIs separately, and rechecked DOI strings not found in PubMed through Crossref. Response-level outcomes were primary because citation units were nested within responses. The author reviewed a deterministic stratified sample of 50 units for parser capture and citation identity.</p><p><strong>Abstract: </strong>RESULTS: Among 130 citation-expected responses, 69 contained a DOI that resolved to a different PMID than the PMID supplied by the model, 100 contained at least one automated metadata mismatch, and 28 contained no citation. Those 100 were 69 DOI-PMID, 29 other, and 2 unresolvable. All 20 abstention trap responses abstained. Among 116 units whose reported DOI resolved in PubMed, 99 resolved to a different PMID and 17 to the reported PMID. Human review found correct parser capture in 48 of 50 sampled units. Two section boundary errors were corrected before the final rerun; the primary counts did not change. Human review classified 32 sampled units as incoherent, 7 as coherent, 6 as having an unresolvable PMID, and 5 as containing no citation. Clinical claim support was not assessed.</p><p><strong>Abstract: </strong>CONCLUSION: Citation identity failures occurred in all five tested model snapshots. Evaluation and clinical evidence workflows should confirm that the PMID, DOI, title, and journal identify the same source before treating a model-supplied citation as evidence.</p>","PeriodicalId":48956,"journal":{"name":"Applied Clinical Informatics","volume":"17 4","pages":"950-954"},"PeriodicalIF":2.7,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13668380/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149185359","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Comparative Analysis of Large Language Model Performance on USMLE Step 1-Style Allergy/Immunology Questions: Evaluating Correctness and Consistency.","authors":"Moshe Carroll, Sabrina Kentis, Hannah Kareff, Clyde Schechter, Sunit Jariwala","doi":"10.1055/a-2946-7393","DOIUrl":"10.1055/a-2946-7393","url":null,"abstract":"<p><strong>Abstract: </strong>BACKGROUND: Large language models (LLMs) are rapidly transforming medical education, yet their performance in Allergy/Immunology remains insufficiently characterized. Furthermore, concerns regarding accuracy, consistency, and sensitivity to input format persist.</p><p><strong>Abstract: </strong>OBJECTIVES: This study aimed to evaluate and compare the accuracy and response consistency of three leading LLMs-ChatGPT-5, Gemini 2.5, and Grok 4-on Allergy/Immunology United States Medical Licensing Examination (USMLE) Step 1-style questions under different prompt conditions.</p><p><strong>Abstract: </strong>METHODS: Thirty-five USMLE Step 1-style questions were selected. Questions were presented to each model in two formats: single-question prompts and a combined prompt containing all questions. Fifteen trials were conducted for each format per model. Performance was assessed using mean accuracy, and variability was measured using Shannon entropy. Mixed-effects models tested the effects of model, prompt condition, and question difficulty.</p><p><strong>Abstract: </strong>RESULTS: Overall accuracy differed significantly (<i>p</i> < 0.001), with Gemini (80.7%) and Grok (80.5%) achieving higher mean scores than ChatGPT (74.3%). Single-item prompts yielded superior performance, with Grok (93.1%) and Gemini (90.9%) demonstrating the highest accuracy. Transitioning to a combined prompt significantly reduced accuracy for all models. Accuracy also decreased with increasing question difficulty for all models. Grok demonstrated superior reliability, maintaining the lowest overall response entropy, whereas ChatGPT exhibited the highest variability.</p><p><strong>Abstract: </strong>CONCLUSION: On Allergy/Immunology Step 1-style questions, Gemini and Grok demonstrated higher accuracy than ChatGPT, although their overall accuracies remained approximately 81%. Grok offered the most consistent performance. All models demonstrated substantial sensitivity to prompt complexity and inherent performance limitations. These findings underscore the importance of prompt optimization and support the supplementary role of these models in medical education.</p>","PeriodicalId":48956,"journal":{"name":"Applied Clinical Informatics","volume":" ","pages":"754-762"},"PeriodicalIF":2.7,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13549735/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148851405","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Andrew Svetly, Elie Razzouk, Mary McLean, Kenneth Frye, Andrew Little, Lana Massaro, Cheyenne Quailey, Conner Svetly, Aatish Patel, Jesus Montiel
{"title":"Large-Scale Implementation of Ambient Artificial Intelligence Documentation and Its Effects on Electronic Health Record Efficiency and Clinician Well-Being.","authors":"Andrew Svetly, Elie Razzouk, Mary McLean, Kenneth Frye, Andrew Little, Lana Massaro, Cheyenne Quailey, Conner Svetly, Aatish Patel, Jesus Montiel","doi":"10.1055/a-2936-7516","DOIUrl":"10.1055/a-2936-7516","url":null,"abstract":"<p><strong>Abstract: </strong>BACKGROUND: Ambient clinical documentation tools are increasingly used to reduce administrative burden and improve provider experience. However, evidence describing their effects at scale across large, multisite health systems remains limited.</p><p><strong>Abstract: </strong>OBJECTIVES: The objective of this study is to evaluate the effect of Dragon Ambient eXperience (DAX) Copilot on documentation efficiency, note composition practices, and provider well-being across a multistate health system.</p><p><strong>Abstract: </strong>METHODS: Due to rolling registration and training, we conducted a retrospective index-date-aligned pre-post analysis of objective electronic health record activity metrics, supplemented by a postimplementation survey. Physicians and advanced practice providers in ambulatory settings between June 2024 and November 2025 were included. Epic Signal data were analyzed for each provider over the 8 months before and after DAX activation. Metrics included active time in notes per appointment, proportion of note content originating from manual typing, copy/paste, or conventional voice recognition, and average note length. Surveys were administered 45 days postactivation to assess satisfaction, burnout, intent to remain with the organization, and work-life balance.</p><p><strong>Abstract: </strong>RESULTS: A total of 210 providers met the inclusion criteria, defined as generating ≥25% of total note content using ambient voice technology and having at least 16 months of Signal Data (8 months pre- and 8 months postintervention). Compared with the preimplementation period, active note time decreased significantly (5.54-4.09 min per appointment; 26.2% reduction; <i>p</i> < 0.001). Despite a +28.8% increase in note length, manual documentation behaviors declined substantially, including typing (-51.7%), copy/paste (-42.7%), and conventional voice recognition (-67.8%). Survey responses (<i>n</i> = 233; 26.4% response rate) indicated improvements across domains, with most respondents reporting reduced burnout, increased job satisfaction, improved work-life balance, and a strong preference to continue using DAX.</p><p><strong>Abstract: </strong>CONCLUSION: Deployment of ambient artificial intelligence documentation technology at scale was associated with improvements in documentation efficiency, substantial reductions in manual input, and improved provider well-being, supporting its potential to meaningfully alleviate documentation burden.</p>","PeriodicalId":48956,"journal":{"name":"Applied Clinical Informatics","volume":" ","pages":"747-753"},"PeriodicalIF":2.7,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13537836/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148800406","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Autosaved Incident Report Drafts: Log-Based Evidence of Completion, Return-to-Complete Behavior, and Reminder-Linked Submission in a Multisite Patient Safety Reporting System.","authors":"Szu-Chang Wang","doi":"10.1055/a-2971-4388","DOIUrl":"10.1055/a-2971-4388","url":null,"abstract":"<p><strong>Abstract: </strong>BACKGROUND: Patient safety reporting studies usually examine intention, submitted reports, or reporting rates, leaving unclear what happens after a report is started but before it is submitted, deleted, or left unfinished. We evaluated whether autosave and fixed-time reminders make this hidden stage observable.</p><p><strong>Abstract: </strong>OBJECTIVES: This study aimed to quantify draft outcomes, interaction burden, completion timing, return-to-draft behavior, and reminder-linked submission among autosaved incident report drafts.</p><p><strong>Abstract: </strong>METHODS: This was a retrospective log-based study of draft header records and action logs from March 2024 through March 2026. Reminder exposure was defined as a draft remaining incomplete at the daily 05:31 batch; return as a load action within 24 hours after first eligibility. Analyses used descriptive statistics and site-adjusted logistic regression with reporter-clustered standard errors.</p><p><strong>Abstract: </strong>RESULTS: Among 6,835 drafts, 5,397 (79.0%) were submitted, 1,268 (18.6%) were deleted, and 170 (2.5%) were pending. Completed reports were usually rapid (median: 15.0 min; 81.7% within 60 min), but 1,961 drafts (28.7%) had at least one return/load action. Among submitted drafts with action logs, the median interaction burden was 2 actions (interquartile range: 1-3). Overall, 1,617 drafts (23.7%) were reminder-exposed; 337 (20.8%) returned within 24 hours. Return within 24 hours was strongly associated with submission (74.2 vs. 19.8%; adjusted odds ratio: 11.16, 95% confidence interval: 7.29-17.09).</p><p><strong>Abstract: </strong>CONCLUSION: Draft-level logs expose the previously unmeasured middle of incident reporting. Autosave captured interrupted work, and reminder-linked return identified a high-yield recovery point for at-risk drafts. These findings show how interruption-resilient design can support actual report completion, not only reporting intention or submitted-report counts.</p>","PeriodicalId":48956,"journal":{"name":"Applied Clinical Informatics","volume":"17 4","pages":"912-921"},"PeriodicalIF":2.7,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13657795/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149167208","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}