Journal of the American Medical Informatics Association最新文献

{"title":"Effect of use of ambient listening technology on patient-reported communication and satisfaction ratings.","authors":"Sandra L Taylor, Debbie Aizenberg, Melissa Jost, Yunyi Ren, Courtney R Lyles, Jason Y Adams","doi":"10.1093/jamia/ocag103","DOIUrl":"10.1093/jamia/ocag103","url":null,"abstract":"<p><strong>Objectives: </strong>Evaluate the impact of ambient listening AI scribe technology for drafting notes on patient-reported communication and satisfaction ratings of their physician during ambulatory visits.</p><p><strong>Materials and methods: </strong>We conducted a pilot (April-August 2024) during which 31 physicians across multiple specialties could use an AI scribe during ambulatory visits. Patient surveys from AI-scribe supported visits were compared with pre-pilot surveys in the previous 12 months, and to 125 control physicians, following a difference-in-differences approach. Logistic regression evaluated change in the odds of receiving the highest score for 5 questions, focused on patient ratings of communication (4 items) and overall satisfaction with their physician (1 item).</p><p><strong>Results: </strong>Among 31 917 surveys (7164 pilot; 24 753 control), patient ratings were high. The pre-post change in communication and satisfaction scores did not differ significantly between pilot and control physicians. However, for 3 questions (concern provider showed, explanation provider gave, and likelihood of recommending their provider), ratings increased for pilot physicians while declining for controls without reaching statistical significance.</p><p><strong>Discussion: </strong>We did not find evidence of AI scribes worsening or enhancing patient experiences. Continued evaluation is warranted to ensure positive patient experiences as AI tools expand in clinical care.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"1780-1784"},"PeriodicalIF":7.1,"publicationDate":"2026-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13580746/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148303841","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Loss function influence on hyperparameter optimization for observational healthcare prediction models.","authors":"Fleur Vereijken, Jenna M Reps, Peter Rijnbeek, Ross D Williams","doi":"10.1093/jamia/ocag075","DOIUrl":"10.1093/jamia/ocag075","url":null,"abstract":"<p><strong>Objectives: </strong>Prediction models are increasingly used in healthcare for risk stratification and personalized care. Many models are developed using machine learning, which requires tuning hyperparameters to maximize performance based on a chosen loss function metric. In healthcare, the area under the receiver operating characteristic curve (AUROC) is commonly used for this purpose, but it may not always be the most appropriate choice for every clinical application. We empirically characterize whether the choice of loss function metric in hyperparameter optimization leads to systematic differences in model behavior across several clinical prediction tasks using real-world healthcare data.</p><p><strong>Methods: </strong>We utilized fifteen different loss function metrics to guide hyperparameter selection across three clinical prediction tasks and four machine learning algorithms. We then compared how loss function metric choice affected selected hyperparameters, overall performance, and individual predicted probabilities.</p><p><strong>Results: </strong>We observed that certain hyperparameters tended to have similar optimal values across different loss function metrics, although this pattern differed by algorithm. The best-performing models, evaluated using AUROC, were often not the models with hyperparameters optimized using AUROC. While models performed similarly at a population level, based on discrimination and calibration. The choice of the loss function metric had significant impact on the individual predicted risk for a patient.</p><p><strong>Discussion: </strong>The predictive multiplicity observed can have significant impact on the patient level, while not observed in the population level model evaluation.</p><p><strong>Conclusion: </strong>Predictive multiplicity can have a serious impact on patient treatment decisions but is not yet well understood.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"1474-1484"},"PeriodicalIF":7.1,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13386010/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147943576","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Harnessing institutional knowledge: mixed methods evaluation of peer coaching in a multi-site EHR transition.","authors":"Julian Brunner, Ryan Sterling, Ekaterina Cole, James Weiss, Brenda Faas, Casey Nosbush, Ellen A Ahlness, Edwin S Wong, Seppo T Rinne, Sarah L Cutrona","doi":"10.1093/jamia/ocag071","DOIUrl":"10.1093/jamia/ocag071","url":null,"abstract":"<p><strong>Objective: </strong>Health systems undertaking electronic health record (EHR) transitions often struggle to prepare and support clinicians in learning and using the new system. We evaluated a national peer coaching program-the National EHRM Supplemental Staffing Unit (NESSU)-designed to support clinicians during the U.S. Department of Veterans Affairs' (VA's) transition to a new EHR. Our goal was to assess NESSU's reach, perceived usefulness, and association with key EHR user outcomes, and to characterize how NESSU achieved its observed impacts.</p><p><strong>Materials and methods: </strong>Using a convergent mixed-methods design, we surveyed EHR users at the most recent VA facility to implement the new EHR. Descriptive statistics summarized program reach and perceived helpfulness. Regression models assessed associations between NESSU participation and 3 outcomes: burnout, EHR-related stress, and EHR confidence. Qualitative data included 62 interviews with users and open-ended survey responses. We used structured coding and thematic analysis to identify themes.</p><p><strong>Results: </strong>Among 385 respondents, 58.4% reported receiving NESSU support and 83.6% of those rated it as helpful. NESSU participation was associated with lower rates of burnout (29% vs 41%, P = 0.016) but not with differences in EHR confidence or EHR-related stress. Qualitative analysis yielded 4 themes describing how NESSU functioned (filling education gaps, providing responsive support, offering expert guidance, and drawing upon notable interpersonal skills) and one theme describing its overall impact.</p><p><strong>Discussion: </strong>Findings demonstrate that peer coaching can address important support needs during EHR transitions.</p><p><strong>Conclusion: </strong>Scalable, clinician-led peer coaching may represent an essential component of large-scale EHR transitions, supporting both implementation and clinician well-being.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"1485-1493"},"PeriodicalIF":7.1,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13386017/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147965310","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Explainability in context: calibrating appropriate trust and reliance in artificial intelligence.","authors":"Sharon E Davis, Megan E Salwei","doi":"10.1093/jamia/ocag082","DOIUrl":"10.1093/jamia/ocag082","url":null,"abstract":"<p><strong>Background and significance: </strong>Predictive artificial intelligence (AI) promises to transform care delivery, enhance patient safety, and improve health outcomes. Realizing these benefits will require careful design, implementation, and monitoring strategies to avoid unintended consequences, including automation bias (i.e., erroneously favoring recommendations from automated systems). Automation bias is particularly concerning due to the variability of AI performance across time and populations, leading to predictions that may be variably incorrect, uncertain, or unfair.</p><p><strong>Approach: </strong>We advocate for an expanded view of explainable AI that uses contextual information to help end users calibrate appropriate levels of trust and reliance. We propose multiple levels of contextualization-model, setting, subpopulation, and patient-that together provide insight for clinicians to evaluate the reliability of individual predictions. This includes information about historical and in-the-moment AI performance, algorithmic fairness, and prediction uncertainty.</p><p><strong>Conclusion: </strong>We outline an approach to integrate context-based explanations into decision support workflows to aid clinician interpretation without adding cognitive burden.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"1554-1563"},"PeriodicalIF":7.1,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13385988/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148037246","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Alert fatigue measurement in clinical decision support: a systematic review.","authors":"Cara E Ray, Geneva M Wilson, Ashley M Hughes, Cassie Cunningham Goedken, Eric Ping-Fei Liu, Margaret A Fitzpatrick, Katie J Suda, Satya Manasa Kota, Chinonyerem Nwankpa, Charlesnika T Evans","doi":"10.1093/jamia/ocag064","DOIUrl":"10.1093/jamia/ocag064","url":null,"abstract":"<p><strong>Background: </strong>Alert fatigue is defined as alert dismissals due to excessive or irrelevant alerts and is frequently cited as a barrier to clinical decision support system use and impact. However, the criteria for determining the presence or absence of alert fatigue are poorly defined. The objective of this systematic review of systematic reviews was to identify operationalized definitions and measures of alert fatigue or alert-related metrics.</p><p><strong>Methods: </strong>Systematic reviews reporting at least one alert-related metric or measure/operationalization of alert fatigue for physician-directed electronic alerts were included. The Cochrane Library, Embase, and PubMed were searched from database start to 2024. The Revised Assessment of Multiple Systematic Reviews was used to assess study quality and risk of bias. Data were synthesized narratively and with descriptive statistics.</p><p><strong>Results: </strong>A total of 22 studies were included in the review. Studies reported between 1 and 11 alert metrics. Studies were most often of medium quality. Reporting of primary study characteristics was frequently judged to be insufficient. Only one article reported an operational definition of alert fatigue. The most common alert metrics were quantity, override rate, and acceptance rate.</p><p><strong>Discussion: </strong>Alert fatigue measurement methods are not clearly or consistently defined in systematic reviews related to alert fatigue in clinical decision support. Reporting of other primary study characteristics is often limited. We recommend that future efforts use a significant, sustained decrease in appropriate alert response rates from an established baseline as a measure of alert fatigue.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"1523-1531"},"PeriodicalIF":7.1,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13385993/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147965325","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Theory and practice in biomedical informatics: a framework for discovery.","authors":"William W Stead, Constantin F Aliferis, Lisa Bastarache, Nancy M Lorenzi, W Ed Hammond","doi":"10.1093/jamia/ocag079","DOIUrl":"10.1093/jamia/ocag079","url":null,"abstract":"<p><strong>Objective: </strong>Clarify disciplinary foundations and internal structure of biomedical informatics.</p><p><strong>Methods: </strong>We analyze BMI's emergence at disciplinary intersections and map its internal structure across 4 domains: theory and practice of knowledge discovery, knowledge representation and reasoning, knowledge architecture, and knowledge-driven transformation. We compare BMI with mathematics, computer science, biostatistics, and biomedical engineering, and illustrate emergent characteristics through a precision medicine example.</p><p><strong>Results: </strong>BMI's distinctive contribution-elucidating the structure of biomedical knowledge and developing methods to discover, preserve, and make knowledge actionable-requires strength across all 4 domains. BMI developed these domains pragmatically: building systems, extracting principles, and formalizing theories. The discipline must now complement empirical approaches with rigorous theoretical work: assessing adequacy of existing theories, identifying gaps, and orchestrating collaborative development.</p><p><strong>Conclusions: </strong>BMI creates emergent capabilities across disciplines. As biomedicine becomes increasingly complex, BMI must strengthen its theoretical foundations while demonstrating transformative potential of knowledge spanning biological scales and time.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"1532-1537"},"PeriodicalIF":7.1,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13385994/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148165428","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Explainable machine learning in healthcare: methods, interpretation, and applications for clinical research.","authors":"Krishna Padmanabhan, Minxin Lu, Dai Feng, Natalia Kan-Dobrosky, Sai Konduri, Heather J Litman, Achilleas Livieratos","doi":"10.1093/jamia/ocag077","DOIUrl":"10.1093/jamia/ocag077","url":null,"abstract":"<p><strong>Objectives: </strong>To provide a practical and methodologically grounded overview of explainable machine learning (XML) approaches in healthcare, with emphasis on their interpretation and application in clinical research and decision support. By moving beyond traditional predictive models, this primer aims to foster trust, transparency, and informed clinical decision-making, ultimately bridging the gap between data science and medical practice.</p><p><strong>Materials and methods: </strong>We present a structured review of commonly used XML methodologies, including global and local interpretability tools such as SHapley Additive exPlanations (SHAP), Local Interpretable Model-Agnostic Explanations (LIME), Partial Dependence Plots (PDP), and Individual Conditional Expectation (ICE) plots. For each method, we explain the underlying mechanism at a high level, visualize representative outputs, and provide structured guidance on interpretation, appropriate use, and limitations, illustrated using the publicly available Heart Disease dataset.</p><p><strong>Results: </strong>XML techniques provided intuitive visual and quantitative insights into how predictors influence model predictions. Global methods characterized population-level feature effects, whereas local methods revealed patient-level contributions useful for individualized interpretation. Our worked examples demonstrate how XML outputs can identify nonlinear relationships, detect interaction effects, and reveal heterogeneity in predicted risk across patients, addressing key challenges in translating ML predictions into interpretable outputs for clinical research.</p><p><strong>Discussion and conclusion: </strong>XML tools offer valuable interpretability for ML models and support more transparent and accountable ML applications in clinical research. By providing a methodologically grounded overview alongside practical implementation examples and structured guidance on each method's strengths and limitations, this primer helps bridge the gap between advanced ML methodology and clinical applicability. Thoughtful adoption of XML approaches may facilitate better understanding, communication, and critical evaluation of ML predictions in healthcare research, ultimately supporting evidence-based clinical decision-making.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"1510-1522"},"PeriodicalIF":7.1,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13386015/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148145324","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"What do clinicians edit in ambient AI-drafted clinical documentation? A qualitative content analysis.","authors":"Yawen Guo, Di Hu, Ziqi Yang, Seungjun Kim, Brian Tran, Jamie Lee, Sitha Vallabhaneni, Rachael Zehrung, Sairam Sutari, Steven Tam, Emilie Chow, Danielle Perret, Deepti Pandita, Kai Zheng","doi":"10.1093/jamia/ocag073","DOIUrl":"10.1093/jamia/ocag073","url":null,"abstract":"<p><strong>Objective: </strong>Ambient artificial intelligence (AI) documentation is increasingly used to draft clinical notes from patient-provider conversations, but how clinicians revise and finalize these drafts is not well understood. This qualitative content analysis study characterizes real-world edits to AI-generated drafts and identifies opportunities for improvement of AI design and the implementation process.</p><p><strong>Materials and methods: </strong>Eight coders analyzed clinical documentation generated by ambient AI from 200 clinical encounters. We developed an inductive coding framework with 11 codes across 3 categories: clinical content, terminology, and language style. Interrater reliability was assessed using Cohen's kappa. We then applied thematic analysis to synthesize patterns across the coded edits.</p><p><strong>Results: </strong>The most frequently edited content pertained to clinical facts including orders (eg, procedures, lab tests) (40.0%), symptoms (30.3%), medication prescriptions (27.3%), and diagnosis descriptions (25.9%). In comparison, edits related to terminology use (11.6%) and language style (7.2%) were less frequent. The results of our thematic analysis show that most edits can be categorized into one of the following 5 types: to revise factual discrepancies, to add medical specialty-specific details, to express diagnostic certainties, to convert patient expressions into objective assessments recorded in medical terms, and to reorganize or condense content.</p><p><strong>Conclusion and discussion: </strong>Clinicians routinely revise ambient AI drafts to modify factual details and clinical specificity. Future work on AI development and clinical implementation should emphasize specialty customization and support personalized documentation practices, alongside clinician education that promotes robust and consistent review routines to ensure documentation quality.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"1457-1465"},"PeriodicalIF":7.1,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13386006/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147935110","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Generative artificial intelligence for inpatient documentation summarization: mixed-methods quality assessment and early real-world experience.","authors":"Steve G Peters, Jens P Boyum, Sean R Legler, Kate J Heise, Angie Griffin, Heather A Heaton","doi":"10.1093/jamia/ocag076","DOIUrl":"10.1093/jamia/ocag076","url":null,"abstract":"<p><strong>Objective: </strong>We evaluated the quality and adoption of a large language model (LLM)-based summarization tool for ongoing hospital care.</p><p><strong>Materials and methods: </strong>An AI summarization tool was created to provide a \"Patient Story\", specialty-specific \"Recent Notes\", and \"Recent Events\" over the previous 24 or 72 hours. We conducted a pragmatic mixed-methods quality assessment at three tertiary-care academic hospitals utilizing (1) Provider Documentation Summarization Quality Instrument (PDSQI-9), (2) utilization analytics, and (3) qualitative end user feedback. The PDSQI-9 included whether summaries were accurate, cited, comprehensible, organized, succinct, non-stigmatizing, synthesized, thorough, and useful. 512 users were given access, from whom 52 participants submitted 205 surveys (10.2% response rate).</p><p><strong>Results: </strong>52 respondents submitted an average of 4.3 surveys (range 1-8). Users rated the tool favorably across all PDSQI-9 domains, with a combined average score of 4.68 (range 4.56-4.78, S.D. 0.67) across the eight domains scored on a 5-point modified Likert scale. Utilization metrics demonstrated strong uptake with frequent views. Positive qualitative feedback revealed cognitive offloading for complex patients and effective summarization of medical problems. Critical feedback showed a need to cross-reference narrative notes to current-state data and lack of detailed specialty-specific summarization.</p><p><strong>Discussion: </strong>Generative artificial intelligence has emerged as a potentially transformative technology for generating succinct, verifiable summaries of ongoing care. In this pragmatic implementation, end-users indicated high perceived quality across PDSQI-9 domains.</p><p><strong>Conclusions: </strong>An LLM-based summarization tool for ongoing hospitalization care was rated of high quality by diverse clinicians in real-world settings, demonstrated a favorable safety profile, and showed sustained utilization.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"1466-1473"},"PeriodicalIF":7.1,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13385995/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147943608","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Fine-tuning and evaluating large language models for patient safety tasks: classification of contributing factors in incident reports.","authors":"Ying Wang, Lorelle Bowditch, Charlotte Molloy, Yinghua Yu, Peter Hibbert, Farah Magrabi","doi":"10.1093/jamia/ocag068","DOIUrl":"10.1093/jamia/ocag068","url":null,"abstract":"<p><strong>Objective: </strong>To evaluate and compare the performance of large language models (LLMs) in identifying contributing factors (CFs) underlying patient safety incident investigations.</p><p><strong>Materials and methods: </strong>Four open-source, lightweight LLMs, including BERT, LLaMA2, GPT2, and Phi-2 were applied to classify CFs across 6 sociotechnical system-levels encompassing 12 categories (eg, person, task, and organizational factors). Reports of real-world patient safety investigations from public health systems were extracted and labelled by domain experts (n_report/CFs = 300/1338). Data were split into training (n = 852), validation (n = 98), and test sets (n = 388). Performance was evaluated using specificity, precision, recall, and F1 scores.</p><p><strong>Results: </strong>The fine-tuned encoder-based BERT model achieved the highest performance, with a micro-averaged F1 score of 63.6%, outperforming all decoder-based models. Among the decoder models, Phi-2 demonstrated the strongest performance (F1 = 54.9%), exceeding both LLaMA2 and GPT2. BERT performed consistently across 6 system-levels but often misclassified \"organization\" as \"person\".</p><p><strong>Discussion: </strong>LLMs hold promise for automating the extraction of CFs from complex safety narratives, particularly for frequently reported system-levels such as \"person\" and \"tasks\". Such automation may substantially reduce the manual effort required to analyse reports of patient safety investigations while supporting more consistent analysis across large incident datasets.</p><p><strong>Conclusion: </strong>Applying LLMs to analyse the underlying causes of patient safety incidents depends on developing high-quality, domain-specific datasets that enhance the representation of patient safety knowledge and improve model understanding of incident causation. Improving data coverage for rare system-levels is essential to address the current limitations of LLMs in capturing nuanced patient safety concepts and domain-specific reasoning.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"1494-1503"},"PeriodicalIF":7.1,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13385986/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147976669","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}