Journal of the American Medical Informatics Association最新文献

{"title":"Letter to the editors in response to \"Leveraging artificial intelligence to summarize abstracts in lay language for increasing research accessibility and transparency\".","authors":"Ethan Layne, Francesco Cei, Giovanni E Cacciamani","doi":"10.1093/jamia/ocaf024","DOIUrl":"10.1093/jamia/ocaf024","url":null,"abstract":"","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"1087-1088"},"PeriodicalIF":4.7,"publicationDate":"2025-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12089783/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143617684","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Recovering missing electronic health record mortality data with a machine learning-enhanced data linkage process.","authors":"John P Powers, Samyuktha Nandhakumar, Sofia Z Dard, Paul Kovach, Peter J Leese","doi":"10.1093/jamia/ocaf060","DOIUrl":"10.1093/jamia/ocaf060","url":null,"abstract":"<p><strong>Objective: </strong>To develop a continual process for linking more comprehensive external mortality data to electronic health records (EHRs) for a large healthcare system, which can serve as a template for other healthcare systems.</p><p><strong>Materials and methods: </strong>Monthly updates of state death records were arranged, and an automated pipeline was developed to identify matches with patients in the EHR. A machine learning classifier was used to closely match human classification performance of potential record matches.</p><p><strong>Results: </strong>The automated linkage process achieved high performance in classifying potential record matches, with a sensitivity of 99.3% and specificity of 98.8% relative to manual classification. Only 22.4% of identified patient deaths were previously indicated in the EHR.</p><p><strong>Discussion and conclusions: </strong>We developed a solution for recovering missing mortality data for EHR that is effective, scalable for cost and computation, and sustainable over time. These recovered mortality data now supplement the EHR data available for research purposes.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"1061-1065"},"PeriodicalIF":4.7,"publicationDate":"2025-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12089760/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143993921","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Assessing health literacy and diversity within the All of Us Research Program.","authors":"Catina O'Leary, Milton Mickey Eder, Sumana Goli, Sam Pettyjohn, Elizabeth Rattine-Flaherty, Yousra Jatt, Linda B Cottler","doi":"10.1093/jamia/ocae225","DOIUrl":"10.1093/jamia/ocae225","url":null,"abstract":"<p><strong>Objective: </strong>The objective was to understand the association between people with adequate and inadequate health literacy (HL) in the All of Us cohort.</p><p><strong>Materials and methods: </strong>Overall, health survey responses to 3 questions from 246 555 people, ages 18-77 years in the controlled tier V7 dataset, were used to assess and compare HL. HL scores ranged from 3 to 15, with scores ≤9 indicating inadequate HL and >9 indicating adequate HL.</p><p><strong>Results: </strong>Cohort participants' responses indicate 92.4% met criteria for adequate HL. Persons with inadequate HL versus adequate HL were likely to be Gen X, male, Black, report an income less than $25k, and have less than a high school education. Furthermore, the rate of HL may not represent that for the broader US population.</p><p><strong>Discussion: </strong>All of Us participants had much higher rates of HL than that for the 2003 National Assessment of Adult Literacy, suggesting approximately over 90% of the US population has HL challenges. The All of Us cohort's high rates of HL may reflect response and recruitment bias. Given the emphasis on diversity and inclusion within the cohort, and understanding HL as the ability to find, understand, and use health information, revisiting the recruitment strategies and, potentially, the assessment of HL within the All of Us cohort is recommended.</p><p><strong>Conclusion: </strong>Factoring HL into diversity and inclusion research recruitment efforts will require review and testing of innovative approaches to community recruitment, engagement, and retention methods. Infusing HL into precision medicine can advance opportunities for individual improvement in health promotion and disease management. Future population level efforts in precision medicine should consider more sensitive measures to critical social determinants of health, such as health literacy, to more carefully characterize diversity and inclusion in these studies.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"1025-1031"},"PeriodicalIF":4.7,"publicationDate":"2025-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12089785/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144020167","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Evaluating the effectiveness of biomedical fine-tuning for large language models on clinical tasks.","authors":"Felix J Dorfner, Amin Dada, Felix Busch, Marcus R Makowski, Tianyu Han, Daniel Truhn, Jens Kleesiek, Madhumita Sushil, Lisa C Adams, Keno K Bressem","doi":"10.1093/jamia/ocaf045","DOIUrl":"10.1093/jamia/ocaf045","url":null,"abstract":"<p><strong>Objectives: </strong>Large language models (LLMs) have shown potential in biomedical applications, leading to efforts to fine-tune them on domain-specific data. However, the effectiveness of this approach remains unclear. This study aims to critically evaluate the performance of biomedically fine-tuned LLMs against their general-purpose counterparts across a range of clinical tasks.</p><p><strong>Materials and methods: </strong>We evaluated the performance of biomedically fine-tuned LLMs against their general-purpose counterparts on clinical case challenges from NEJM and JAMA, and on multiple clinical tasks, such as information extraction, document summarization and clinical coding. We used a diverse set of benchmarks specifically chosen to be outside the likely fine-tuning datasets of biomedical models, ensuring a fair assessment of generalization capabilities.</p><p><strong>Results: </strong>Biomedical LLMs generally underperformed compared to general-purpose models, especially on tasks not focused on probing medical knowledge. While on the case challenges, larger biomedical and general-purpose models showed similar performance (eg, OpenBioLLM-70B: 66.4% vs Llama-3-70B-Instruct: 65% on JAMA), smaller biomedical models showed more pronounced underperformance (OpenBioLLM-8B: 30% vs Llama-3-8B-Instruct: 64.3% on NEJM). Similar trends appeared across CLUE benchmarks, with general-purpose models often achieving higher scores in text generation, question answering, and coding. Notably, biomedical LLMs also showed a higher tendency to hallucinate.</p><p><strong>Discussion: </strong>Our findings challenge the assumption that biomedical fine-tuning inherently improves LLM performance, as general-purpose models consistently performed better on unseen medical tasks. Retrieval-augmented generation may offer a more effective strategy for clinical adaptation.</p><p><strong>Conclusion: </strong>Fine-tuning LLMs on biomedical data may not yield the anticipated benefits. Alternative approaches, such as retrieval augmentation, should be further explored for effective and reliable clinical integration of LLMs.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"1015-1024"},"PeriodicalIF":4.7,"publicationDate":"2025-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12089759/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143796799","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Primary care staff members' experiences with managing electronic health record inbox messages.","authors":"Adam Rule, Phillip Vang, Mark A Micek, Brian G Arndt","doi":"10.1093/jamia/ocaf067","DOIUrl":"10.1093/jamia/ocaf067","url":null,"abstract":"<p><strong>Objective: </strong>Clinical staff often help clinicians review and respond to messages from patients. This study aimed to characterize primary care staff members' experiences with inbox work.</p><p><strong>Materials and methods: </strong>In this qualitative study, we conducted direct observations and focus groups with clinical staff at 4 academic primary care clinics. We used inductive thematic analysis to code the resulting notes and transcripts for themes in staff members' experience with inbox work.</p><p><strong>Results: </strong>Nine medical assistants and 3 nurses participated in the study. Staff described inbox work as fragmented, feeling like an assembly line, requiring frequent communication with other team members to clarify and manage tasks, and requiring navigation of expectations that varied between patients, clinicians, and clinics. Staff described some messages as being more difficult to manage due to how requests were posed, challenges with subsequent communication, and mismatches between data from different sources. Staff also described how tools that structured or automated message management aided inbox work.</p><p><strong>Discussion: </strong>Staff addressed routine messages by following known protocols and appreciated tools that structured their inbox work. However, staff also regularly encountered messages with information that conflicted with clinic records or that contained multiple, redundant, or vague requests. Addressing these messages required additional work to clarify information (ie, data work) and manage resulting tasks (ie, articulation work).</p><p><strong>Conclusion: </strong>Clinic workflows and health information technology should support not only the readily standardized work of addressing routine messages but also the more varied work of preparing messages to be addressed in the first place.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"1040-1049"},"PeriodicalIF":4.7,"publicationDate":"2025-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12089763/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144005422","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Harnessing the power of large language models for clinical tasks and synthesis of scientific literature.","authors":"Suzanne Bakken","doi":"10.1093/jamia/ocaf071","DOIUrl":"10.1093/jamia/ocaf071","url":null,"abstract":"","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":"32 6","pages":"983-984"},"PeriodicalIF":4.7,"publicationDate":"2025-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12089756/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144103139","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A resource for Logical Observation Identifiers Names and Codes terms that may be associated with identifying information.","authors":"Mehdi Nourelahi, Eugene M Sadhu, Malarkodi J Samayamuthu, Shyam Visweswaran","doi":"10.1093/jamia/ocaf061","DOIUrl":"10.1093/jamia/ocaf061","url":null,"abstract":"<p><strong>Objectives: </strong>The primary objective was to compile a comprehensive list of Logical Observation Identifiers Names and Codes (LOINC) terms that may be associated with patient, healthcare provider, and healthcare facility identifying information.</p><p><strong>Materials and methods: </strong>We developed a 2-step procedure for identifying LOINC terms, which consists of a keyword search of Long Common Names and filtering on selected property values, followed by expert physician review to confirm and categorize the terms.</p><p><strong>Results: </strong>The final list comprises 1309 LOINC terms potentially associated with identifying information of patients, providers, and facilities. This list is publicly available on GitHub.</p><p><strong>Discussion: </strong>Compared with electronic health record data coded with other terminologies, LOINC-coded data present unique challenges for deidentification, and a resource of LOINC terms that may be associated with identifying information will be helpful for this purpose.</p><p><strong>Conclusion: </strong>This resource is valuable for deidentifying LOINC-coded data, ensuring compliance with the Health Insurance Portability and Accountability Act (HIPAA), and preserving the privacy of patients, providers, and facilities.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"1066-1070"},"PeriodicalIF":4.7,"publicationDate":"2025-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12089774/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144020066","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Beyond Phecodes: leveraging PheMAP to identify patients lacking diagnosis codes in electronic health records.","authors":"Chao Yan, Monika E Grabowska, Rut Thakkar, Alyson L Dickson, Peter J Embí, QiPing Feng, Joshua C Denny, Vern Eric Kerchberger, Bradley A Malin, Wei-Qi Wei","doi":"10.1093/jamia/ocaf055","DOIUrl":"10.1093/jamia/ocaf055","url":null,"abstract":"<p><strong>Objective: </strong>Diagnosis codes documented in electronic health records (EHR) are often relied upon to clinically phenotype patients for biomedical research. However, these diagnoses can be incomplete and inaccurate, leading to false negatives when searching for patients with phenotypes of interest. This study aims to determine whether PheMAP, a comprehensive knowledgebase integrating multiple clinical terminologies beyond diagnosis to capture phenotypes, can effectively identify patients lacking relevant EHR diagnosis codes.</p><p><strong>Materials and methods: </strong>We investigated a collection of 3.5 million patient records from Vanderbilt University Medical Center's EHR and focused on 4 well-studied phenotypes: (1) type 2 diabetes mellitus (T2DM), (2) dementia, (3) prostate cancer, and (4) sensorineural hearing loss. We applied PheMAP to match structured concepts in patient records and calculated a phenotype risk score (PheScore) to indicate patient-phenotype similarity. Patients meeting predefined PheScore criteria but lacking diagnosis codes were identified. Clinically knowledgeable experts adjudicated randomly selected patients per phenotype as Positive, Possibly Positive, or Negative.</p><p><strong>Results: </strong>Our approach indicated that 5.3% of patients lacked a diagnosis for T2DM, 4.5% for dementia, 2.2% for prostate cancer, and 0.2% for sensorineural hearing loss. The expert review indicated 100% precision (for Possibly Positive or Positive cases) for dementia and sensorineural hearing loss, and 90.0% and 85.0% precision for T2DM and prostate cancer, respectively. Excluding Possibly Positive cases, the precision for T2DM and prostate cancer was 88.9% and 81.3%, respectively.</p><p><strong>Conclusions: </strong>Leveraging clinical terminologies incorporated by PheMAP can effectively identify patients with phenotypes who lack EHR diagnosis codes, thereby enhancing phenotyping quality and related research reliability.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"1007-1014"},"PeriodicalIF":4.7,"publicationDate":"2025-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12089765/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143744134","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Development and validation of the provider documentation summarization quality instrument for large language models.","authors":"Emma Croxford, Yanjun Gao, Nicholas Pellegrino, Karen Wong, Graham Wills, Elliot First, Miranda Schnier, Kyle Burton, Cris Ebby, Jillian Gorski, Matthew Kalscheur, Samy Khalil, Marie Pisani, Tyler Rubeor, Peter Stetson, Frank Liao, Cherodeep Goswami, Brian Patterson, Majid Afshar","doi":"10.1093/jamia/ocaf068","DOIUrl":"10.1093/jamia/ocaf068","url":null,"abstract":"<p><strong>Objectives: </strong>As large language models (LLMs) are integrated into electronic health record (EHR) workflows, validated instruments are essential to evaluate their performance before implementation and as models and documentation practices evolve. Existing instruments for provider documentation quality are often unsuitable for the complexities of LLM-generated text and lack validation on real-world data. The Provider Documentation Summarization Quality Instrument (PDSQI-9) was developed to evaluate LLM-generated clinical summaries. This study aimed to validate the PDSQI-9 across key aspects of construct validity.</p><p><strong>Materials and methods: </strong>Multi-document summaries were generated from real-world EHR data across multiple specialties using several LLMs (GPT-4o, Mixtral 8x7b, and Llama 3-8b). Validation included Pearson correlation analyses for substantive validity, factor analysis and Cronbach's α for structural validity, inter-rater reliability (ICC and Krippendorff's α) for generalizability, a semi-Delphi process for content validity, and comparisons of high- versus low-quality summaries for discriminant validity. Raters underwent standardized training to ensure consistent application of the instrument.</p><p><strong>Results: </strong>Seven physician raters evaluated 779 summaries and answered 8329 questions, achieving over 80% power for inter-rater reliability. The PDSQI-9 demonstrated strong internal consistency (Cronbach's α = 0.879; 95% CI, 0.867-0.891) and high inter-rater reliability (ICC = 0.867; 95% CI, 0.867-0.868), supporting structural validity and generalizability. Factor analysis identified a 4-factor model explaining 58% of the variance, representing organization, clarity, accuracy, and utility. Substantive validity was supported by correlations between note length and scores for Succinct (ρ = -0.200, P = .029) and Organized (ρ = -0.190, P = .037). The semi-Delphi process ensured clinically relevant attributes, and discriminant validity distinguished high- from low-quality summaries (P<.001).</p><p><strong>Discussion: </strong>The PDSQI-9 showed high inter-rater reliability, internal consistency, and a meaningful factor structure that reliably captured key dimensions of documentation quality. It distinguished between high- and low-quality summaries, supporting its practical utility for health systems needing an evaluation instrument for LLMs.</p><p><strong>Conclusions: </strong>The PDSQI-9 demonstrates robust construct validity, supporting its use in clinical practice to evaluate LLM-generated summaries and facilitate safer, more effective integration of LLMs into healthcare workflows.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":"1050-1060"},"PeriodicalIF":4.7,"publicationDate":"2025-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12089781/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144025188","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Dependence of premature ventricular complexes on heart rate-it's not that simple.","authors":"Adrien Osakwe, Noah Wightman, Marc W Deyell, Zachary Laksman, Alvin Shrier, Gil Bub, Leon Glass, Thomas M Bury","doi":"10.1093/jamia/ocaf069","DOIUrl":"https://doi.org/10.1093/jamia/ocaf069","url":null,"abstract":"<p><strong>Objective: </strong>Frequent premature ventricular complexes (PVCs) can lead to adverse health conditions such as cardiomyopathy. The linear correlation between PVC frequency and heart rate (as positive, negative, or neutral) on a 24-hour Holter recording has been proposed as a way to classify patients and guide treatment with beta-blockers. Our objective was to evaluate the robustness of this classification to measurement methodology, different 24-hour periods, and nonlinear dependencies of PVCs on heart rate.</p><p><strong>Materials and methods: </strong>We analyzed 82 multi-day Holter recordings (1-7 days) collected from 48 patients with frequent PVCs (burden 1%-44%). For each record, linear correlation between PVC frequency and heart rate was computed for different 24-hour periods and using different length intervals to determine PVC frequency.</p><p><strong>Results: </strong>Using a 1-hour interval, the correlation between PVC frequency and heart rate was consistently positive, negative, or neutral on different days in only 36.6% of patients. Using shorter time intervals, the correlation was consistent in 56.1% of patients. Shorter time intervals revealed nonlinear and piecewise linear relationships between PVC frequency and heart rate in many patients.</p><p><strong>Discussion: </strong>The variability of the correlation between PVC frequency and heart rate across different 24-hour periods and interval durations suggests that the relationship is neither strictly linear nor stationary. A better understanding of the mechanism driving the PVCs, combined with computational and biological models that represent these mechanisms, may provide insight into the observed nonlinear behavior and guide more robust classification strategies.</p><p><strong>Conclusion: </strong>Linear correlation as a tool to classify patients with frequent PVCs should be used with caution. It is sensitive to the specific 24-hour period analyzed and the methodology used to segment the data. More sophisticated classification approaches that can capture nonlinear and time-varying dependencies should be developed and considered in clinical practice.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":""},"PeriodicalIF":4.7,"publicationDate":"2025-05-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144055982","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}