Journal of the American Medical Informatics Association最新文献

{"title":"Enhancing systematic literature reviews with generative artificial intelligence: development, applications, and performance evaluation.","authors":"Ying Li, Surabhi Datta, Majid Rastegar-Mojarad, Kyeryoung Lee, Hunki Paek, Julie Glasgow, Chris Liston, Long He, Xiaoyan Wang, Yingxin Xu","doi":"10.1093/jamia/ocaf030","DOIUrl":"https://doi.org/10.1093/jamia/ocaf030","url":null,"abstract":"<p><strong>Objectives: </strong>We developed and validated a large language model (LLM)-assisted system for conducting systematic literature reviews in health technology assessment (HTA) submissions.</p><p><strong>Materials and methods: </strong>We developed a five-module system using abstracts acquired from PubMed: (1) literature search query setup; (2) study protocol setup using population, intervention/comparison, outcome, and study type (PICOs) criteria; (3) LLM-assisted abstract screening; (4) LLM-assisted data extraction; and (5) data summarization. The system incorporates a human-in-the-loop design, allowing real-time PICOs criteria adjustment. This is achieved by collecting information on disagreements between the LLM and human reviewers regarding inclusion/exclusion decisions and their rationales, enabling informed PICOs refinement. We generated four evaluation sets including relapsed and refractory multiple myeloma (RRMM) and advanced melanoma to evaluate the LLM's performance in three key areas: (1) recommending inclusion/exclusion decisions during abstract screening, (2) providing valid rationales for abstract exclusion, and (3) extracting relevant information from included abstracts.</p><p><strong>Results: </strong>The system demonstrated relatively high performance across all evaluation sets. For abstract screening, it achieved an average sensitivity of 90%, F1 score of 82, accuracy of 89%, and Cohen's κ of 0.71, indicating substantial agreement between human reviewers and LLM-based results. In identifying specific exclusion rationales, the system attained accuracies of 97% and 84%, and F1 scores of 98 and 89 for RRMM and advanced melanoma, respectively. For data extraction, the system achieved an F1 score of 93.</p><p><strong>Discussion: </strong>Results showed high sensitivity, Cohen's κ, and PABAK for abstract screening, and high F1 scores for data extraction. This human-in-the-loop AI-assisted SLR system demonstrates the potential of GPT-4's in context learning capabilities by eliminating the need for manually annotated training data. In addition, this LLM-based system offers subject matter experts greater control through prompt adjustment and real-time feedback, enabling iterative refinement of PICOs criteria based on performance metrics.</p><p><strong>Conclusion: </strong>The system demonstrates potential to streamline systematic literature reviews, potentially reducing time, cost, and human errors while enhancing evidence generation for HTA submissions.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":""},"PeriodicalIF":4.7,"publicationDate":"2025-02-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143558513","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A perspective on individualized treatment effects estimation from time-series health data.","authors":"Ghadeer O Ghosheh, Moritz Gögl, Tingting Zhu","doi":"10.1093/jamia/ocae323","DOIUrl":"https://doi.org/10.1093/jamia/ocae323","url":null,"abstract":"<p><strong>Objectives: </strong>The objective of this study is to provide an overview of the current landscape of individualized treatment effects (ITE) estimation, specifically focusing on methodologies proposed for time-series electronic health records (EHRs). We aim to identify gaps in the literature, discuss challenges, and propose future research directions to advance the field of personalized medicine.</p><p><strong>Materials and methods: </strong>We conducted a comprehensive literature review to identify and analyze relevant works on ITE estimation for time-series data. The review focused on theoretical assumptions, types of treatment settings, and computational frameworks employed in the existing literature.</p><p><strong>Results: </strong>The literature reveals a growing body of work on ITE estimation for tabular data, while methodologies specific to time-series EHRs are limited. We summarize and discuss the latest advancements, including the types of models proposed, the theoretical foundations, and the computational approaches used.</p><p><strong>Discussion: </strong>The limitations and challenges of current ITE estimation methods for time-series data are discussed, including the lack of standardized evaluation metrics and the need for more diverse and representative datasets. We also highlight considerations and potential biases that may arise in personalized treatment effect estimation.</p><p><strong>Conclusion: </strong>This work provides a comprehensive overview of ITE estimation for time-series EHR data, offering insights into the current state of the field and identifying future research directions. By addressing the limitations and challenges, we hope to encourage further exploration and innovation in this exciting and under-studied area of personalized medicine.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":""},"PeriodicalIF":4.7,"publicationDate":"2025-02-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143558469","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Deciphering genomic codes using advanced natural language processing techniques: a scoping review.","authors":"Shuyan Cheng, Yishu Wei, Yiliang Zhou, Zihan Xu, Drew N Wright, Jinze Liu, Yifan Peng","doi":"10.1093/jamia/ocaf029","DOIUrl":"10.1093/jamia/ocaf029","url":null,"abstract":"<p><strong>Objectives: </strong>The vast and complex nature of human genomic sequencing data presents challenges for effective analysis. This review aims to investigate the application of natural language processing (NLP) techniques, particularly large language models (LLMs) and transformer architectures, in deciphering genomic codes, focusing on tokenization, transformer models, and regulatory annotation prediction. The goal of this review is to assess data and model accessibility in the most recent literature, gaining a better understanding of the existing capabilities and constraints of these tools in processing genomic sequencing data.</p><p><strong>Materials and methods: </strong>Following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, our scoping review was conducted across PubMed, Medline, Scopus, Web of Science, Embase, and ACM Digital Library. Studies were included if they focused on NLP methodologies applied to genomic sequencing data analysis, without restrictions on publication date or article type.</p><p><strong>Results: </strong>A total of 26 studies published between 2021 and April 2024 were selected for review. The review highlights that tokenization and transformer models enhance the processing and understanding of genomic data, with applications in predicting regulatory annotations like transcription-factor binding sites and chromatin accessibility.</p><p><strong>Discussion: </strong>The application of NLP and LLMs to genomic sequencing data interpretation is a promising field that can help streamline the processing of large-scale genomic data while also providing a better understanding of its complex structures. It has the potential to drive advancements in personalized medicine by offering more efficient and scalable solutions for genomic analysis. Further research is also needed to discuss and overcome current limitations, enhancing model transparency and applicability.</p><p><strong>Conclusion: </strong>This review highlights the growing role of NLP, particularly LLMs, in genomic sequencing data analysis. While these models improve data processing and regulatory annotation prediction, challenges remain in accessibility and interpretability. Further research is needed to refine their application in genomics.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":""},"PeriodicalIF":4.7,"publicationDate":"2025-02-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143505805","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"MedBot vs RealDoc: efficacy of large language modeling in physician-patient communication for rare diseases.","authors":"Magdalena T Weber, Richard Noll, Alexandra Marchl, Carlo Facchinello, Achim Grünewaldt, Christian Hügel, Khader Musleh, Thomas O F Wagner, Holger Storf, Jannik Schaaf","doi":"10.1093/jamia/ocaf034","DOIUrl":"https://doi.org/10.1093/jamia/ocaf034","url":null,"abstract":"<p><strong>Objectives: </strong>This study assesses the abilities of 2 large language models (LLMs), GPT-4 and BioMistral 7B, in responding to patient queries, particularly concerning rare diseases, and compares their performance with that of physicians.</p><p><strong>Materials and methods: </strong>A total of 103 patient queries and corresponding physician answers were extracted from EXABO, a question-answering forum dedicated to rare respiratory diseases. The responses provided by physicians and generated by LLMs were ranked on a Likert scale by a panel of 4 experts based on 4 key quality criteria for health communication: correctness, comprehensibility, relevance, and empathy.</p><p><strong>Results: </strong>The performance of generative pretrained transformer 4 (GPT-4) was significantly better than the performance of the physicians and BioMistral 7B. While the overall ranking considers GPT-4's responses to be mostly correct, comprehensive, relevant, and emphatic, the responses provided by BioMistral 7B were only partially correct and empathetic. The responses given by physicians rank in between. The experts concur that an LLM could lighten the load for physicians, rigorous validation is considered essential to guarantee dependability and efficacy.</p><p><strong>Discussion: </strong>Open-source models such as BioMistral 7B offer the advantage of privacy by running locally in health-care settings. GPT-4, on the other hand, demonstrates proficiency in communication and knowledge depth. However, challenges persist, including the management of response variability, the balancing of comprehensibility with medical accuracy, and the assurance of consistent performance across different languages.</p><p><strong>Conclusion: </strong>The performance of GPT-4 underscores the potential of LLMs in facilitating physician-patient communication. However, it is imperative that these systems are handled with care, as erroneous responses have the potential to cause harm without the requisite validation procedures.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":""},"PeriodicalIF":4.7,"publicationDate":"2025-02-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143505806","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Building an allergy reconciliation module to eliminate allergy discrepancies in electronic health records.","authors":"Suzanne V Blackley, Ying-Chih Lo, Sheril Varghese, Frank Y Chang, Oliver D James, Diane L Seger, Kimberly G Blumenthal, Foster R Goss, Li Zhou","doi":"10.1093/jamia/ocaf022","DOIUrl":"https://doi.org/10.1093/jamia/ocaf022","url":null,"abstract":"<p><strong>Objective: </strong>Accurate, complete allergy histories are critical for decision-making and medication prescription. However, allergy information is often spread across the electronic health record (EHR); thus, allergy lists are often inaccurate or incomplete. Discrepant allergy information can lead to suboptimal or unsafe clinical care and contribute to alert fatigue. We developed an allergy reconciliation module within Mass General Brigham (MGB)'s EHR to support accurate and intuitive reconciliation of discrepancies in the allergy list, thereby enhancing patient safety.</p><p><strong>Materials and methods: </strong>We combined data-driven methods and knowledge from domain experts to develop 5 mechanisms to compare allergy information across the EHR and designed a user interface to display discrepancies and suggested reconciliation actions, with links to relevant data sources. Qualitative and quantitative analyses were conducted to assess the module's performance and measure user acceptance.</p><p><strong>Results: </strong>We implemented and tested the proposed allergy reconciliation mechanisms and module. A comprehensive integration workflow was developed for the module, which was piloted among 111 primary care physicians at MGB. F1 scores of the reconciliation mechanisms range from 0.86 to 1.0. Qualitative analysis showed majority positive feedback from pilot users.</p><p><strong>Discussion: </strong>Our allergy reconciliation module achieved high performance, and physicians who used it largely accepted its recommendations. However, 56% of the pilot group ultimately did not use the module. User engagement and education are likely needed to increase adoption.</p><p><strong>Conclusion: </strong>We built a module to automatically identify discrepancies within patients' allergy records and remind providers to reconcile and update the allergy list. Its high accuracy shows promise for enhancing patient safety and utility of drug allergy alerts.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":""},"PeriodicalIF":4.7,"publicationDate":"2025-02-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143450818","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Integrating State-Space Modeling, Parameter Estimation, Deep Learning, and Docking Techniques in Drug Repurposing: A Case Study on COVID-19 Cytokine Storm.","authors":"Abhisek Bakshi, Kaustav Gangopadhyay, Sujit Basak, Rajat K De, Souvik Sengupta, Abhijit Dasgupta","doi":"10.1093/jamia/ocaf035","DOIUrl":"https://doi.org/10.1093/jamia/ocaf035","url":null,"abstract":"<p><strong>Objective: </strong>This study addresses the significant challenges posed by emerging SARS-CoV-2 variants, particularly in developing diagnostics and therapeutics. Drug repurposing is investigated by identifying critical regulatory proteins impacted by the virus, providing rapid and effective therapeutic solutions for better disease management.</p><p><strong>Materials and methods: </strong>We employed a comprehensive approach combining mathematical modeling and efficient parameter estimation to study the transient responses of regulatory proteins in both normal and virus-infected cells. Proportional-integral-derivative (PID) controllers were used to pinpoint specific protein targets for therapeutic intervention. Additionally, advanced deep learning models and molecular docking techniques were applied to analyze drug-target and drug-drug interactions, ensuring both efficacy and safety of the proposed treatments. This approach was applied to a case study focused on the cytokine storm in COVID-19, centering on Angiotensin-converting enzyme 2 (ACE2), which plays a key role in SARS-CoV-2 infection.</p><p><strong>Results: </strong>Our findings suggest that activating ACE2 presents a promising therapeutic strategy, whereas inhibiting AT1R seems less effective. Deep learning models, combined with molecular docking, identified Lomefloxacin and Fostamatinib as stable drugs with no significant thermodynamic interactions, suggesting their safe concurrent use in managing COVID-19-induced cytokine storms.</p><p><strong>Discussion: </strong>The results highlight the potential of ACE2 activation in mitigating lung injury and severe inflammation caused by SARS-CoV-2. This integrated approach accelerates the identification of safe and effective treatment options for emerging viral variants.</p><p><strong>Conclusion: </strong>This framework provides an efficient method for identifying critical regulatory proteins and advancing drug repurposing, contributing to the rapid development of therapeutic strategies for COVID-19 and future global pandemics.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":""},"PeriodicalIF":4.7,"publicationDate":"2025-02-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143450819","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Correction to: Inpatient nurses' preferences and decisions with risk information visualization.","authors":"","doi":"10.1093/jamia/ocaf028","DOIUrl":"https://doi.org/10.1093/jamia/ocaf028","url":null,"abstract":"","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":""},"PeriodicalIF":4.7,"publicationDate":"2025-02-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143442478","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Reply to Layne et al.'s Letter to the Editor.","authors":"Cathy Shyr, Paul A Harris","doi":"10.1093/jamia/ocaf026","DOIUrl":"https://doi.org/10.1093/jamia/ocaf026","url":null,"abstract":"","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":""},"PeriodicalIF":4.7,"publicationDate":"2025-02-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143416026","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Principles and implementation strategies for equitable and representative academic partnerships in global health informatics research.","authors":"Elizabeth Campbell, Oliver J Bear Don't Walk, Hamish Fraser, Judy Gichoya, Kavishwar B Wagholikar, Andrew S Kanter, Felix Holl, Sansanee Craig","doi":"10.1093/jamia/ocaf015","DOIUrl":"https://doi.org/10.1093/jamia/ocaf015","url":null,"abstract":"<p><strong>Objective: </strong>Developing equitable, sustainable informatics solutions is key to scalability and long-term success for projects in the global health informatics (GHI) domain. This paper presents key strategies for incorporating principles of health equity in the GHI project lifecycle.</p><p><strong>Materials and methods: </strong>The American Medical Informatics Association (AMIA) GHI Working Group organized a collaborative workshop at the 2023 AMIA Annual Symposium that included the presentation of five case studies of how principles of health equity have been incorporated into projects situated in low-and-middle-income countries and with Indigenous communities in the U.S. and best practices for operationalizing these principles into other informatics projects.</p><p><strong>Results: </strong>We present five principles: (1) Inclusion and Participation in Ethical, Sustainable Collaborations; (2) Engaging Community-Based Participatory Research Approaches; (3) Stakeholder Engagement; (4) Scalability and Sustainability; (5) Representation in Knowledge Creation, along with strategies that informatics researchers may use to incorporate these principles into their work.</p><p><strong>Discussion: </strong>Presented case studies and subsequent focus groups yielded key concepts and strategies to promote health equity that may be operationalized across GHI projects.</p><p><strong>Conclusion: </strong>Equitable, sustainable, and scalable GHI projects require intentional integration of community and stakeholder perspectives in project development, implementation, and knowledge creation processes.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":""},"PeriodicalIF":4.7,"publicationDate":"2025-02-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143411280","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Public health informatics specialists in state and local public health workforce: insights from public health workforce interests and needs survey.","authors":"Sripriya Rajamani, Jonathon P Leider, Divya Rupini Gunashekar, Brian E Dixon","doi":"10.1093/jamia/ocaf019","DOIUrl":"https://doi.org/10.1093/jamia/ocaf019","url":null,"abstract":"<p><strong>Objective: </strong>Modernizing and strengthening the US public health data and information infrastructure requires a strong public health informatics (PHI) workforce. The study objectives were to characterize existing PHI specialists and assess informatics-related training needs.</p><p><strong>Materials and methods: </strong>To examine the PHI workforce, the 2021 Public Health Workforce Interests and Needs Survey (PH WINS), a nationally representative survey with 44 732 governmental public health (PH) respondents was analyzed. The survey included data from 47 state health agencies-central office, 29 large local health departments (Big Cities Health Coalition members), and 259 other local/regional health departments. Analysis focused on \"public health informatics specialist\" (PHI), \"information system manager/information technology specialist\" (IT/IS), \"public health science\" (PHS), and \"clinical and laboratory\" (CL) roles.</p><p><strong>Results: </strong>PHI specialists account for less than 2% of the governmental PH workforce. A majority were female (68%), White (55%), and close to half in 31-50 age category (49%). Most (74%) were in non-supervisory roles and <1% in managerial/executive roles, with less than one-third (29%) earning >$75 000 salary. Skill gaps on informatics-related tasks included: identify appropriate data/information sources; collect valid data for decision making; participate in quality improvement processes; identify evidence-based approaches. The PHI specialists reported lower skill gaps in data/informatics areas when compared to other public health roles (PHS and CL), and this was consistent across state/local settings.</p><p><strong>Discussion: </strong>Given the scale of work needed for modernization of information systems, PH agencies need more individuals in informatics roles. To attract PHI specialists, better salaries, clear PHI job classifications and permanent PHI workers are needed, which requires sustained investments from federal and state governments.</p><p><strong>Conclusion: </strong>Efforts to train PHI specialists, recruit and retain them in the governmental public health workforce, and address hiring issues in public health agencies are essential next steps to transform the US public health enterprise.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":" ","pages":""},"PeriodicalIF":4.7,"publicationDate":"2025-02-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143400645","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}