IEEE International Conference on Healthcare Informatics. IEEE International Conference on Healthcare Informatics最新文献

筛选
英文 中文
Curation and Extraction of Drug-Related Entities from Reddit Platform. Reddit平台毒品相关实体的管理与提取。
IEEE International Conference on Healthcare Informatics. IEEE International Conference on Healthcare Informatics Pub Date : 2026-01-01 Epub Date: 2026-08-12 DOI: 10.1109/ichi69079.2026.00027
Zewei Wang, Zihan Xu, Yishu Wei, Michael Chary, Yifan Peng
{"title":"Curation and Extraction of Drug-Related Entities from Reddit Platform.","authors":"Zewei Wang, Zihan Xu, Yishu Wei, Michael Chary, Yifan Peng","doi":"10.1109/ichi69079.2026.00027","DOIUrl":"10.1109/ichi69079.2026.00027","url":null,"abstract":"<p><p>Physicians learn primarily about illicit drugs from clinical overdose cases, limiting their understanding of real-world usage. Meanwhile, drug users share first-hand experiences online, offering insights into dosage and effects of drugs. To bridge this gap, we introduce ReDose (REddit Drug DOSe and Effect), a dataset of 6,435 Reddit posts on substance use. A board-certified toxicologist primarily annotated both the training and test sets, while two medical science students contributed to the test set, labeling DRUG, DOSE, and EFFECT entities. We benchmarked 6,267 annotations using BERT-based, large language model (LLM)-based, and Retrieval-Augmented Generation (RAG) models. BiomedBERT achieved an F1-score of 0.843 for DRUG, while Llama-3 70B outperformed GPT-4 (F1 = 0.79 vs. 0.72). EFFECT extraction remains challenging, with GPT-4 achieving a recall of 0.41. ReDose captures patient-curated narratives to advance medical data extraction from social media.</p>","PeriodicalId":73284,"journal":{"name":"IEEE International Conference on Healthcare Informatics. IEEE International Conference on Healthcare Informatics","volume":"2026 ","pages":"128-136"},"PeriodicalIF":0.0,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13501415/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148815152","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Staging and Pseudotime Inference of Alzheimer's Disease Progression using Multimodal Imaging Data. 利用多模态成像数据对阿尔茨海默病进展的分期和伪时间推断。
IEEE International Conference on Healthcare Informatics. IEEE International Conference on Healthcare Informatics Pub Date : 2026-01-01 Epub Date: 2026-08-12 DOI: 10.1109/ichi69079.2026.00051
Akira A Nair, Zixuan Wen, Zexuan Wang, Jingwen Yan, Andrew J Saykin, Heng Huang, Paul M Thompson, Christos Davatzikos, Li Shen
{"title":"Staging and Pseudotime Inference of Alzheimer's Disease Progression using Multimodal Imaging Data.","authors":"Akira A Nair, Zixuan Wen, Zexuan Wang, Jingwen Yan, Andrew J Saykin, Heng Huang, Paul M Thompson, Christos Davatzikos, Li Shen","doi":"10.1109/ichi69079.2026.00051","DOIUrl":"10.1109/ichi69079.2026.00051","url":null,"abstract":"<p><p>Alzheimer's disease (AD) is a neurodegenerative disorder characterized by progressive cognitive decline, driven by the accumulation of amyloid-beta plaques, tau tangles, and neuronal atrophy. This study analyzes three imaging modalities corresponding to the three hallmark biomarkers of AD: amyloid PET, tau PET, and structural MRI. Using cortical measurements from the ADNI dataset, we apply PHATE (Potential of Heat-diffusion for Affinity-based Trajectory Embedding), a dimensionality reduction technique, to uncover continuous trajectories of disease progression. We derive pseudotime values from PHATE embeddings via Slingshot, a principal-curve-based pseudotime inference algorithm. In parallel, we apply SuStaIn (Subtype and Stage Inference), a machine learning model that uncovers distinct biomarker event sequences and assigns subjects to discrete disease stages. We observe strong correspondence between SuStaIn-predicted stages and PHATE-derived pseudotimes, indicating temporal coherence across modeling frameworks. SuStaIn further reveals non-overlapping, modality-specific sequences of biomarker abnormalities, consistent with prior neuropathological models. We validate these sequences using pseudotime-aligned kernel density models and clinical measures. Together, our results support the integrative use of these approaches for evaluating AD progression. Future steps will focus on anchoring pseudotime to real-world clinical timelines and experimentally validating SuStaIn-predicted biomarker cascades.</p>","PeriodicalId":73284,"journal":{"name":"IEEE International Conference on Healthcare Informatics. IEEE International Conference on Healthcare Informatics","volume":"2026 ","pages":"331-340"},"PeriodicalIF":0.0,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13480406/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148802442","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Toward Complete Hospital Discharge Summarization with Abstract Meaning Representation. 基于抽象意义表示的完整出院摘要研究。
IEEE International Conference on Healthcare Informatics. IEEE International Conference on Healthcare Informatics Pub Date : 2026-01-01 Epub Date: 2026-08-12 DOI: 10.1109/ichi69079.2026.00023
Paul Landes, Sitara Rao, Barbara Di Eugenio, Aaron Chaise
{"title":"Toward Complete Hospital Discharge Summarization with Abstract Meaning Representation.","authors":"Paul Landes, Sitara Rao, Barbara Di Eugenio, Aaron Chaise","doi":"10.1109/ichi69079.2026.00023","DOIUrl":"10.1109/ichi69079.2026.00023","url":null,"abstract":"<p><p>Discharge summaries are lengthy medical documents that summarize a hospital in-patient visit. Automatically generating them can reduce documentation burden and return clinician time to patient care. Whereas Large Language Model (LLMs) could be used for this task, their Achilles heel is hallucinations, which can have drastic consequences for clinical documentation. We present an evidence-driven alignment framework for discharge summarization at the clinical encounter level, that treats provenance as a first-class constraint, using semantic graphs and deep learning models. Each summary sentence is selected and organized via cross-document semantic alignment and is accompanied by explicit evidence links to its source spans. We show our results on two corpora: a publicly available corpus (MIMIC-III) and clinical notes written by physicians at the University of Illinois Hospital (UIC Health). Additionally, we make source code and trained models available.</p>","PeriodicalId":73284,"journal":{"name":"IEEE International Conference on Healthcare Informatics. IEEE International Conference on Healthcare Informatics","volume":"2026 ","pages":"93-102"},"PeriodicalIF":0.0,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13602993/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149042365","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Do LLMs Surpass Encoders for Biomedical NER? llm是否超越了生物医学NER编码器?
IEEE International Conference on Healthcare Informatics. IEEE International Conference on Healthcare Informatics Pub Date : 2025-06-01 Epub Date: 2025-07-22 DOI: 10.1109/ICHI64645.2025.00048
Motasem S Obeidat, Md Sultan Al Nahian, Ramakanth Kavuluru
{"title":"Do LLMs Surpass Encoders for Biomedical NER?","authors":"Motasem S Obeidat, Md Sultan Al Nahian, Ramakanth Kavuluru","doi":"10.1109/ICHI64645.2025.00048","DOIUrl":"10.1109/ICHI64645.2025.00048","url":null,"abstract":"<p><p>Recognizing spans of biomedical concepts and their types (e.g., drug or gene) in free text, often called biomedical named entity recognition (NER), is a basic component of information extraction (IE) pipelines. Without a strong NER component, other applications, such as knowledge discovery and information retrieval, are not practical. State-of-the-art in NER shifted from traditional ML models to deep neural networks with transformer-based encoder models (e.g., BERT) emerging as the current standard. However, decoder models (also called large language models or LLMs) are gaining traction in IE. But LLM-driven NER often ignores positional information due to the generative nature of decoder models. Furthermore, they are computationally very expensive (both in inference time and hardware needs). Hence, it is worth exploring if they actually excel at biomedical NER and assess any associated trade-offs (performance vs efficiency). This is exactly what we do in this effort employing the same BIO entity tagging scheme (that retains positional information) using five different datasets with varying proportions of longer entities. Our results show that the LLMs chosen (Mistral and Llama: 8B range) often outperform best encoder models (BERT-(un)cased, BiomedBERT, and DeBERTav3: 300M range) by 2-8% in F-scores except for one dataset, where they equal encoder performance. This gain is more prominent among longer entities of length ≥ 3 tokens. However, LLMs are one to two orders of magnitude more expensive at inference time and may need cost prohibitive hardware. Thus, when performance differences are small or real time user feedback is needed, encoder models might still be more suitable than LLMs.</p>","PeriodicalId":73284,"journal":{"name":"IEEE International Conference on Healthcare Informatics. IEEE International Conference on Healthcare Informatics","volume":"2025 ","pages":"352-358"},"PeriodicalIF":0.0,"publicationDate":"2025-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12335919/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144818434","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
What to consider when developing a new molecular HIV surveillance tool: Perspectives of key stakeholders working in HIV prevention and treatment. 在开发一种新的HIV分子监测工具时需要考虑什么:从事HIV预防和治疗工作的关键利益相关者的观点。
IEEE International Conference on Healthcare Informatics. IEEE International Conference on Healthcare Informatics Pub Date : 2025-06-01 Epub Date: 2025-07-22 DOI: 10.1109/ichi64645.2025.00072
Shantrel S Canidate, Hannah R Gracy, Sean McIntosh, Yiyang Liu, Rebecca Fisk-Hoffman, Shannon Rich, Carla Mavian, Robert L Cook, Mattia Prosperi, Marco Salemi
{"title":"What to consider when developing a new molecular HIV surveillance tool: Perspectives of key stakeholders working in HIV prevention and treatment.","authors":"Shantrel S Canidate, Hannah R Gracy, Sean McIntosh, Yiyang Liu, Rebecca Fisk-Hoffman, Shannon Rich, Carla Mavian, Robert L Cook, Mattia Prosperi, Marco Salemi","doi":"10.1109/ichi64645.2025.00072","DOIUrl":"10.1109/ichi64645.2025.00072","url":null,"abstract":"<p><p>Developing and validating novel molecular HIV surveillance (MHS) tools capable of predicting the growth and trajectory of localized outbreaks driven by specific transmission clusters is key to the <i>Ending the HIV Epidemic in the United States initiative</i>. This study explored stakeholders' perspectives on HIV prevention and treatment regarding a developing deep-learning framework, <i>DeepDynaForecast,</i> and its ability to predict HIV transmission cluster trajectories and inform decision-making on HIV prevention and treatment scale-up approaches in Florida. We conducted five virtual focus group discussions with 16 clinical health professionals and state and local public health personnel. Focus group discussions were audio-recorded, transcribed using Zoom transcription, and manually coded using a reflexive thematic analysis approach. Overall, participants reported a high level of acceptability for using MHS tools. However, when exploring their perspectives on using the DeepDynaForecast tool,participants discussed their acceptance criteria, including key features that the DeepDynaForecast tool should have and the need to determine the data types the tool should generate to meet their needs and be deemed acceptable. Before implementation, participants felt the tool should undergo extensive software testing, followed by end-users receiving comprehensive training and the developers determining how the DeepDynaForecast tool could integrate with existing MHS tools. Likewise, participants discussed using the data generated by DeepDynaForecast to increase HIV prevention, education, outreach activities, and mobilization efforts in communities where the most HIV diagnoses occur, as well as increase behavioral change communication efforts. Participants also expressed concerns about HIV-related stigma, a potentially dangerous unintended consequence of using existing and new MHS tools. Current MHS tools have helped inform and evaluate HIV prevention and treatment efforts in the US. A novel MHS tool such as DeepDynaForecast may be critical to achieving the Ending the HIV Epidemic (EHE) goals and curbing the spread of HIV in Florida and in the US.</p>","PeriodicalId":73284,"journal":{"name":"IEEE International Conference on Healthcare Informatics. IEEE International Conference on Healthcare Informatics","volume":"2025 ","pages":"588-597"},"PeriodicalIF":0.0,"publicationDate":"2025-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12345360/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144857199","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Lab-AI: Using Retrieval Augmentation to Enhance Language Models for Personalized Lab Test Interpretation in Clinical Medicine. 实验室人工智能:使用检索增强来增强临床医学个性化实验室测试解释的语言模型。
Xiaoyu Wang, Haoyong Ouyang, Balu Bhasuran, Xiao Luo, Karim Hanna, Mia Liza A Lustria, Zhe He
{"title":"Lab-AI: Using Retrieval Augmentation to Enhance Language Models for Personalized Lab Test Interpretation in Clinical Medicine.","authors":"Xiaoyu Wang, Haoyong Ouyang, Balu Bhasuran, Xiao Luo, Karim Hanna, Mia Liza A Lustria, Zhe He","doi":"10.1109/ICHI64645.2025.00025","DOIUrl":"10.1109/ICHI64645.2025.00025","url":null,"abstract":"<p><p>Accurate interpretation of lab results is crucial in clinical medicine, yet most patient portals use universal normal ranges, ignoring conditional factors like age and gender. This study introduces Lab-AI, an interactive system that offers personalized normal ranges using retrieval-augmented generation (RAG) from credible health sources. Lab-AI has two modules: factor retrieval and normal range retrieval. We tested these on 122 lab tests: 40 with conditional factors and 82 without. For tests with factors, normal ranges depend on patient-specific information. Our results show GPT-4-turbo with RAG achieved a 0.948 F1 score for factor retrieval and 0.995 accuracy for normal range retrieval. GPT-4-turbo with RAG outperformed the best non-RAG system by 33.5% in factor retrieval and showed 132% and 100% improvements in question-level and lab-level performance, respectively, for normal range retrieval. These findings highlight Lab-AI's potential to enhance patient understanding of lab results.</p>","PeriodicalId":73284,"journal":{"name":"IEEE International Conference on Healthcare Informatics. IEEE International Conference on Healthcare Informatics","volume":"2025 ","pages":"359-368"},"PeriodicalIF":0.0,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12707499/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145776375","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Attention-based Imputation of Missing Values in Electronic Health Records Tabular Data. 基于注意力的电子健康记录表格数据缺失值估算。
IEEE International Conference on Healthcare Informatics. IEEE International Conference on Healthcare Informatics Pub Date : 2024-06-01 Epub Date: 2024-08-22 DOI: 10.1109/ichi61247.2024.00030
Ibna Kowsar, Shourav B Rabbani, Manar D Samad
{"title":"Attention-based Imputation of Missing Values in Electronic Health Records Tabular Data.","authors":"Ibna Kowsar, Shourav B Rabbani, Manar D Samad","doi":"10.1109/ichi61247.2024.00030","DOIUrl":"10.1109/ichi61247.2024.00030","url":null,"abstract":"<p><p>The imputation of missing values (IMV) in electronic health records tabular data is crucial to enable machine learning for patient-specific predictive modeling. While IMV methods are developed in biostatistics and recently in machine learning, deep learning-based solutions have shown limited success in learning tabular data. This paper proposes a novel attention-based missing value imputation framework that learns to reconstruct data with missing values leveraging between-feature (self-attention) or between-sample attentions. We adopt data manipulation methods used in contrastive learning to improve the generalization of the trained imputation model. The proposed self-attention imputation method outperforms state-of-the-art statistical and machine learning-based (decision-tree) imputation methods, reducing the normalized root mean squared error by 18.4% to 74.7% on five tabular data sets and 52.6% to 82.6% on two electronic health records data sets. The proposed attention-based missing value imputation method shows superior performance across a wide range of missingness (10% to 50%) when the values are missing completely at random.</p>","PeriodicalId":73284,"journal":{"name":"IEEE International Conference on Healthcare Informatics. IEEE International Conference on Healthcare Informatics","volume":"2024 ","pages":"177-182"},"PeriodicalIF":0.0,"publicationDate":"2024-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11463999/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142395730","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Online Transfer Learning for RSV Case Detection. RSV病例检测的在线迁移学习。
IEEE International Conference on Healthcare Informatics. IEEE International Conference on Healthcare Informatics Pub Date : 2024-06-01 Epub Date: 2024-08-22 DOI: 10.1109/ichi61247.2024.00074
Yiming Sun, Yuhe Gao, Runxue Bao, Gregory F Cooper, Jessi Espino, Harry Hochheiser, Marian G Michaels, John M Aronis, Chenxi Song, Ye Ye
{"title":"Online Transfer Learning for RSV Case Detection.","authors":"Yiming Sun, Yuhe Gao, Runxue Bao, Gregory F Cooper, Jessi Espino, Harry Hochheiser, Marian G Michaels, John M Aronis, Chenxi Song, Ye Ye","doi":"10.1109/ichi61247.2024.00074","DOIUrl":"10.1109/ichi61247.2024.00074","url":null,"abstract":"<p><p>Transfer learning has become a pivotal technique in machine learning and has proven to be effective in various real-world applications. However, utilizing this technique for classification tasks with sequential data often faces challenges, primarily attributed to the scarcity of class labels. To address this challenge, we introduce <i>Multi-Source Adaptive Weighting</i> (MSAW), an online multi-source transfer learning method. MSAW integrates a dynamic weighting mechanism into an ensemble framework, enabling automatic adjustment of weights based on the relevance and contribution of each source (representing historical knowledge) and target model (learning from newly acquired data). We demonstrate the effectiveness of MSAW by applying it to detect Respiratory Syncytial Virus cases within Emergency Department visits, utilizing multiple years of electronic health records from the University of Pittsburgh Medical Center. Our method demonstrates performance improvements over many baselines, including refining pre-trained models with online learning as well as three static weighting approaches, showing MSAW's capacity to integrate historical knowledge with progressively accumulated new data. This study indicates the potential of online transfer learning in healthcare, particularly for developing machine learning models that dynamically adapt to evolving situations where new data is incrementally accumulated.</p>","PeriodicalId":73284,"journal":{"name":"IEEE International Conference on Healthcare Informatics. IEEE International Conference on Healthcare Informatics","volume":"2024 ","pages":"512-521"},"PeriodicalIF":0.0,"publicationDate":"2024-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12086431/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144103272","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Patient Engagement and Medication Consistency with Adolescent Heart Transplant Recipients. 青少年心脏移植受者的患者参与和用药一致性。
IEEE International Conference on Healthcare Informatics. IEEE International Conference on Healthcare Informatics Pub Date : 2024-06-01 Epub Date: 2024-08-22 DOI: 10.1109/ichi61247.2024.00068
Michael O Killian, Sonnie E Mayewski, Schyler E Brumm, Lisa Schelbe, Mia Liza A Lustria, Dipankar Gupta
{"title":"Patient Engagement and Medication Consistency with Adolescent Heart Transplant Recipients.","authors":"Michael O Killian, Sonnie E Mayewski, Schyler E Brumm, Lisa Schelbe, Mia Liza A Lustria, Dipankar Gupta","doi":"10.1109/ichi61247.2024.00068","DOIUrl":"10.1109/ichi61247.2024.00068","url":null,"abstract":"<p><p>This study explored the relationship between in-app messaging and patient time between doses of medication during mHealth video directly observed therapy (vDOT) intervention. The vDOT intervention sought to improve medication adherence among adolescent heart transplant recipients. The study utilized a 12-week vDOT pilot with ten adolescent patients, focusing on tacrolimus, a critical immunosuppressant medication. vDOT aimed to enhance patient engagement, provide interpersonal support, and ensure medication-taking consistency (inter-dose timeliness and intervals between doses). Results demonstrated the relationship between in-app communication and more consistent medication-taking behaviors, revealing that messages containing praise, rapport-building, and medically relevant information were positively associated with patients taking medication every 12 hours. The findings suggest that integrating direct observation and personal engagement via mHealth apps could be a promising strategy for managing medication adherence in pediatric transplant care.</p>","PeriodicalId":73284,"journal":{"name":"IEEE International Conference on Healthcare Informatics. IEEE International Conference on Healthcare Informatics","volume":"2024 ","pages":"478-483"},"PeriodicalIF":0.0,"publicationDate":"2024-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13600702/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149042370","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Leveraging Professional Radiologists' Expertise to Enhance LLMs' Evaluation for AI-generated Radiology Reports. 利用专业放射学专家的专业知识,加强法律硕士对人工智能生成的放射学报告的评估。
IEEE International Conference on Healthcare Informatics. IEEE International Conference on Healthcare Informatics Pub Date : 2024-06-01 Epub Date: 2024-08-22 DOI: 10.1109/ichi61247.2024.00058
Qingqing Zhu, Xiuying Chen, Qiao Jin, Benjamin Hou, Tejas Sudharshan Mathai, Pritam Mukherjee, Xin Gao, Ronald M Summers, Zhiyong Lu
{"title":"Leveraging Professional Radiologists' Expertise to Enhance LLMs' Evaluation for AI-generated Radiology Reports.","authors":"Qingqing Zhu, Xiuying Chen, Qiao Jin, Benjamin Hou, Tejas Sudharshan Mathai, Pritam Mukherjee, Xin Gao, Ronald M Summers, Zhiyong Lu","doi":"10.1109/ichi61247.2024.00058","DOIUrl":"10.1109/ichi61247.2024.00058","url":null,"abstract":"<p><p>In radiology, Artificial Intelligence (AI) has significantly advanced report generation, but automatic evaluation of these AI-produced reports remains challenging. Current metrics, such as Conventional Natural Language Generation (NLG) and Clinical Efficacy (CE), often fall short in capturing the semantic intricacies of clinical contexts or overemphasize clinical details, undermining report clarity. To overcome these issues, our proposed method synergizes the expertise of professional radiologists with Large Language Models (LLMs), like GPT-3.5 and GPT-4. Utilizing In-Context Instruction Learning (ICIL) and Chain of Thought (CoT) reasoning, our approach aligns LLM evaluations with radiologist standards, enabling detailed comparisons between human and AI-generated reports. This is further enhanced by a Regression model that aggregates sentence evaluation scores. Experimental results show that our \"Detailed GPT-4 (5-shot)\" model achieves a correlation that is 0.48, outperforming the METEOR metric by 0.19, while our \"Regressed GPT-4\" model shows even greater alignment(0.64) with expert evaluations, exceeding the best existing metric by a 0.35 margin. Moreover, the robustness of our explanations has been validated through a thorough iterative strategy. We plan to publicly release annotations from radiology experts, setting a new standard for accuracy in future assessments. This underscores the potential of our approach in enhancing the quality assessment of AI-driven medical reports.</p>","PeriodicalId":73284,"journal":{"name":"IEEE International Conference on Healthcare Informatics. IEEE International Conference on Healthcare Informatics","volume":"2024 ","pages":"402-411"},"PeriodicalIF":0.0,"publicationDate":"2024-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11651630/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142848618","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
相关产品
×
本文献相关产品
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信
小红书