Journal of the American Medical Informatics Association最新文献

筛选
英文 中文
A machine-learning prediction model to identify risk of firearm injury using electronic health records data. 利用电子健康记录数据识别枪支伤害风险的机器学习预测模型。
IF 4.7 2区 医学
Journal of the American Medical Informatics Association Pub Date : 2024-09-04 DOI: 10.1093/jamia/ocae222
Hui Zhou, Claudia Nau, Fagen Xie, Richard Contreras, Deborah Ling Grant, Sonya Negriff, Margo Sidell, Corinna Koebnick, Rulin Hechter
{"title":"A machine-learning prediction model to identify risk of firearm injury using electronic health records data.","authors":"Hui Zhou, Claudia Nau, Fagen Xie, Richard Contreras, Deborah Ling Grant, Sonya Negriff, Margo Sidell, Corinna Koebnick, Rulin Hechter","doi":"10.1093/jamia/ocae222","DOIUrl":"https://doi.org/10.1093/jamia/ocae222","url":null,"abstract":"<p><strong>Importance: </strong>Firearm injuries constitute a public health crisis. At the healthcare encounter level, they are, however, rare events.</p><p><strong>Objective: </strong>To develop a predictive model to identify healthcare encounters of adult patients at increased risk of firearm injury to target screening and prevention efforts.</p><p><strong>Materials and methods: </strong>Electronic health records data from Kaiser Permanente Southern California (KPSC) were used to identify healthcare encounters of patients with fatal and non-fatal firearm injuries, as well as healthcare visits of a sample of matched controls during 2010-2018. More than 170 predictors, including diagnoses, healthcare utilization, and neighborhood characteristics were identified. Extreme gradient boosting (XGBoost) and a split sample design were used to train and test a model that predicted risk of firearm injury within the next 3 years at the encounter level.</p><p><strong>Results: </strong>A total of 3879 firearm injuries were identified among 5 288 529 KPSC adult members. Prevalence at the healthcare encounter level was 0.01%. The 15 most important predictors included demographics, healthcare utilization, and neighborhood-level socio-economic factors. The sensitivity and specificity of the final model were 0.83 and 0.56, respectively. A very high-risk group (top 1% of predicted risk) yielded a positive predictive value of 0.14% and sensitivity of 13%. This high-risk group potentially reduces screening burden by a factor of 11.7, compared to universal screening. Results for alternative probability cutoffs are presented.</p><p><strong>Discussion: </strong>Our model can support more targeted screening in healthcare settings, resulting in improved efficiency of firearm injury risk assessment and prevention efforts.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":null,"pages":null},"PeriodicalIF":4.7,"publicationDate":"2024-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142134295","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Cigarette smoking, e-cigarette use, and sociodemographic correlates of mental health and tobacco-related disease risk in the All of Us research program. 在 "我们所有人 "研究项目中,吸烟、使用电子烟以及心理健康和烟草相关疾病风险的社会人口学相关因素。
IF 4.7 2区 医学
Journal of the American Medical Informatics Association Pub Date : 2024-09-04 DOI: 10.1093/jamia/ocae237
Thomas R Kirchner, Danning Tian, Jian Li, Pranjal Srivastava, Yihao Zheng
{"title":"Cigarette smoking, e-cigarette use, and sociodemographic correlates of mental health and tobacco-related disease risk in the All of Us research program.","authors":"Thomas R Kirchner, Danning Tian, Jian Li, Pranjal Srivastava, Yihao Zheng","doi":"10.1093/jamia/ocae237","DOIUrl":"https://doi.org/10.1093/jamia/ocae237","url":null,"abstract":"<p><strong>Significance: </strong>Research on the conditions under which electronic cigarette (EC) use produces a net reduction in the population harm attributable to combusted cigarette (CC) use requires the triangulation of information from cohort(s) of smokers, non-smokers, EC users, and dual-users of all varieties.</p><p><strong>Materials and methods: </strong>This project utilizes data from the All of Us Research Program to contrast a panel of wellness and disease-risk indicators across a range of self-reported tobacco-use profiles, including smokers, current, and former EC users. This article focuses on the tobacco use history and current tobacco use status among All of Us participants enrolled between May 2017 and February 2023 (Registered Controlled Tier Curated Data Repository [CDR] v7).</p><p><strong>Results: </strong>The present analytic sample included an unweighted total of N = 412 211 individuals with information on ever-use of both CC and EC. Among them, 155 901 individuals have a history of CC use, with 65 206 identified as current smokers. EC usage is reported by 64 002 individuals, with 16 619 being current users. Model predicted analyses identified distinct patterns in CC and EC usage across demographic and socioeconomic variables, with younger ages favoring ECs.</p><p><strong>Discussion: </strong>Age was observed to significantly affect EC usage, and gender differences reveal that males were significantly more likely to use CC and/or EC than females or African Americans of any gender. Higher educational achievement and income were associated with lower use of both CC and EC, while lower levels of mental health were observed to increase the likelihood of using CC and EC products.</p><p><strong>Conclusion: </strong>Findings suggest the potential for the All of Us Research Program for investigation of causal factors driving both behavioral use transitions and cessation outcomes.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":null,"pages":null},"PeriodicalIF":4.7,"publicationDate":"2024-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142134296","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
CACER: Clinical concept Annotations for Cancer Events and Relations. CACER:癌症事件和关系的临床概念注释。
IF 4.7 2区 医学
Journal of the American Medical Informatics Association Pub Date : 2024-09-03 DOI: 10.1093/jamia/ocae231
Yujuan Velvin Fu, Giridhar Kaushik Ramachandran, Ahmad Halwani, Bridget T McInnes, Fei Xia, Kevin Lybarger, Meliha Yetisgen, Özlem Uzuner
{"title":"CACER: Clinical concept Annotations for Cancer Events and Relations.","authors":"Yujuan Velvin Fu, Giridhar Kaushik Ramachandran, Ahmad Halwani, Bridget T McInnes, Fei Xia, Kevin Lybarger, Meliha Yetisgen, Özlem Uzuner","doi":"10.1093/jamia/ocae231","DOIUrl":"https://doi.org/10.1093/jamia/ocae231","url":null,"abstract":"<p><strong>Objective: </strong>Clinical notes contain unstructured representations of patient histories, including the relationships between medical problems and prescription drugs. To investigate the relationship between cancer drugs and their associated symptom burden, we extract structured, semantic representations of medical problem and drug information from the clinical narratives of oncology notes.</p><p><strong>Materials and methods: </strong>We present Clinical concept Annotations for Cancer Events and Relations (CACER), a novel corpus with fine-grained annotations for over 48 000 medical problems and drug events and 10 000 drug-problem and problem-problem relations. Leveraging CACER, we develop and evaluate transformer-based information extraction models such as Bidirectional Encoder Representations from Transformers (BERT), Fine-tuned Language Net Text-To-Text Transfer Transformer (Flan-T5), Large Language Model Meta AI (Llama3), and Generative Pre-trained Transformers-4 (GPT-4) using fine-tuning and in-context learning (ICL).</p><p><strong>Results: </strong>In event extraction, the fine-tuned BERT and Llama3 models achieved the highest performance at 88.2-88.0 F1, which is comparable to the inter-annotator agreement (IAA) of 88.4 F1. In relation extraction, the fine-tuned BERT, Flan-T5, and Llama3 achieved the highest performance at 61.8-65.3 F1. GPT-4 with ICL achieved the worst performance across both tasks.</p><p><strong>Discussion: </strong>The fine-tuned models significantly outperformed GPT-4 in ICL, highlighting the importance of annotated training data and model optimization. Furthermore, the BERT models performed similarly to Llama3. For our task, large language models offer no performance advantage over the smaller BERT models.</p><p><strong>Conclusions: </strong>We introduce CACER, a novel corpus with fine-grained annotations for medical problems, drugs, and their relationships in clinical narratives of oncology notes. State-of-the-art transformer models achieved performance comparable to IAA for several extraction tasks.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":null,"pages":null},"PeriodicalIF":4.7,"publicationDate":"2024-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142120998","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Foundation model-driven distributed learning for enhanced retinal age prediction. 用于增强视网膜年龄预测的基础模型驱动分布式学习。
IF 4.7 2区 医学
Journal of the American Medical Informatics Association Pub Date : 2024-09-03 DOI: 10.1093/jamia/ocae220
Christopher Nielsen, Raissa Souza, Matthias Wilms, Nils D Forkert
{"title":"Foundation model-driven distributed learning for enhanced retinal age prediction.","authors":"Christopher Nielsen, Raissa Souza, Matthias Wilms, Nils D Forkert","doi":"10.1093/jamia/ocae220","DOIUrl":"https://doi.org/10.1093/jamia/ocae220","url":null,"abstract":"<p><strong>Objectives: </strong>The retinal age gap (RAG) is emerging as a potential biomarker for various diseases of the human body, yet its utility depends on machine learning models capable of accurately predicting biological retinal age from fundus images. However, training generalizable models is hindered by potential shortages of diverse training data. To overcome these obstacles, this work develops a novel and computationally efficient distributed learning framework for retinal age prediction.</p><p><strong>Materials and methods: </strong>The proposed framework employs a memory-efficient 8-bit quantized version of RETFound, a cutting-edge foundation model for retinal image analysis, to extract features from fundus images. These features are then used to train an efficient linear regression head model for predicting retinal age. The framework explores federated learning (FL) as well as traveling model (TM) approaches for distributed training of the linear regression head. To evaluate this framework, we simulate a client network using fundus image data from the UK Biobank. Additionally, data from patients with type 1 diabetes from the UK Biobank and the Brazilian Multilabel Ophthalmological Dataset (BRSET) were utilized to explore the clinical utility of the developed methods.</p><p><strong>Results: </strong>Our findings reveal that the developed distributed learning framework achieves retinal age prediction performance on par with centralized methods, with FL and TM providing similar performance (mean absolute error of 3.57 ± 0.18 years for centralized learning, 3.60 ± 0.16 years for TM, and 3.63 ± 0.19 years for FL). Notably, the TM was found to converge with fewer local updates than FL. Moreover, patients with type 1 diabetes exhibited significantly higher RAG values than healthy controls in all models, for both the UK Biobank and BRSET datasets (P < .001).</p><p><strong>Discussion: </strong>The high computational and memory efficiency of the developed distributed learning framework makes it well suited for resource-constrained environments.</p><p><strong>Conclusion: </strong>The capacity of this framework to integrate data from underrepresented populations for training of retinal age prediction models could significantly enhance the accessibility of the RAG as an important disease biomarker.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":null,"pages":null},"PeriodicalIF":4.7,"publicationDate":"2024-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142120999","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Implementation and delivery of electronic health records training programs for nurses working in inpatient settings: a scoping review. 针对住院病人护士的电子病历培训项目的实施和交付:范围界定综述。
IF 4.7 2区 医学
Journal of the American Medical Informatics Association Pub Date : 2024-09-03 DOI: 10.1093/jamia/ocae228
Oliver T Nguyen, Steven D Vo, Taeheon Lee, Kenrick D Cato, Hwayoung Cho
{"title":"Implementation and delivery of electronic health records training programs for nurses working in inpatient settings: a scoping review.","authors":"Oliver T Nguyen, Steven D Vo, Taeheon Lee, Kenrick D Cato, Hwayoung Cho","doi":"10.1093/jamia/ocae228","DOIUrl":"https://doi.org/10.1093/jamia/ocae228","url":null,"abstract":"<p><strong>Objectives: </strong>Well-designed electronic health records (EHRs) training programs for clinical practice are known to be valuable. Training programs should be role-specific and there is a need to identify key implementation factors of EHR training programs for nurses. This scoping review (1) characterizes the EHR training programs used and (2) identifies their implementation facilitators and barriers.</p><p><strong>Materials and methods: </strong>We searched MEDLINE, CINAHL, PsycINFO, and Web of Science on September 3, 2023, for peer-reviewed articles that described EHR training program implementation or delivery to nurses in inpatient settings without any date restrictions. We mapped implementation factors to the Consolidated Framework for Implementation Research. Additional themes were inductively identified by reviewing these findings.</p><p><strong>Results: </strong>This review included 30 articles. Healthcare systems' approaches to implementing and delivering EHR training programs were highly varied. For implementation factors, we observed themes in innovation (eg, ability to practice EHR skills after training is over, personalizing training, training pace), inner setting (eg, availability of computers, clear documentation requirements and expectations), individual (eg, computer literacy, learning preferences), and implementation process (eg, trainers and support staff hold nursing backgrounds, establishing process for dissemination of EHR updates). No themes in the outer setting were observed.</p><p><strong>Discussion: </strong>We found that multilevel factors can influence the implementation and delivery of EHR training programs for inpatient nurses. Several areas for future research were identified, such as evaluating nurse preceptorship models and developing training programs for ongoing EHR training (eg, in response to new EHR workflows or features).</p><p><strong>Conclusions: </strong>This scoping review highlighted numerous factors pertaining to training interventions, healthcare systems, and implementation approaches. Meanwhile, it is unclear how external factors outside of a healthcare system influence EHR training programs. Additional studies are needed that focus on EHR retraining programs, comparing outcomes of different training models, and how to effectively disseminate updates with the EHR to nurses.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":null,"pages":null},"PeriodicalIF":4.7,"publicationDate":"2024-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142121000","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Reliable generation of privacy-preserving synthetic electronic health record time series via diffusion models. 通过扩散模型可靠生成保护隐私的合成电子健康记录时间序列。
IF 4.7 2区 医学
Journal of the American Medical Informatics Association Pub Date : 2024-09-02 DOI: 10.1093/jamia/ocae229
Muhang Tian, Bernie Chen, Allan Guo, Shiyi Jiang, Anru R Zhang
{"title":"Reliable generation of privacy-preserving synthetic electronic health record time series via diffusion models.","authors":"Muhang Tian, Bernie Chen, Allan Guo, Shiyi Jiang, Anru R Zhang","doi":"10.1093/jamia/ocae229","DOIUrl":"https://doi.org/10.1093/jamia/ocae229","url":null,"abstract":"<p><strong>Objective: </strong>Electronic health records (EHRs) are rich sources of patient-level data, offering valuable resources for medical data analysis. However, privacy concerns often restrict access to EHRs, hindering downstream analysis. Current EHR deidentification methods are flawed and can lead to potential privacy leakage. Additionally, existing publicly available EHR databases are limited, preventing the advancement of medical research using EHR. This study aims to overcome these challenges by generating realistic and privacy-preserving synthetic EHRs time series efficiently.</p><p><strong>Materials and methods: </strong>We introduce a new method for generating diverse and realistic synthetic EHR time series data using denoizing diffusion probabilistic models. We conducted experiments on 6 databases: Medical Information Mart for Intensive Care III and IV, the eICU Collaborative Research Database (eICU), and non-EHR datasets on Stocks and Energy. We compared our proposed method with 8 existing methods.</p><p><strong>Results: </strong>Our results demonstrate that our approach significantly outperforms all existing methods in terms of data fidelity while requiring less training effort. Additionally, data generated by our method yield a lower discriminative accuracy compared to other baseline methods, indicating the proposed method can generate data with less privacy risk.</p><p><strong>Discussion: </strong>The proposed model utilizes a mixed diffusion process to generate realistic synthetic EHR samples that protect patient privacy. This method could be useful in tackling data availability issues in the field of healthcare by reducing barrier to EHR access and supporting research in machine learning for health.</p><p><strong>Conclusion: </strong>The proposed diffusion model-based method can reliably and efficiently generate synthetic EHR time series, which facilitates the downstream medical data analysis. Our numerical results show the superiority of the proposed method over all other existing methods.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":null,"pages":null},"PeriodicalIF":4.7,"publicationDate":"2024-09-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142121001","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Socioeconomic disparities in kidney transplant access for patients with end-stage kidney disease within the All of Us Research Program. 在 "我们所有人 "研究计划中,终末期肾病患者接受肾移植的社会经济差距。
IF 4.7 2区 医学
Journal of the American Medical Informatics Association Pub Date : 2024-09-02 DOI: 10.1093/jamia/ocae178
Jiayuan Wang, Kellie C Cho, Ekamol Tantisattamo
{"title":"Socioeconomic disparities in kidney transplant access for patients with end-stage kidney disease within the All of Us Research Program.","authors":"Jiayuan Wang, Kellie C Cho, Ekamol Tantisattamo","doi":"10.1093/jamia/ocae178","DOIUrl":"https://doi.org/10.1093/jamia/ocae178","url":null,"abstract":"<p><strong>Objectives: </strong>Disparity in kidney transplant access has been demonstrated by a disproportionately low rate of kidney transplantation in socioeconomically disadvantaged patients. However, the information is not from national representative populations with end-stage kidney disease (ESKD). We aim to examine whether socioeconomic disparity for kidney transplant access exists by utilizing data from the All of Us Research Program.</p><p><strong>Materials and methods: </strong>We analyzed data of adult ESKD patients using the All of Us Researcher Workbench. The association of socioeconomic data including types of health insurance, levels of education, and household incomes with kidney transplant access was evaluated by multivariable logistic regression analysis adjusted by baseline demographic, medical comorbidities, and behavioral information.</p><p><strong>Results: </strong>Among 4078 adults with ESKD, mean diagnosis age was 54 and 51.64% were male. The majority had Medicare (39.6%), were non-graduate college (75.79%), and earned $10 000-24 999 annual income (20.16%). After adjusting for potential confounders, insurance status emerged as a significant predictor of kidney transplant access. Individuals covered by Medicaid (adjusted odds ratio [AOR] 0.45; 95% confidence interval [CI], 0.35-0.58; P-value < .001) or uninsured (AOR 0.21; 95% CI, 0.12-0.37; P-value < .001) exhibited lower odds of transplantation compared to those with private insurance.</p><p><strong>Discussion/conclusion: </strong>Our findings reveal the influence of insurance status and socioeconomic factors on access to kidney transplantation among ESKD patients. Addressing these disparities through expanded insurance coverage and improved healthcare access is vital for promoting equitable treatment and enhancing health outcomes in vulnerable populations.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":null,"pages":null},"PeriodicalIF":4.7,"publicationDate":"2024-09-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142121002","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Large language models in biomedicine and health: current research landscape and future directions. 生物医学和健康领域的大型语言模型:当前研究状况和未来发展方向。
IF 4.7 2区 医学
Journal of the American Medical Informatics Association Pub Date : 2024-09-01 DOI: 10.1093/jamia/ocae202
Zhiyong Lu, Yifan Peng, Trevor Cohen, Marzyeh Ghassemi, Chunhua Weng, Shubo Tian
{"title":"Large language models in biomedicine and health: current research landscape and future directions.","authors":"Zhiyong Lu, Yifan Peng, Trevor Cohen, Marzyeh Ghassemi, Chunhua Weng, Shubo Tian","doi":"10.1093/jamia/ocae202","DOIUrl":"10.1093/jamia/ocae202","url":null,"abstract":"","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":null,"pages":null},"PeriodicalIF":4.7,"publicationDate":"2024-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11339542/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142019383","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Relation extraction using large language models: a case study on acupuncture point locations. 使用大型语言模型进行关系提取:穴位位置案例研究。
IF 4.7 2区 医学
Journal of the American Medical Informatics Association Pub Date : 2024-08-29 DOI: 10.1093/jamia/ocae233
Yiming Li, Xueqing Peng, Jianfu Li, Xu Zuo, Suyuan Peng, Donghong Pei, Cui Tao, Hua Xu, Na Hong
{"title":"Relation extraction using large language models: a case study on acupuncture point locations.","authors":"Yiming Li, Xueqing Peng, Jianfu Li, Xu Zuo, Suyuan Peng, Donghong Pei, Cui Tao, Hua Xu, Na Hong","doi":"10.1093/jamia/ocae233","DOIUrl":"https://doi.org/10.1093/jamia/ocae233","url":null,"abstract":"<p><strong>Objective: </strong>In acupuncture therapy, the accurate location of acupoints is essential for its effectiveness. The advanced language understanding capabilities of large language models (LLMs) like Generative Pre-trained Transformers (GPTs) and Llama present a significant opportunity for extracting relations related to acupoint locations from textual knowledge sources. This study aims to explore the performance of LLMs in extracting acupoint-related location relations and assess the impact of fine-tuning on GPT's performance.</p><p><strong>Materials and methods: </strong>We utilized the World Health Organization Standard Acupuncture Point Locations in the Western Pacific Region (WHO Standard) as our corpus, which consists of descriptions of 361 acupoints. Five types of relations (\"direction_of\", \"distance_of\", \"part_of\", \"near_acupoint\", and \"located_near\") (n = 3174) between acupoints were annotated. Four models were compared: pre-trained GPT-3.5, fine-tuned GPT-3.5, pre-trained GPT-4, as well as pretrained Llama 3. Performance metrics included micro-average exact match precision, recall, and F1 scores.</p><p><strong>Results: </strong>Our results demonstrate that fine-tuned GPT-3.5 consistently outperformed other models in F1 scores across all relation types. Overall, it achieved the highest micro-average F1 score of 0.92.</p><p><strong>Discussion: </strong>The superior performance of the fine-tuned GPT-3.5 model, as shown by its F1 scores, underscores the importance of domain-specific fine-tuning in enhancing relation extraction capabilities for acupuncture-related tasks. In light of the findings from this study, it offers valuable insights into leveraging LLMs for developing clinical decision support and creating educational modules in acupuncture.</p><p><strong>Conclusion: </strong>This study underscores the effectiveness of LLMs like GPT and Llama in extracting relations related to acupoint locations, with implications for accurately modeling acupuncture knowledge and promoting standard implementation in acupuncture training and practice. The findings also contribute to advancing informatics applications in traditional and complementary medicine, showcasing the potential of LLMs in natural language processing.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":null,"pages":null},"PeriodicalIF":4.7,"publicationDate":"2024-08-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142114242","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A review of reinforcement learning for natural language processing and applications in healthcare. 回顾强化学习在自然语言处理和医疗保健中的应用。
IF 4.7 2区 医学
Journal of the American Medical Informatics Association Pub Date : 2024-08-29 DOI: 10.1093/jamia/ocae215
Ying Liu, Haozhu Wang, Huixue Zhou, Mingchen Li, Yu Hou, Sicheng Zhou, Fang Wang, Rama Hoetzlein, Rui Zhang
{"title":"A review of reinforcement learning for natural language processing and applications in healthcare.","authors":"Ying Liu, Haozhu Wang, Huixue Zhou, Mingchen Li, Yu Hou, Sicheng Zhou, Fang Wang, Rama Hoetzlein, Rui Zhang","doi":"10.1093/jamia/ocae215","DOIUrl":"https://doi.org/10.1093/jamia/ocae215","url":null,"abstract":"<p><strong>Importance: </strong>Reinforcement learning (RL) represents a pivotal avenue within natural language processing (NLP), offering a potent mechanism for acquiring optimal strategies in task completion. This literature review studies various NLP applications where RL has demonstrated efficacy, with notable applications in healthcare settings.</p><p><strong>Objectives: </strong>To systematically explore the applications of RL in NLP, focusing on its effectiveness in acquiring optimal strategies, particularly in healthcare settings, and provide a comprehensive understanding of RL's potential in NLP tasks.</p><p><strong>Materials and methods: </strong>Adhering to the PRISMA guidelines, an exhaustive literature review was conducted to identify instances where RL has exhibited success in NLP applications, encompassing dialogue systems, machine translation, question-answering, text summarization, and information extraction. Our methodological approach involves closely examining the technical aspects of RL methodologies employed in these applications, analyzing algorithms, states, rewards, actions, datasets, and encoder-decoder architectures.</p><p><strong>Results: </strong>The review of 93 papers yields insights into RL algorithms, prevalent techniques, emergent trends, and the fusion of RL methods in NLP healthcare applications. It clarifies the strategic approaches employed, datasets utilized, and the dynamic terrain of RL-NLP systems, thereby offering a roadmap for research and development in RL and machine learning techniques in healthcare. The review also addresses ethical concerns to ensure equity, transparency, and accountability in the evolution and application of RL-based NLP technologies, particularly within sensitive domains such as healthcare.</p><p><strong>Discussion: </strong>The findings underscore the promising role of RL in advancing NLP applications, particularly in healthcare, where its potential to optimize decision-making and enhance patient outcomes is significant. However, the ethical challenges and technical complexities associated with RL demand careful consideration and ongoing research to ensure responsible and effective implementation.</p><p><strong>Conclusions: </strong>By systematically exploring RL's applications in NLP and providing insights into technical analysis, ethical implications, and potential advancements, this review contributes to a deeper understanding of RL's role for language processing.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":null,"pages":null},"PeriodicalIF":4.7,"publicationDate":"2024-08-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142114241","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
相关产品
×
本文献相关产品
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信