Journal of the American Medical Informatics Association最新文献

筛选
英文 中文
Utilizing ChatGPT as a scientific reasoning engine to differentiate conflicting evidence and summarize challenges in controversial clinical questions. 利用 ChatGPT 作为科学推理引擎,区分相互矛盾的证据,总结有争议的临床问题所面临的挑战。
IF 4.7 2区 医学
Journal of the American Medical Informatics Association Pub Date : 2024-06-20 DOI: 10.1093/jamia/ocae100
Shiyao Xie, Wenjing Zhao, Guanghui Deng, Guohua He, Na He, Zhenhua Lu, Weihua Hu, Mingming Zhao, Jian Du
{"title":"Utilizing ChatGPT as a scientific reasoning engine to differentiate conflicting evidence and summarize challenges in controversial clinical questions.","authors":"Shiyao Xie, Wenjing Zhao, Guanghui Deng, Guohua He, Na He, Zhenhua Lu, Weihua Hu, Mingming Zhao, Jian Du","doi":"10.1093/jamia/ocae100","DOIUrl":"10.1093/jamia/ocae100","url":null,"abstract":"<p><strong>Objective: </strong>Synthesizing and evaluating inconsistent medical evidence is essential in evidence-based medicine. This study aimed to employ ChatGPT as a sophisticated scientific reasoning engine to identify conflicting clinical evidence and summarize unresolved questions to inform further research.</p><p><strong>Materials and methods: </strong>We evaluated ChatGPT's effectiveness in identifying conflicting evidence and investigated its principles of logical reasoning. An automated framework was developed to generate a PubMed dataset focused on controversial clinical topics. ChatGPT analyzed this dataset to identify consensus and controversy, and to formulate unsolved research questions. Expert evaluations were conducted 1) on the consensus and controversy for factual consistency, comprehensiveness, and potential harm and, 2) on the research questions for relevance, innovation, clarity, and specificity.</p><p><strong>Results: </strong>The gpt-4-1106-preview model achieved a 90% recall rate in detecting inconsistent claim pairs within a ternary assertions setup. Notably, without explicit reasoning prompts, ChatGPT provided sound reasoning for the assertions between claims and hypotheses, based on an analysis grounded in relevance, specificity, and certainty. ChatGPT's conclusions of consensus and controversies in clinical literature were comprehensive and factually consistent. The research questions proposed by ChatGPT received high expert ratings.</p><p><strong>Discussion: </strong>Our experiment implies that, in evaluating the relationship between evidence and claims, ChatGPT considered more detailed information beyond a straightforward assessment of sentimental orientation. This ability to process intricate information and conduct scientific reasoning regarding sentiment is noteworthy, particularly as this pattern emerged without explicit guidance or directives in prompts, highlighting ChatGPT's inherent logical reasoning capabilities.</p><p><strong>Conclusion: </strong>This study demonstrated ChatGPT's capacity to evaluate and interpret scientific claims. Such proficiency can be generalized to broader clinical research literature. ChatGPT effectively aids in facilitating clinical studies by proposing unresolved challenges based on analysis of existing studies. However, caution is advised as ChatGPT's outputs are inferences drawn from the input literature and could be harmful to clinical practice.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":null,"pages":null},"PeriodicalIF":4.7,"publicationDate":"2024-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11187493/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140960509","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Streamlining social media information retrieval for public health research with deep learning. 利用深度学习为公共卫生研究简化社交媒体信息提取。
IF 4.7 2区 医学
Journal of the American Medical Informatics Association Pub Date : 2024-06-20 DOI: 10.1093/jamia/ocae118
Yining Hua, Jiageng Wu, Shixu Lin, Minghui Li, Yujie Zhang, Dinah Foer, Siwen Wang, Peilin Zhou, Jie Yang, Li Zhou
{"title":"Streamlining social media information retrieval for public health research with deep learning.","authors":"Yining Hua, Jiageng Wu, Shixu Lin, Minghui Li, Yujie Zhang, Dinah Foer, Siwen Wang, Peilin Zhou, Jie Yang, Li Zhou","doi":"10.1093/jamia/ocae118","DOIUrl":"10.1093/jamia/ocae118","url":null,"abstract":"<p><strong>Objective: </strong>Social media-based public health research is crucial for epidemic surveillance, but most studies identify relevant corpora with keyword-matching. This study develops a system to streamline the process of curating colloquial medical dictionaries. We demonstrate the pipeline by curating a Unified Medical Language System (UMLS)-colloquial symptom dictionary from COVID-19-related tweets as proof of concept.</p><p><strong>Methods: </strong>COVID-19-related tweets from February 1, 2020, to April 30, 2022 were used. The pipeline includes three modules: a named entity recognition module to detect symptoms in tweets; an entity normalization module to aggregate detected entities; and a mapping module that iteratively maps entities to Unified Medical Language System concepts. A random 500 entity samples were drawn from the final dictionary for accuracy validation. Additionally, we conducted a symptom frequency distribution analysis to compare our dictionary to a pre-defined lexicon from previous research.</p><p><strong>Results: </strong>We identified 498 480 unique symptom entity expressions from the tweets. Pre-processing reduces the number to 18 226. The final dictionary contains 38 175 unique expressions of symptoms that can be mapped to 966 UMLS concepts (accuracy = 95%). Symptom distribution analysis found that our dictionary detects more symptoms and is effective at identifying psychiatric disorders like anxiety and depression, often missed by pre-defined lexicons.</p><p><strong>Conclusions: </strong>This study advances public health research by implementing a novel, systematic pipeline for curating symptom lexicons from social media data. The final lexicon's high accuracy, validated by medical professionals, underscores the potential of this methodology to reliably interpret, and categorize vast amounts of unstructured social media data into actionable medical insights across diverse linguistic and regional landscapes.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":null,"pages":null},"PeriodicalIF":4.7,"publicationDate":"2024-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11187427/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140892377","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
To weight or not to weight? The effect of selection bias in 3 large electronic health record-linked biobanks and recommendations for practice. 加权还是不加权?3 个大型电子健康记录链接生物库中选择偏差的影响及实践建议。
IF 4.7 2区 医学
Journal of the American Medical Informatics Association Pub Date : 2024-06-20 DOI: 10.1093/jamia/ocae098
Maxwell Salvatore, Ritoban Kundu, Xu Shi, Christopher R Friese, Seunggeun Lee, Lars G Fritsche, Alison M Mondul, David Hanauer, Celeste Leigh Pearce, Bhramar Mukherjee
{"title":"To weight or not to weight? The effect of selection bias in 3 large electronic health record-linked biobanks and recommendations for practice.","authors":"Maxwell Salvatore, Ritoban Kundu, Xu Shi, Christopher R Friese, Seunggeun Lee, Lars G Fritsche, Alison M Mondul, David Hanauer, Celeste Leigh Pearce, Bhramar Mukherjee","doi":"10.1093/jamia/ocae098","DOIUrl":"10.1093/jamia/ocae098","url":null,"abstract":"<p><strong>Objectives: </strong>To develop recommendations regarding the use of weights to reduce selection bias for commonly performed analyses using electronic health record (EHR)-linked biobank data.</p><p><strong>Materials and methods: </strong>We mapped diagnosis (ICD code) data to standardized phecodes from 3 EHR-linked biobanks with varying recruitment strategies: All of Us (AOU; n = 244 071), Michigan Genomics Initiative (MGI; n = 81 243), and UK Biobank (UKB; n = 401 167). Using 2019 National Health Interview Survey data, we constructed selection weights for AOU and MGI to represent the US adult population more. We used weights previously developed for UKB to represent the UKB-eligible population. We conducted 4 common analyses comparing unweighted and weighted results.</p><p><strong>Results: </strong>For AOU and MGI, estimated phecode prevalences decreased after weighting (weighted-unweighted median phecode prevalence ratio [MPR]: 0.82 and 0.61), while UKB estimates increased (MPR: 1.06). Weighting minimally impacted latent phenome dimensionality estimation. Comparing weighted versus unweighted phenome-wide association study for colorectal cancer, the strongest associations remained unaltered, with considerable overlap in significant hits. Weighting affected the estimated log-odds ratio for sex and colorectal cancer to align more closely with national registry-based estimates.</p><p><strong>Discussion: </strong>Weighting had a limited impact on dimensionality estimation and large-scale hypothesis testing but impacted prevalence and association estimation. When interested in estimating effect size, specific signals from untargeted association analyses should be followed up by weighted analysis.</p><p><strong>Conclusion: </strong>EHR-linked biobanks should report recruitment and selection mechanisms and provide selection weights with defined target populations. Researchers should consider their intended estimands, specify source and target populations, and weight EHR-linked biobank analyses accordingly.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":null,"pages":null},"PeriodicalIF":4.7,"publicationDate":"2024-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11187425/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140917398","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A system-wide approach to digital equity: the Digital Access Coordinator program in primary care. 全系统的数字平等方法:初级保健中的数字访问协调员计划。
IF 4.7 2区 医学
Journal of the American Medical Informatics Association Pub Date : 2024-06-20 DOI: 10.1093/jamia/ocae104
Jorge A Rodriguez, Michelle Zelen, Jessica Szulak, Katie Moore, Lee Park
{"title":"A system-wide approach to digital equity: the Digital Access Coordinator program in primary care.","authors":"Jorge A Rodriguez, Michelle Zelen, Jessica Szulak, Katie Moore, Lee Park","doi":"10.1093/jamia/ocae104","DOIUrl":"10.1093/jamia/ocae104","url":null,"abstract":"<p><strong>Introduction: </strong>The transition to digital tools prompted by the pandemic made evident digital disparities. To address digital literacy gaps, we implemented a system-wide digital navigation program.</p><p><strong>Methods: </strong>The Digital Access Coordinator (DAC) program consists of 12 multilingual navigators who support patients in enrolling and using the patient portal and digital tools. We implemented the program in our primary care network which consists of 1.25 million patients across 1211 clinicians.</p><p><strong>Results: </strong>From May 2021 to November 2022, the DACs completed outreach to 16 045 patients. Of the 13 413 patients they reached, they successfully enrolled 8193 (61%) patients in the patient portal. Of those patients they enrolled, most patients were of Other race, Hispanic ethnicity, and were English-speaking (44%) and Spanish-speaking patients (44%). Using our embedded model, we increased enrollment across 7 clinics (mean increase: 21.3%, standard deviation: 9.2%). Additionally, we identified key approaches for implementing a digital navigation program.</p><p><strong>Conclusion: </strong>Organizations can support patient portal enrollment, a key part of digital health equity, by creating and prioritizing digital navigation programs.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":null,"pages":null},"PeriodicalIF":4.7,"publicationDate":"2024-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11187422/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140917394","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A comparative study of large language model-based zero-shot inference and task-specific supervised classification of breast cancer pathology reports. 基于大语言模型的乳腺癌病理报告零点推理与特定任务监督分类的比较研究。
IF 4.7 2区 医学
Journal of the American Medical Informatics Association Pub Date : 2024-06-20 DOI: 10.1093/jamia/ocae146
Madhumita Sushil, Travis Zack, Divneet Mandair, Zhiwei Zheng, Ahmed Wali, Yan-Ning Yu, Yuwei Quan, Dmytro Lituiev, Atul J Butte
{"title":"A comparative study of large language model-based zero-shot inference and task-specific supervised classification of breast cancer pathology reports.","authors":"Madhumita Sushil, Travis Zack, Divneet Mandair, Zhiwei Zheng, Ahmed Wali, Yan-Ning Yu, Yuwei Quan, Dmytro Lituiev, Atul J Butte","doi":"10.1093/jamia/ocae146","DOIUrl":"10.1093/jamia/ocae146","url":null,"abstract":"<p><strong>Objective: </strong>Although supervised machine learning is popular for information extraction from clinical notes, creating large annotated datasets requires extensive domain expertise and is time-consuming. Meanwhile, large language models (LLMs) have demonstrated promising transfer learning capability. In this study, we explored whether recent LLMs could reduce the need for large-scale data annotations.</p><p><strong>Materials and methods: </strong>We curated a dataset of 769 breast cancer pathology reports, manually labeled with 12 categories, to compare zero-shot classification capability of the following LLMs: GPT-4, GPT-3.5, Starling, and ClinicalCamel, with task-specific supervised classification performance of 3 models: random forests, long short-term memory networks with attention (LSTM-Att), and the UCSF-BERT model.</p><p><strong>Results: </strong>Across all 12 tasks, the GPT-4 model performed either significantly better than or as well as the best supervised model, LSTM-Att (average macro F1-score of 0.86 vs 0.75), with advantage on tasks with high label imbalance. Other LLMs demonstrated poor performance. Frequent GPT-4 error categories included incorrect inferences from multiple samples and from history, and complex task design, and several LSTM-Att errors were related to poor generalization to the test set.</p><p><strong>Discussion: </strong>On tasks where large annotated datasets cannot be easily collected, LLMs can reduce the burden of data labeling. However, if the use of LLMs is prohibitive, the use of simpler models with large annotated datasets can provide comparable results.</p><p><strong>Conclusions: </strong>GPT-4 demonstrated the potential to speed up the execution of clinical NLP studies by reducing the need for large annotated datasets. This may increase the utilization of NLP-based variables and outcomes in clinical studies.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":null,"pages":null},"PeriodicalIF":4.7,"publicationDate":"2024-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141428021","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Implementation of a health information technology safety classification system in the Veterans Health Administration's Informatics Patient Safety Office. 在退伍军人健康管理局信息学患者安全办公室实施医疗信息技术安全分类系统。
IF 6.4 2区 医学
Journal of the American Medical Informatics Association Pub Date : 2024-06-20 DOI: 10.1093/jamia/ocae107
Danielle Kato, Joe Lucas, Dean F Sittig
{"title":"Implementation of a health information technology safety classification system in the Veterans Health Administration's Informatics Patient Safety Office.","authors":"Danielle Kato, Joe Lucas, Dean F Sittig","doi":"10.1093/jamia/ocae107","DOIUrl":"10.1093/jamia/ocae107","url":null,"abstract":"<p><strong>Objective: </strong>Implement the 5-type health information technology (HIT) patient safety concern classification system for HIT patient safety issues reported to the Veterans Health Administration's Informatics Patient Safety Office.</p><p><strong>Materials and methods: </strong>A team of informatics safety analysts retrospectively classified 1 year of HIT patient safety issues by type of HIT patient safety concern using consensus discussions. The processes established during retrospective classification were then applied to incoming HIT safety issues moving forward.</p><p><strong>Results: </strong>Of 140 issues retrospectively reviewed, 124 met the classification criteria. The majority were HIT failures (eg, software defects) (33.1%) or configuration and implementation problems (29.8%). Unmet user needs and external system interactions accounted for 20.2% and 10.5%, respectively. Absence of HIT safety features accounted for 2.4% of issues, and 4% did not have enough information to classify.</p><p><strong>Conclusion: </strong>The 5-type HIT safety concern classification framework generated actionable categories helping organizations effectively respond to HIT patient safety risks.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":null,"pages":null},"PeriodicalIF":6.4,"publicationDate":"2024-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140960495","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Acute myocardial infarction prognosis prediction with reliable and interpretable artificial intelligence system. 利用可靠、可解释的人工智能系统预测急性心肌梗塞的预后。
IF 4.7 2区 医学
Journal of the American Medical Informatics Association Pub Date : 2024-06-20 DOI: 10.1093/jamia/ocae114
Minwook Kim, Donggil Kang, Min Sun Kim, Jeong Cheon Choe, Sun-Hack Lee, Jin Hee Ahn, Jun-Hyok Oh, Jung Hyun Choi, Han Cheol Lee, Kwang Soo Cha, Kyungtae Jang, WooR I Bong, Giltae Song, Hyewon Lee
{"title":"Acute myocardial infarction prognosis prediction with reliable and interpretable artificial intelligence system.","authors":"Minwook Kim, Donggil Kang, Min Sun Kim, Jeong Cheon Choe, Sun-Hack Lee, Jin Hee Ahn, Jun-Hyok Oh, Jung Hyun Choi, Han Cheol Lee, Kwang Soo Cha, Kyungtae Jang, WooR I Bong, Giltae Song, Hyewon Lee","doi":"10.1093/jamia/ocae114","DOIUrl":"10.1093/jamia/ocae114","url":null,"abstract":"<p><strong>Objective: </strong>Predicting mortality after acute myocardial infarction (AMI) is crucial for timely prescription and treatment of AMI patients, but there are no appropriate AI systems for clinicians. Our primary goal is to develop a reliable and interpretable AI system and provide some valuable insights regarding short, and long-term mortality.</p><p><strong>Materials and methods: </strong>We propose the RIAS framework, an end-to-end framework that is designed with reliability and interpretability at its core and automatically optimizes the given model. Using RIAS, clinicians get accurate and reliable predictions which can be used as likelihood, with global and local explanations, and \"what if\" scenarios to achieve desired outcomes as well.</p><p><strong>Results: </strong>We apply RIAS to AMI prognosis prediction data which comes from the Korean Acute Myocardial Infarction Registry. We compared FT-Transformer with XGBoost and MLP and found that FT-Transformer has superiority in sensitivity and comparable performance in AUROC and F1 score to XGBoost. Furthermore, RIAS reveals the significance of statin-based medications, beta-blockers, and age on mortality regardless of time period. Lastly, we showcase reliable and interpretable results of RIAS with local explanations and counterfactual examples for several realistic scenarios.</p><p><strong>Discussion: </strong>RIAS addresses the \"black-box\" issue in AI by providing both global and local explanations based on SHAP values and reliable predictions, interpretable as actual likelihoods. The system's \"what if\" counterfactual explanations enable clinicians to simulate patient-specific scenarios under various conditions, enhancing its practical utility.</p><p><strong>Conclusion: </strong>The proposed framework provides reliable and interpretable predictions along with counterfactual examples.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":null,"pages":null},"PeriodicalIF":4.7,"publicationDate":"2024-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11187491/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141159027","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A taxonomy for advancing systematic error analysis in multi-site electronic health record-based clinical concept extraction. 在基于多站点电子健康记录的临床概念提取中推进系统误差分析的分类法。
IF 4.7 2区 医学
Journal of the American Medical Informatics Association Pub Date : 2024-06-20 DOI: 10.1093/jamia/ocae101
Sunyang Fu, Liwei Wang, Huan He, Andrew Wen, Nansu Zong, Anamika Kumari, Feifan Liu, Sicheng Zhou, Rui Zhang, Chenyu Li, Yanshan Wang, Jennifer St Sauver, Hongfang Liu, Sunghwan Sohn
{"title":"A taxonomy for advancing systematic error analysis in multi-site electronic health record-based clinical concept extraction.","authors":"Sunyang Fu, Liwei Wang, Huan He, Andrew Wen, Nansu Zong, Anamika Kumari, Feifan Liu, Sicheng Zhou, Rui Zhang, Chenyu Li, Yanshan Wang, Jennifer St Sauver, Hongfang Liu, Sunghwan Sohn","doi":"10.1093/jamia/ocae101","DOIUrl":"10.1093/jamia/ocae101","url":null,"abstract":"<p><strong>Background: </strong>Error analysis plays a crucial role in clinical concept extraction, a fundamental subtask within clinical natural language processing (NLP). The process typically involves a manual review of error types, such as contextual and linguistic factors contributing to their occurrence, and the identification of underlying causes to refine the NLP model and improve its performance. Conducting error analysis can be complex, requiring a combination of NLP expertise and domain-specific knowledge. Due to the high heterogeneity of electronic health record (EHR) settings across different institutions, challenges may arise when attempting to standardize and reproduce the error analysis process.</p><p><strong>Objectives: </strong>This study aims to facilitate a collaborative effort to establish common definitions and taxonomies for capturing diverse error types, fostering community consensus on error analysis for clinical concept extraction tasks.</p><p><strong>Materials and methods: </strong>We iteratively developed and evaluated an error taxonomy based on existing literature, standards, real-world data, multisite case evaluations, and community feedback. The finalized taxonomy was released in both .dtd and .owl formats at the Open Health Natural Language Processing Consortium. The taxonomy is compatible with several different open-source annotation tools, including MAE, Brat, and MedTator.</p><p><strong>Results: </strong>The resulting error taxonomy comprises 43 distinct error classes, organized into 6 error dimensions and 4 properties, including model type (symbolic and statistical machine learning), evaluation subject (model and human), evaluation level (patient, document, sentence, and concept), and annotation examples. Internal and external evaluations revealed strong variations in error types across methodological approaches, tasks, and EHR settings. Key points emerged from community feedback, including the need to enhancing clarity, generalizability, and usability of the taxonomy, along with dissemination strategies.</p><p><strong>Conclusion: </strong>The proposed taxonomy can facilitate the acceleration and standardization of the error analysis process in multi-site settings, thus improving the provenance, interpretability, and portability of NLP models. Future researchers could explore the potential direction of developing automated or semi-automated methods to assist in the classification and standardization of error analysis.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":null,"pages":null},"PeriodicalIF":4.7,"publicationDate":"2024-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11187420/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140917396","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
On the utility of using the All of Us Research Program as a resource to study military service members and veterans. 关于利用 "我们大家 "研究计划作为研究军人和退伍军人的资源的实用性。
IF 6.4 2区 医学
Journal of the American Medical Informatics Association Pub Date : 2024-06-18 DOI: 10.1093/jamia/ocae153
Ben Porter
{"title":"On the utility of using the All of Us Research Program as a resource to study military service members and veterans.","authors":"Ben Porter","doi":"10.1093/jamia/ocae153","DOIUrl":"https://doi.org/10.1093/jamia/ocae153","url":null,"abstract":"<p><strong>Objectives: </strong>To illustrate the utility of the All of Us Research Program for studying military and veteran health.</p><p><strong>Materials and methods: </strong>Results were derived from the All of Us Researcher Workbench Controlled Tier v7. Specific variables examined were family history of post-traumatic stress disorder (PTSD), medical encounters, and body mass index/body size.</p><p><strong>Results: </strong>There are 37 363 military and veteran participants enrolled in the All of Us Research Program. The population is older (M = 63.3 years), White (71.3%), and male (83.2%), consistent with military and veteran populations. Participants reported a high prevalence of PTSD (13.4%), obesity (40.2%), and abdominal obesity (77.1%).</p><p><strong>Discussion and conclusion: </strong>The breadth and depth of health data from service members and veterans enrolled in the All of Us Research Program allow researchers to address pressing health questions in these populations. Future enrollment and data releases will make this an increasingly powerful and useful study for understanding military and veteran health.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":null,"pages":null},"PeriodicalIF":6.4,"publicationDate":"2024-06-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141421682","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A novel approach to patient portal activation data to power equity improvements. 病人门户激活数据的新方法,为改善公平提供动力。
IF 6.4 2区 医学
Journal of the American Medical Informatics Association Pub Date : 2024-06-17 DOI: 10.1093/jamia/ocae152
Anoop Muniyappa, Benjamin Weia, Nicole Ling, Julie O'Brien, Mariamawit Tamerat, William Daniel Soulsby, Joanne Yim, Aris Oates
{"title":"A novel approach to patient portal activation data to power equity improvements.","authors":"Anoop Muniyappa, Benjamin Weia, Nicole Ling, Julie O'Brien, Mariamawit Tamerat, William Daniel Soulsby, Joanne Yim, Aris Oates","doi":"10.1093/jamia/ocae152","DOIUrl":"https://doi.org/10.1093/jamia/ocae152","url":null,"abstract":"<p><strong>Background: </strong>There are significant disparities in access and utilization of patient portals by age, language, race, and ethnicity.</p><p><strong>Materials and methods: </strong>We developed ambulatory and inpatient portal activation equity dashboards to understand disparities in initial portal activation, identify targets for improvement, and enable monitoring of interventions over time. We selected key metrics focused on episodes of care and filters to enable high-level overviews and granular data selection to meet the needs of health system leaders and individual clinical units.</p><p><strong>Results: </strong>In addition to highlighting disparities by age, preferred language, race and ethnicity, and insurance payor, the dashboards enabled development and monitoring of interventions to improve portal activation and equity.</p><p><strong>Discussion and conclusions: </strong>Data visualization tools that provide easily accessible, timely, and customizable data can enable a variety of stakeholders to understand and address healthcare disparities, such as patient portal activation. Further institutional efforts are needed to address the persistent inequities highlighted by these dashboards.</p>","PeriodicalId":50016,"journal":{"name":"Journal of the American Medical Informatics Association","volume":null,"pages":null},"PeriodicalIF":6.4,"publicationDate":"2024-06-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141421680","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
相关产品
×
本文献相关产品
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信