{"title":"Correction to: Evaluating the ChatGPT family of models for biomedical reasoning and classification.","authors":"","doi":"10.1093/jamia/ocae083","DOIUrl":"https://doi.org/10.1093/jamia/ocae083","url":null,"abstract":"","PeriodicalId":236137,"journal":{"name":"Journal of the American Medical Informatics Association : JAMIA","volume":"10 4","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-04-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140727910","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Engaging knowers in the design and implementation of digital health innovations.","authors":"Suzanne Bakken","doi":"10.1093/jamia/ocae051","DOIUrl":"https://doi.org/10.1093/jamia/ocae051","url":null,"abstract":"","PeriodicalId":236137,"journal":{"name":"Journal of the American Medical Informatics Association : JAMIA","volume":"689 4","pages":"795-796"},"PeriodicalIF":0.0,"publicationDate":"2024-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140782436","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Span-based Model for Extracting Overlapping PICO Entities from RCT Publications","authors":"Gongbo Zhang, Yiliang Zhou, Yan Hu, Hua Xu, Chunhua Weng, Yifan Peng","doi":"10.48550/arXiv.2401.06791","DOIUrl":"https://doi.org/10.48550/arXiv.2401.06791","url":null,"abstract":"OBJECTIVES\u0000Extracting PICO (Populations, Interventions, Comparison, and Outcomes) entities is fundamental to evidence retrieval. We present a novel method, PICOX, to extract overlapping PICO entities.\u0000\u0000\u0000MATERIALS AND METHODS\u0000PICOX first identifies entities by assessing whether a word marks the beginning or conclusion of an entity. Then, it uses a multi-label classifier to assign one or more PICO labels to a span candidate. PICOX was evaluated using one of the best-performing baselines, EBM-NLP, and three more datasets, ie, PICO-Corpus and RCT publications on Alzheimer's Disease or COVID-19, using entity-level precision, recall, and F1 scores.\u0000\u0000\u0000RESULTS\u0000PICOX achieved superior precision, recall, and F1 scores across the board, with the micro F1 score improving from 45.05 to 50.87 (p ≪ .01). On the PICO-Corpus, PICOX obtained higher recall and F1 scores than the baseline and improved the micro recall score from 56.66 to 67.33. On the COVID-19 dataset, PICOX also outperformed the baseline and improved the micro F1 score from 77.10 to 80.32. On the AD dataset, PICOX demonstrated comparable F1 scores with higher precision when compared to the baseline.\u0000\u0000\u0000CONCLUSION\u0000PICOX excels in identifying overlapping entities and consistently surpasses a leading baseline across multiple datasets. Ablation studies reveal that its data augmentation strategy effectively minimizes false positives and improves precision.","PeriodicalId":236137,"journal":{"name":"Journal of the American Medical Informatics Association : JAMIA","volume":"10 3","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-01-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140512196","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yao Yao, Hanchu Zhou, Zhidong Cao, D. Zeng, Qingpeng Zhang
{"title":"Optimal adaptive nonpharmaceutical interventions to mitigate the outbreak of respiratory infections following the COVID-19 pandemic: a deep reinforcement learning study in Hong Kong, China","authors":"Yao Yao, Hanchu Zhou, Zhidong Cao, D. Zeng, Qingpeng Zhang","doi":"10.2139/ssrn.4243792","DOIUrl":"https://doi.org/10.2139/ssrn.4243792","url":null,"abstract":"BACKGROUND\u0000Long-lasting nonpharmaceutical interventions (NPIs) suppressed the infection of COVID-19 but came at a substantial economic cost and the elevated risk of the outbreak of respiratory infectious diseases (RIDs) following the pandemic. Policymakers need data-driven evidence to guide the relaxation with adaptive NPIs that consider the risk of both COVID-19 and other RIDs outbreaks, as well as the available healthcare resources.\u0000\u0000\u0000METHODS\u0000Combining the COVID-19 data of the sixth wave in Hong Kong between May 31, 2022 and August 28, 2022, 6-year epidemic data of other RIDs (2014-2019), and the healthcare resources data, we constructed compartment models to predict the epidemic curves of RIDs after the COVID-19-targeted NPIs. A deep reinforcement learning (DRL) model was developed to learn the optimal adaptive NPIs strategies to mitigate the outbreak of RIDs after COVID-19-targeted NPIs are lifted with minimal health and economic cost. The performance was validated by simulations of 1000 days starting August 29, 2022. We also extended the model to Beijing context.\u0000\u0000\u0000FINDINGS\u0000Without any NPIs, Hong Kong experienced a major COVID-19 resurgence far exceeding the hospital bed capacity. Simulation results showed that the proposed DRL-based adaptive NPIs successfully suppressed the outbreak of COVID-19 and other RIDs to lower than capacity. DRL carefully controlled the epidemic curve to be close to the full capacity so that herd immunity can be reached in a relatively short period with minimal cost. DRL derived more stringent adaptive NPIs in Beijing.\u0000\u0000\u0000INTERPRETATION\u0000DRL is a feasible method to identify the optimal adaptive NPIs that lead to minimal health and economic cost by facilitating gradual herd immunity of COVID-19 and mitigating the other RIDs outbreaks without overwhelming the hospitals. The insights can be extended to other countries/regions.","PeriodicalId":236137,"journal":{"name":"Journal of the American Medical Informatics Association : JAMIA","volume":"14 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116024810","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Vince C. Hartman, Sanika S. Bapat, M. Weiner, B. Navi, E. Sholle, T. Campion
{"title":"A Method to Automate the Discharge Summary Hospital Course for Neurology Patients","authors":"Vince C. Hartman, Sanika S. Bapat, M. Weiner, B. Navi, E. Sholle, T. Campion","doi":"10.48550/arXiv.2305.06416","DOIUrl":"https://doi.org/10.48550/arXiv.2305.06416","url":null,"abstract":"OBJECTIVE\u0000Generation of automated clinical notes has been posited as a strategy to mitigate physician burnout. In particular, an automated narrative summary of a patient's hospital stay could supplement the hospital course section of the discharge summary that inpatient physicians document in electronic health record (EHR) systems. In the current study, we developed and evaluated an automated method for summarizing the hospital course section using encoder-decoder sequence-to-sequence transformer models.\u0000\u0000\u0000MATERIALS AND METHODS\u0000We fine-tuned BERT and BART models and optimized for factuality through constraining beam search, which we trained and tested using EHR data from patients admitted to the neurology unit of an academic medical center.\u0000\u0000\u0000RESULTS\u0000The approach demonstrated good ROUGE scores with an R-2 of 13.76. In a blind evaluation, 2 board-certified physicians rated 62% of the automated summaries as meeting the standard of care, which suggests the method may be useful clinically.\u0000\u0000\u0000DISCUSSION AND CONCLUSION\u0000To our knowledge, this study is among the first to demonstrate an automated method for generating a discharge summary hospital course that approaches a quality level of what a physician would write.","PeriodicalId":236137,"journal":{"name":"Journal of the American Medical Informatics Association : JAMIA","volume":"46 29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127536158","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The Standard Problem","authors":"Enrico W. Coiera","doi":"10.48550/arXiv.2304.09114","DOIUrl":"https://doi.org/10.48550/arXiv.2304.09114","url":null,"abstract":"OBJECTIVE\u0000This article proposes a framework to support the scientific research of standards so that they can be better measured, evaluated, and designed.\u0000\u0000\u0000METHODS\u0000Beginning with the notion of common models, the framework describes the general standard problem-the seeming impossibility of creating a singular, persistent, and definitive standard which is not subject to change over time in an open system.\u0000\u0000\u0000RESULTS\u0000The standard problem arises from uncertainty driven by variations in operating context, standard quality, differences in implementation, and drift over time. As a result, fitting work using conformance services is needed to repair these gaps between a standard and what is required for real-world use. To guide standards design and repair, a framework for measuring performance in context is suggested, based on signal detection theory and technomarkers. Based on the type of common model in operation, different conformance strategies are identified: (1) Universal conformance (all agents access the same standard); (2) Mediated conformance (an interoperability layer supports heterogeneous agents); and (3) Localized conformance (autonomous adaptive agents manage their own needs). Conformance methods include incremental design, modular design, adaptors, and creating interactive and adaptive agents.\u0000\u0000\u0000DISCUSSION\u0000Machine learning should have a major role in adaptive fitting. Research to guide the choice and design of conformance services may focus on the stability and homogeneity of shared tasks, and whether common models are shared ahead of time or adjusted at task time.\u0000\u0000\u0000CONCLUSION\u0000This analysis conceptually decouples interoperability and standardization. While standards facilitate interoperability, interoperability is achievable without standardization.","PeriodicalId":236137,"journal":{"name":"Journal of the American Medical Informatics Association : JAMIA","volume":"47 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-04-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122354235","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Siqi Li, Pinyan Liu, G. G. Nascimento, Xinru Wang, F. Leite, Bibhas Chakraborty, Chuan Hong, Yilin Ning, F. Xie, Zhen Ling Teo, D. Ting, Hamed Haddadi, M. Ong, Marco Aur'elio Peres, Nan Liu
{"title":"Federated and distributed learning applications for electronic health records and structured medical data: A scoping review","authors":"Siqi Li, Pinyan Liu, G. G. Nascimento, Xinru Wang, F. Leite, Bibhas Chakraborty, Chuan Hong, Yilin Ning, F. Xie, Zhen Ling Teo, D. Ting, Hamed Haddadi, M. Ong, Marco Aur'elio Peres, Nan Liu","doi":"10.48550/arXiv.2304.07310","DOIUrl":"https://doi.org/10.48550/arXiv.2304.07310","url":null,"abstract":"OBJECTIVES\u0000Federated learning (FL) has gained popularity in clinical research in recent years to facilitate privacy-preserving collaboration. Structured data, one of the most prevalent forms of clinical data, has experienced significant growth in volume concurrently, notably with the widespread adoption of electronic health records in clinical practice. This review examines FL applications on structured medical data, identifies contemporary limitations, and discusses potential innovations.\u0000\u0000\u0000MATERIALS AND METHODS\u0000We searched 5 databases, SCOPUS, MEDLINE, Web of Science, Embase, and CINAHL, to identify articles that applied FL to structured medical data and reported results following the PRISMA guidelines. Each selected publication was evaluated from 3 primary perspectives, including data quality, modeling strategies, and FL frameworks.\u0000\u0000\u0000RESULTS\u0000Out of the 1193 papers screened, 34 met the inclusion criteria, with each article consisting of one or more studies that used FL to handle structured clinical/medical data. Of these, 24 utilized data acquired from electronic health records, with clinical predictions and association studies being the most common clinical research tasks that FL was applied to. Only one article exclusively explored the vertical FL setting, while the remaining 33 explored the horizontal FL setting, with only 14 discussing comparisons between single-site (local) and FL (global) analysis.\u0000\u0000\u0000CONCLUSIONS\u0000The existing FL applications on structured medical data lack sufficient evaluations of clinically meaningful benefits, particularly when compared to single-site analyses. Therefore, it is crucial for future FL applications to prioritize clinical motivations and develop designs and methodologies that can effectively support and aid clinical practice and research.","PeriodicalId":236137,"journal":{"name":"Journal of the American Medical Informatics Association : JAMIA","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-04-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"120975234","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The impact of the COVID-19 pandemic on daily rhythms","authors":"N. Luong, I. Barnett, Talayeh Aledavood","doi":"10.48550/arXiv.2303.04535","DOIUrl":"https://doi.org/10.48550/arXiv.2303.04535","url":null,"abstract":"OBJECTIVE\u0000The COVID-19 pandemic has significantly impacted daily activity rhythms and life routines with people adjusting to new work schedules, exercise routines, and other everyday life activities. This study examines temporal changes in daily activity rhythms and routines during the COVID-19 pandemic, emphasizing disproportionate changes among working adult subgroups.\u0000\u0000\u0000MATERIALS AND METHODS\u0000In June 2021, we conducted a year-long study to collect high-resolution fitness tracker data and questionnaire responses from 128 working adults. Questionnaire data were analyzed to explore changes in exercise and work routines during the pandemic. We build temporal distributions of daily step counts to quantify their daily movement rhythms, then measure their consistency over time using the inverse of the Earth mover's distance. Linear mixed-effects models were employed to compare movement rhythm variability among subpopulations.\u0000\u0000\u0000RESULTS\u0000During the pandemic, our cohort exhibited a shift in exercise routines, with a decrease in nonwalking physical exercises, while walking remained unchanged. Migrants and those living alone had less consistent daily movement rhythms compared to others. Those preferring on-site work maintained more consistent daily movement rhythms. Men and migrants returned to work more quickly after pandemic restriction measures were eased.\u0000\u0000\u0000DISCUSSION\u0000Our findings quantitatively show the pandemic's unequal impact on different subpopulations. This study opens new research avenues to explore why certain groups return to on-site work, exercise levels, or daily movement rhythms more slowly compared to prepandemic times.\u0000\u0000\u0000CONCLUSIONS\u0000Considering the pandemic's unequal impact on subpopulations, organizations and policymakers should address diverse needs and offer tailored support during future crises.","PeriodicalId":236137,"journal":{"name":"Journal of the American Medical Informatics Association : JAMIA","volume":"70 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126044728","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ian M. Campbell, D. Karavite, Morgan L. McManus, Frederick C Cusick, David C. Junod, Sarah E. Sheppard, E. Lourie, Eric D. Shelov, H. Hakonarson, A. Luberti, Naveen Muthu, R. Grundmeier
{"title":"Clinical Decision Support with a Comprehensive in-EHR Patient Tracking System Improves Genetic Testing Follow Up","authors":"Ian M. Campbell, D. Karavite, Morgan L. McManus, Frederick C Cusick, David C. Junod, Sarah E. Sheppard, E. Lourie, Eric D. Shelov, H. Hakonarson, A. Luberti, Naveen Muthu, R. Grundmeier","doi":"10.1101/2023.01.24.23284923","DOIUrl":"https://doi.org/10.1101/2023.01.24.23284923","url":null,"abstract":"Objective We sought to develop and evaluate an electronic health record (EHR) genetic testing tracking system to address the barriers and limitations of existing spreadsheet-based workarounds. Materials and Methods We evaluated the spreadsheet-based system using mixed effects logistic regression to identify factors associated with delayed follow up. These factors informed the design of an EHR-integrated genetic testing tracking system. After deployment we assessed the system in two ways. We analyzed EHR access logs and note data to assess patient outcomes and performed semi-structured interviews with system users to identify impact of the new system on work. Results We found that patient-reported race was a significant predictor of documented genetic testing follow up, indicating a possible inequity in care. We implemented a CDS system including a patient data capture form and management dashboard to facilitate important care tasks. The system significantly speeded review of results and significantly increased documentation of follow-up recommendations. Interviews with system users identified key team members ensuring success and revealed that the system addresses a number of sociotechnical factors that collectively result in safer and more efficient care. Discussion Our new tracking system ended decades of workarounds for identifying and communicating test results and improved clinical workflows. Interview participants related that the system decreased cognitive and time burden which allowed them to focus on direct patient interaction. Conclusion By assembling a multidisciplinary team, we designed a novel patient tracking system that improves genetic testing follow up. Similar approaches may be effective in other clinical settings.","PeriodicalId":236137,"journal":{"name":"Journal of the American Medical Informatics Association : JAMIA","volume":"41 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-01-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115873341","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
K. Lybarger, Nicholas J. Dobbins, Ritche Long, Angad Singh, Patrick Wedgeworth, Özlem Ozuner, Meliha Yetisgen-Yildiz
{"title":"Leveraging Natural Language Processing to Augment Structured Social Determinants of Health Data in the Electronic Health Record","authors":"K. Lybarger, Nicholas J. Dobbins, Ritche Long, Angad Singh, Patrick Wedgeworth, Özlem Ozuner, Meliha Yetisgen-Yildiz","doi":"10.48550/arXiv.2212.07538","DOIUrl":"https://doi.org/10.48550/arXiv.2212.07538","url":null,"abstract":"OBJECTIVE\u0000Social determinants of health (SDOH) impact health outcomes and are documented in the electronic health record (EHR) through structured data and unstructured clinical notes. However, clinical notes often contain more comprehensive SDOH information, detailing aspects such as status, severity, and temporality. This work has two primary objectives: (1) develop a natural language processing information extraction model to capture detailed SDOH information and (2) evaluate the information gain achieved by applying the SDOH extractor to clinical narratives and combining the extracted representations with existing structured data.\u0000\u0000\u0000MATERIALS AND METHODS\u0000We developed a novel SDOH extractor using a deep learning entity and relation extraction architecture to characterize SDOH across various dimensions. In an EHR case study, we applied the SDOH extractor to a large clinical data set with 225 089 patients and 430 406 notes with social history sections and compared the extracted SDOH information with existing structured data.\u0000\u0000\u0000RESULTS\u0000The SDOH extractor achieved 0.86 F1 on a withheld test set. In the EHR case study, we found extracted SDOH information complements existing structured data with 32% of homeless patients, 19% of current tobacco users, and 10% of drug users only having these health risk factors documented in the clinical narrative.\u0000\u0000\u0000CONCLUSIONS\u0000Utilizing EHR data to identify SDOH health risk factors and social needs may improve patient care and outcomes. Semantic representations of text-encoded SDOH information can augment existing structured data, and this more comprehensive SDOH representation can assist health systems in identifying and addressing these social needs.","PeriodicalId":236137,"journal":{"name":"Journal of the American Medical Informatics Association : JAMIA","volume":"165 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-12-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123004659","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}