Jennifer M. Radin, Julia Moore Vogel, Felipe Delgado, Erin Coughlin, Matteo Gadaleta, Jay A. Pandit, Steven R. Steinhubl
{"title":"Long-term changes in wearable sensor data in people with and without Long Covid","authors":"Jennifer M. Radin, Julia Moore Vogel, Felipe Delgado, Erin Coughlin, Matteo Gadaleta, Jay A. Pandit, Steven R. Steinhubl","doi":"10.1038/s41746-024-01238-x","DOIUrl":"10.1038/s41746-024-01238-x","url":null,"abstract":"To better understand the impact of Long COVID on an individual, we explored changes in daily wearable data (step count, resting heart rate (RHR), and sleep quantity) for up to one year in individuals relative to their pre-infection baseline among 279 people with and 274 without long COVID. Participants with Long COVID, defined as symptoms lasting for 30 days or longer, following a SARS-CoV-2 infection had significantly different RHR and activity trajectories than those who did not report Long COVID and were also more likely to be women, younger, unvaccinated, and report more acute-phase (first 2 weeks) symptoms than those without Long COVID. Demographic, vaccine, and acute-phase sensor data differences could be used for early identification of individuals most likely to develop Long COVID complications and track objective evidence of the therapeutic efficacy of any interventions. Trial Registration: https://classic.clinicaltrials.gov/ct2/show/NCT04336020 .","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":null,"pages":null},"PeriodicalIF":12.4,"publicationDate":"2024-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s41746-024-01238-x.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142174955","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Results and implications for generative AI in a large introductory biomedical and health informatics course","authors":"William Hersh, Kate Fultz Hollis","doi":"10.1038/s41746-024-01251-0","DOIUrl":"10.1038/s41746-024-01251-0","url":null,"abstract":"Generative artificial intelligence (AI) systems have performed well at many biomedical tasks, but few studies have assessed their performance directly compared to students in higher-education courses. We compared student knowledge-assessment scores with prompting of 6 large-language model (LLM) systems as they would be used by typical students in a large online introductory course in biomedical and health informatics that is taken by graduate, continuing education, and medical students. The state-of-the-art LLM systems were prompted to answer multiple-choice questions (MCQs) and final exam questions. We compared the scores for 139 students (30 graduate students, 85 continuing education students, and 24 medical students) to the LLM systems. All of the LLMs scored between the 50th and 75th percentiles of students for MCQ and final exam questions. The performance of LLMs raises questions about student assessment in higher education, especially in courses that are knowledge-based and online.","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":null,"pages":null},"PeriodicalIF":12.4,"publicationDate":"2024-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s41746-024-01251-0.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142231291","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A randomized controlled trial investigating experiential virtual reality communication on prudent antibiotic use","authors":"Adéla Plechatá, Guido Makransky, Robert Böhm","doi":"10.1038/s41746-024-01240-3","DOIUrl":"10.1038/s41746-024-01240-3","url":null,"abstract":"Antimicrobial resistance (AMR) is a global health threat. This randomized controlled trial evaluates the impact of experiential virtual reality (VR) versus information provision via VR or leaflet on prudent antibiotic use. A total of 249 (239 analyzed) participants were randomized into three conditions: VR Information + Experience, VR Information, or Leaflet Information. All participants received AMR information, while those in the VR Information + Experience condition additionally engaged in a game, making treatment decisions for their virtual avatar’s infection. Participants in the VR Information + Experience condition showed a significant increase in prudent use intentions from baseline (d = 1.48). This increase was significantly larger compared to the VR Information (d = 0.50) and Leaflet Information (d = 0.79) conditions. The increase in intentions from baseline remained significant at follow-up in the VR Information + Experience condition (d = 1.25). Experiential VR communication shows promise for promoting prudent antibiotics use.","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":null,"pages":null},"PeriodicalIF":12.4,"publicationDate":"2024-09-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s41746-024-01240-3.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142170392","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jack Gallifant, Danielle S. Bitterman, Leo Anthony Celi, Judy W. Gichoya, Joao Matos, Liam G. McCoy, Robin L. Pierce
{"title":"Ethical debates amidst flawed healthcare artificial intelligence metrics","authors":"Jack Gallifant, Danielle S. Bitterman, Leo Anthony Celi, Judy W. Gichoya, Joao Matos, Liam G. McCoy, Robin L. Pierce","doi":"10.1038/s41746-024-01242-1","DOIUrl":"10.1038/s41746-024-01242-1","url":null,"abstract":"Healthcare AI faces an ethical dilemma between selective and equitable deployment, exacerbated by flawed performance metrics. These metrics inadequately capture real-world complexities and biases, leading to premature assertions of effectiveness. Improved evaluation practices, including continuous monitoring and silent evaluation periods, are crucial. To address these fundamental shortcomings, a paradigm shift in AI assessment is needed, prioritizing actual patient outcomes over conventional benchmarking.","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":null,"pages":null},"PeriodicalIF":12.4,"publicationDate":"2024-09-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s41746-024-01242-1.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142170369","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
James Truslow, Angela Spillane, Huiming Lin, Katherine Cyr, Adeeti Ullal, Edith Arnold, Ron Huang, Laura Rhodes, Jennifer Block, Jamie Stark, James Kretlow, Alexis L. Beatty, Andreas Werdich, Deepali Bankar, Matt Bianchi, Ian Shapiro, Jaime Villalpando, Sharon Ravindran, Irida Mance, Adam Phillips, John Earl, Rahul C. Deo, Sumbul A. Desai, Calum A. MacRae
{"title":"Understanding activity and physiology at scale: The Apple Heart & Movement Study","authors":"James Truslow, Angela Spillane, Huiming Lin, Katherine Cyr, Adeeti Ullal, Edith Arnold, Ron Huang, Laura Rhodes, Jennifer Block, Jamie Stark, James Kretlow, Alexis L. Beatty, Andreas Werdich, Deepali Bankar, Matt Bianchi, Ian Shapiro, Jaime Villalpando, Sharon Ravindran, Irida Mance, Adam Phillips, John Earl, Rahul C. Deo, Sumbul A. Desai, Calum A. MacRae","doi":"10.1038/s41746-024-01187-5","DOIUrl":"10.1038/s41746-024-01187-5","url":null,"abstract":"Physical activity or structured exercise is beneficial in a wide range of circumstances. Nevertheless, individual-level data on differential responses to various types of activity are not yet sufficient in scale, duration or level of annotation to understand the mechanisms of discrete outcomes nor to support personalized recommendations. The Apple Heart & Movement Study was designed to passively collect the dense physiologic data accessible on Apple Watch and iPhone from a large real-world cohort distributed across the US in order to address these knowledge gaps.","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":null,"pages":null},"PeriodicalIF":12.4,"publicationDate":"2024-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s41746-024-01187-5.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142160346","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Benjamin Rader, Neil K. R. Sehgal, Julie Michelman, Stefan Mellem, Marinanicole D. Schultheiss, Tom Hoddes, Jamie MacFarlane, Geoff Clark, Shawn O’Banion, Paul Eastham, Gaurav Tuli, James A. Taylor, John S. Brownstein
{"title":"Adherence to non-pharmaceutical interventions following COVID-19 vaccination: a federated cohort study","authors":"Benjamin Rader, Neil K. R. Sehgal, Julie Michelman, Stefan Mellem, Marinanicole D. Schultheiss, Tom Hoddes, Jamie MacFarlane, Geoff Clark, Shawn O’Banion, Paul Eastham, Gaurav Tuli, James A. Taylor, John S. Brownstein","doi":"10.1038/s41746-024-01223-4","DOIUrl":"10.1038/s41746-024-01223-4","url":null,"abstract":"In pandemic mitigation, strategies such as social distancing and mask-wearing are vital to prevent disease resurgence. Yet, monitoring adherence is challenging, as individuals might be reluctant to share behavioral data with public health authorities. To address this challenge and demonstrate a framework for conducting observational research with sensitive data in a privacy-conscious manner, we employ a privacy-centric epidemiological study design: the federated cohort. This approach leverages recent computational advances to allow for distributed participants to contribute to a prospective, observational research study while maintaining full control of their data. We apply this strategy here to explore pandemic intervention adherence patterns. Participants (n = 3808) were enrolled in our federated cohort via the “Google Health Studies” mobile application. Participants completed weekly surveys and contributed empirically measured mobility data from their Android devices between November 2020 to August 2021. Using federated analytics, differential privacy, and secure aggregation, we analyzed data in five 6-week periods, encompassing the pre- and post-vaccination phases. Our results showed that participants largely utilized non-pharmaceutical intervention strategies until they were fully vaccinated against COVID-19, except for individuals without plans to become vaccinated. Furthermore, this project offers a blueprint for conducting a federated cohort study and engaging in privacy-preserving research during a public health emergency.","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":null,"pages":null},"PeriodicalIF":12.4,"publicationDate":"2024-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s41746-024-01223-4.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142160355","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Gregory Holste, Mingquan Lin, Ruiwen Zhou, Fei Wang, Lei Liu, Qi Yan, Sarah H. Van Tassel, Kyle Kovacs, Emily Y. Chew, Zhiyong Lu, Zhangyang Wang, Yifan Peng
{"title":"Author Correction: Harnessing the power of longitudinal medical imaging for eye disease prognosis using Transformer-based sequence modeling","authors":"Gregory Holste, Mingquan Lin, Ruiwen Zhou, Fei Wang, Lei Liu, Qi Yan, Sarah H. Van Tassel, Kyle Kovacs, Emily Y. Chew, Zhiyong Lu, Zhangyang Wang, Yifan Peng","doi":"10.1038/s41746-024-01243-0","DOIUrl":"10.1038/s41746-024-01243-0","url":null,"abstract":"","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":null,"pages":null},"PeriodicalIF":12.4,"publicationDate":"2024-09-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s41746-024-01243-0.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142160361","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
João Guerreiro, Roger Garriga, Toni Lozano Bagén, Brihat Sharma, Niranjan S. Karnik, Aleksandar Matić
{"title":"Transatlantic transferability and replicability of machine-learning algorithms to predict mental health crises","authors":"João Guerreiro, Roger Garriga, Toni Lozano Bagén, Brihat Sharma, Niranjan S. Karnik, Aleksandar Matić","doi":"10.1038/s41746-024-01203-8","DOIUrl":"10.1038/s41746-024-01203-8","url":null,"abstract":"Transferring and replicating predictive algorithms across healthcare systems constitutes a unique yet crucial challenge that needs to be addressed to enable the widespread adoption of machine learning in healthcare. In this study, we explored the impact of important differences across healthcare systems and the associated Electronic Health Records (EHRs) on machine-learning algorithms to predict mental health crises, up to 28 days in advance. We evaluated both the transferability and replicability of such machine learning models, and for this purpose, we trained six models using features and methods developed on EHR data from the Birmingham and Solihull Mental Health NHS Foundation Trust in the UK. These machine learning models were then used to predict the mental health crises of 2907 patients seen at the Rush University System for Health in the US between 2018 and 2020. The best one was trained on a combination of US-specific structured features and frequency features from anonymized patient notes and achieved an AUROC of 0.837. A model with comparable performance, originally trained using UK structured data, was transferred and then tuned using US data, achieving an AUROC of 0.826. Our findings establish the feasibility of transferring and replicating machine learning models to predict mental health crises across diverse hospital systems.","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":null,"pages":null},"PeriodicalIF":12.4,"publicationDate":"2024-09-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s41746-024-01203-8.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142158948","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Gongbo Zhang, Qiao Jin, Yiliang Zhou, Song Wang, Betina Idnay, Yiming Luo, Elizabeth Park, Jordan G. Nestor, Matthew E. Spotnitz, Ali Soroush, Thomas R. Campion Jr., Zhiyong Lu, Chunhua Weng, Yifan Peng
{"title":"Closing the gap between open source and commercial large language models for medical evidence summarization","authors":"Gongbo Zhang, Qiao Jin, Yiliang Zhou, Song Wang, Betina Idnay, Yiming Luo, Elizabeth Park, Jordan G. Nestor, Matthew E. Spotnitz, Ali Soroush, Thomas R. Campion Jr., Zhiyong Lu, Chunhua Weng, Yifan Peng","doi":"10.1038/s41746-024-01239-w","DOIUrl":"10.1038/s41746-024-01239-w","url":null,"abstract":"Large language models (LLMs) hold great promise in summarizing medical evidence. Most recent studies focus on the application of proprietary LLMs. Using proprietary LLMs introduces multiple risk factors, including a lack of transparency and vendor dependency. While open-source LLMs allow better transparency and customization, their performance falls short compared to the proprietary ones. In this study, we investigated to what extent fine-tuning open-source LLMs can further improve their performance. Utilizing a benchmark dataset, MedReview, consisting of 8161 pairs of systematic reviews and summaries, we fine-tuned three broadly-used, open-sourced LLMs, namely PRIMERA, LongT5, and Llama-2. Overall, the performance of open-source models was all improved after fine-tuning. The performance of fine-tuned LongT5 is close to GPT-3.5 with zero-shot settings. Furthermore, smaller fine-tuned models sometimes even demonstrated superior performance compared to larger zero-shot models. The above trends of improvement were manifested in both a human evaluation and a larger-scale GPT4-simulated evaluation.","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":null,"pages":null},"PeriodicalIF":12.4,"publicationDate":"2024-09-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s41746-024-01239-w.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142160323","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Klavdiia Naumova, Arnout Devos, Sai Praneeth Karimireddy, Martin Jaggi, Mary-Anne Hartley
{"title":"MyThisYourThat for interpretable identification of systematic bias in federated learning for biomedical images","authors":"Klavdiia Naumova, Arnout Devos, Sai Praneeth Karimireddy, Martin Jaggi, Mary-Anne Hartley","doi":"10.1038/s41746-024-01226-1","DOIUrl":"10.1038/s41746-024-01226-1","url":null,"abstract":"Distributed collaborative learning is a promising approach for building predictive models for privacy-sensitive biomedical images. Here, several data owners (clients) train a joint model without sharing their original data. However, concealed systematic biases can compromise model performance and fairness. This study presents MyThisYourThat (MyTH) approach, which adapts an interpretable prototypical part learning network to a distributed setting, enabling each client to visualize feature differences learned by others on their own image: comparing one client’s ''This’ with others’ ''That’. Our setting demonstrates four clients collaboratively training two diagnostic classifiers on a benchmark X-ray dataset. Without data bias, the global model reaches 74.14% balanced accuracy for cardiomegaly and 74.08% for pleural effusion. We show that with systematic visual bias in one client, the performance of global models drops to near-random. We demonstrate how differences between local and global prototypes reveal biases and allow their visualization on each client’s data without compromising privacy.","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":null,"pages":null},"PeriodicalIF":12.4,"publicationDate":"2024-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s41746-024-01226-1.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142143891","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}