Anna Devon-Sand MPH , Rory Sayres PhD , Yun Liu PhD , Patricia Strachan MSc , Margaret A. Smith MBA , Trinh Nguyen MA , Justin M. Ko MD , Steven Lin MD
{"title":"A Multiparty Collaboration to Engage Diverse Populations in Community-Centered Artificial Intelligence Research","authors":"Anna Devon-Sand MPH , Rory Sayres PhD , Yun Liu PhD , Patricia Strachan MSc , Margaret A. Smith MBA , Trinh Nguyen MA , Justin M. Ko MD , Steven Lin MD","doi":"10.1016/j.mcpdig.2024.07.001","DOIUrl":"10.1016/j.mcpdig.2024.07.001","url":null,"abstract":"<div><p>Artificial intelligence (AI)-enabled technology has the potential to expand access to high-quality health information and health care services. Learning how diverse users interact with technology enables improvements to the AI model and the user interface, maximizing its potential benefit for a greater number of people. This narrative describes how technology developers, academic researchers, and representatives from a community-based organization collaborated to conduct a community-centered project on emerging health technologies. Our project team comprised representatives from Stanford Medicine, Google, and Santa Clara Family Health Plan’s Blanca Alvarado Community Resource Center. We aimed to understand the usability and acceptability of an AI-driven dermatology tool among East San Jose, California, community members. Specifically, our objectives were as follows: to test a model for cross-sector research of AI-based health technology; to determine the utility of the tool in an ethnically and age-diverse population; to obtain in-depth user experience feedback from participants recruited during community events; to offer free skin health consultations; and to provide resources for receiving follow-up care. We describe a collaborative approach in which each party contributed expertise: knowledge of the community from the community health partner, clinical expertise from the academic research institution, and software and AI expertise from the technology company. Through an iterative process, we identified important community needs, including technological, language, and privacy support. Our approach allowed us to recruit and engage a diverse cohort of participants, over 70% of whom preferred a language other than English. We distill learnings from planning and executing this case study that may help other collaborators bridge the gap between academia, industry, and community in AI health care innovation.</p></div>","PeriodicalId":74127,"journal":{"name":"Mayo Clinic Proceedings. Digital health","volume":"2 3","pages":"Pages 463-469"},"PeriodicalIF":0.0,"publicationDate":"2024-07-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2949761224000713/pdfft?md5=7070082b704aa5765c6681bfe1a2ee2d&pid=1-s2.0-S2949761224000713-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142050372","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Anjali Rajagopal MBBS , Shant Ayanian MD, MS , Alexander J. Ryu MD , Ray Qian MD , Sean R. Legler MD , Eric A. Peeler MD , Meltiady Issa MD, MBA , Trevor J. Coons MHA , Kensaku Kawamoto MD, PhD, MHS
{"title":"Machine Learning Operations in Health Care: A Scoping Review","authors":"Anjali Rajagopal MBBS , Shant Ayanian MD, MS , Alexander J. Ryu MD , Ray Qian MD , Sean R. Legler MD , Eric A. Peeler MD , Meltiady Issa MD, MBA , Trevor J. Coons MHA , Kensaku Kawamoto MD, PhD, MHS","doi":"10.1016/j.mcpdig.2024.06.009","DOIUrl":"10.1016/j.mcpdig.2024.06.009","url":null,"abstract":"<div><p>The use of machine learning tools in health care is rapidly expanding. However, the processes that support these tools in deployment, that is, machine learning operations, are still emerging. The purpose of this work was not only to provide a comprehensive synthesis of existing literature in the field but also to identify gaps and offer insights for adoption in clinical practice. A scoping review was conducted using the MEDLINE, PubMed, Google Scholar, Embase, and Scopus databases. We used MeSH and non-MeSH search terms to identify pertinent articles, with the authors performing 2 screening phases and assigning relevance scores: 148 English language articles most salient to the review were eligible for inclusion; 98 offered the most unique information and these were supplemented by 50 additional sources, yielding 148 references. From the 148 references, we distilled 7 key topic areas, based on a synthesis of the available literature and how that aligned with practitioner needs. The 7 topic areas were machine learning model monitoring; automated retraining systems; ethics, equity, and bias; clinical workflow integration; infrastructure, human resources, and technology stack; regulatory considerations; and financial considerations. This review provides an overview of best practices and knowledge gaps of this domain in health care and identifies the strengths and weaknesses of the literature, which may be useful to health care machine learning practitioners and consumers.</p></div>","PeriodicalId":74127,"journal":{"name":"Mayo Clinic Proceedings. Digital health","volume":"2 3","pages":"Pages 421-437"},"PeriodicalIF":0.0,"publicationDate":"2024-07-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2949761224000701/pdfft?md5=6a899b6234621008d437c9cd437a3eaa&pid=1-s2.0-S2949761224000701-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141961078","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Sungrim Moon PhD , Yuqi Wu PhD , Jay B. Doughty MHA , Mark L. Wieland MD, MPH , Lindsey M. Philpot PhD, MPH , Jungwei W. Fan PhD , Jane W. Njeru MB, ChB
{"title":"Automated Identification of Patients’ Unmet Social Needs in Clinical Text Using Natural Language Processing","authors":"Sungrim Moon PhD , Yuqi Wu PhD , Jay B. Doughty MHA , Mark L. Wieland MD, MPH , Lindsey M. Philpot PhD, MPH , Jungwei W. Fan PhD , Jane W. Njeru MB, ChB","doi":"10.1016/j.mcpdig.2024.06.008","DOIUrl":"10.1016/j.mcpdig.2024.06.008","url":null,"abstract":"<div><h3>Objective</h3><p>To develop natural language processing (NLP) solutions for identifying patients’ unmet social needs to enable timely intervention.</p></div><div><h3>Patients and Methods</h3><p>Design: A retrospective cohort study with review and annotation of clinical notes to identify unmet social needs, followed by using the annotations to develop and evaluate NLP solutions.</p></div><div><h3>Participants</h3><p>A total of 1103 primary care patients seen at a large academic medical center from June 1, 2019, to May 31, 2021 and referred to a community health worker (CHW) program. Clinical notes and portal messages of 200 age and sex-stratified patients were sampled for annotation of unmet social needs.</p></div><div><h3>Systems</h3><p>Two NLP solutions were developed and compared. The first solution employed similarity-based classification on top of sentences represented as semantic embedding vectors. The second solution involved designing of terms and patterns for identifying each domain of unmet social needs in the clinical text.</p></div><div><h3>Measures</h3><p>Precision, recall, and f1-score of the NLP solutions.</p></div><div><h3>Results</h3><p>A total of 5675 clinical notes and 475 portal messages were annotated, with an inter-annotator agreement of 0.938. The best NLP solution achieved an f1-score of 0.95 and was applied to the entire CHW-referred cohort (n=1103), of whom >80% had at least 1 unmet social need within the 6 months before the first CHW referral. Financial strain and health literacy were the top 2 domains of unmet social needs across most of the sex and age strata.</p></div><div><h3>Conclusion</h3><p>Clinical text contains rich information about patients’ unmet social needs. The NLP can achieve good performance in identifying those needs for CHW referral and facilitate data-driven research on social determinants of health.</p></div>","PeriodicalId":74127,"journal":{"name":"Mayo Clinic Proceedings. Digital health","volume":"2 3","pages":"Pages 411-420"},"PeriodicalIF":0.0,"publicationDate":"2024-07-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2949761224000695/pdfft?md5=3d4e51adfa8825faca3821c4c1259474&pid=1-s2.0-S2949761224000695-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141961077","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Wim van Harten MD, PhD , Carine Doggen PhD , Laura Kooij PhD
{"title":"Organizing Virtual Care, Digital Services Replacing Hospital In-Care and Outpatient Care","authors":"Wim van Harten MD, PhD , Carine Doggen PhD , Laura Kooij PhD","doi":"10.1016/j.mcpdig.2024.06.007","DOIUrl":"10.1016/j.mcpdig.2024.06.007","url":null,"abstract":"<div><p>Hospital-based digital care and virtual care are becoming increasingly common and their reach and scope are expanding in terms of patient groups and technological sophistication. The objective of this viewpoint is to provide guidance on design and factors that can be decisive for the organization of virtual care from a hospital’s perspective. Relevant aspects to be taken into account are as follows: characteristics of the technology, in a broader sense, the nature and intensity of provider involvement and supervision, the degree of self-management by the patient and his environment, the relation and cooperation mechanisms with other providers as home care, general practitioner ’s and other specialist care, the matter of (economies of) scale and finally the uniformity of processes over geographic regions and providers. We provide suggestions for further research and future policy related to these aspects.</p></div>","PeriodicalId":74127,"journal":{"name":"Mayo Clinic Proceedings. Digital health","volume":"2 3","pages":"Pages 405-410"},"PeriodicalIF":0.0,"publicationDate":"2024-07-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2949761224000683/pdfft?md5=5c7a7aa9adf7af3122af5e2927b66066&pid=1-s2.0-S2949761224000683-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141950158","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Kamilla A. Bringel , Davi C.M.G. Leone , João Vitor L. de C. Firmino MME , Marcelo C. Rodrigues PhD , Marcelo D.T. de Melo MD, PhD
{"title":"Voice Analysis and Neural Networks as a Clinical Decision Support System for Patients With Lung Diseases","authors":"Kamilla A. Bringel , Davi C.M.G. Leone , João Vitor L. de C. Firmino MME , Marcelo C. Rodrigues PhD , Marcelo D.T. de Melo MD, PhD","doi":"10.1016/j.mcpdig.2024.06.006","DOIUrl":"10.1016/j.mcpdig.2024.06.006","url":null,"abstract":"<div><h3>Objective</h3><p>To analyze the voice of patients with lung diseases, compared with healthy individuals, to detect patterns capable of assessing dyspnea using artificial neural networks (ANNs).</p></div><div><h3>Patients and Methods</h3><p>This research consists of a cross-sectional prospective pilot study performed in a reference tertiary center, which included a group of patients with lung diseases, compared with a group of healthy individuals. Each patient’s voice was recorded in controlled rooms. The following techniques were applied to extract and select signals’ features: statistical analysis, fast Fourier transform, discrete wavelet transform and Mel-Cepstral analysis. In addition, data augmentation was used to avoid overfitting and improve the ANNs accuracy.</p></div><div><h3>Results</h3><p>A total of 195 voices were recorded: 131 from lung disease patients and 64 from healthy individuals, separated according to gender and age. Using data augmentation, 751 additional audio samples were generated: 501 from healthy individuals and 445 from patients with lung disease. Among male participants, 133 samples were related to lung diseases and 197 were related to healthy ones. From them, 264 audios were used for ANNs training, obtaining an accuracy of 89%. In the female group, 312 had lung diseases and 304 were healthy. Among them, 492 audios were used for training, resulting in an accuracy of 87.6%.</p></div><div><h3>Conclusion</h3><p>Spectral analysis techniques applied to voice recordings using ANNs have reported high accuracy in the efficient diagnosis of lung diseases when compared with healthy individuals.</p></div>","PeriodicalId":74127,"journal":{"name":"Mayo Clinic Proceedings. Digital health","volume":"2 3","pages":"Pages 367-374"},"PeriodicalIF":0.0,"publicationDate":"2024-07-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2949761224000671/pdfft?md5=1ce8698311c7e33369926c62a773de89&pid=1-s2.0-S2949761224000671-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141736552","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Andrea Bernasconi MD, MSc , Marco Landi MSc , Clarence S. Yah PhD , Marianne A.B. van der Sande PhD
{"title":"Information and Communication Technology to Enhance the Implementation of the Integrated Management of Childhood Illness: A Systematic Review and Meta-Analysis","authors":"Andrea Bernasconi MD, MSc , Marco Landi MSc , Clarence S. Yah PhD , Marianne A.B. van der Sande PhD","doi":"10.1016/j.mcpdig.2024.06.005","DOIUrl":"10.1016/j.mcpdig.2024.06.005","url":null,"abstract":"<div><h3>Objective</h3><p>To evaluate the impact of Information and Communication Technology (ICT) on the implementation of Integrated Management of Childhood Illness (IMCI) and integrated Community Case Management (iCCM) through a systematic review and meta-analysis (PROSPERO registration number: CRD42024517375).</p></div><div><h3>Methods</h3><p>We searched MEDLINE, EMBASE, Cochrane Library, and gray literature from January 2010 to February 2024, focusing on IMCI/iCCM-related terms (<em>Integrated Management of Childhood Illness, IMCI, integrated Community Case Management, iCCM</em>) and excluding non-ICT interventions. A meta-analysis synthesized the effect of ICT on clinical assessment, disease classification, therapy, and antibiotic prescription through odds ratio (OR; 95% CI) employing a random effects model for significant heterogeneity (I<sup>2</sup>>50%) and conducting subgroup analyses.</p></div><div><h3>Results</h3><p>Of 1005 initial studies, 44 were included, covering 8 interventions for IMCI, 7 for iCCM, and 2 for training. All digital interventions except 1 outperformed traditional paper-based methods. Pooling effect sizes from 16 studies found 5.7 OR for more complete clinical assessments (95% CI, 1.7-19.1; I<sup>2</sup>, 95%); 2.0 for improved disease classification accuracy (95% CI, 0.9-4.4; I<sup>2</sup>, 93%); 1.4 for more appropriate therapy (95% CI, 0.8-2.2; I<sup>2</sup>, 93%); and 0.2 for reduced antibiotic use (95% CI, 0.06-0.55; I<sup>2</sup> 99%).</p></div><div><h3>Conclusion</h3><p>This review is the first to comprehensively quantify the effect of ICT on the implementation of IMCI/iCCM programs, confirming both the benefits and limitations of these technologies. The customization of digital tools for IMCI/iCCM can serve as a model for other health programs. As ICT increasingly supports the achievement of sustainable development goals, the effective digital interventions identified in this review can pave the way for future innovations.</p></div>","PeriodicalId":74127,"journal":{"name":"Mayo Clinic Proceedings. Digital health","volume":"2 3","pages":"Pages 438-452"},"PeriodicalIF":0.0,"publicationDate":"2024-06-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S294976122400066X/pdfft?md5=a80f8815a0f64675f0eb8a1197a59ad3&pid=1-s2.0-S294976122400066X-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141961079","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Peter Wohlfahrt MD, PhD , Dominik Jenča MD , Vojtěch Melenovský MD, PhD , Jolana Mrázková Mgr , Marek Šramko MD, PhD , Martin Kotrč MD , Michael Želízko MD , Věra Adámková MD, PhD , Francisco Lopez-Jimenez MD, MSc, MBA , Jan Piťha MD, PhD , Josef Kautzner MD, PhD
{"title":"Remote, Smart Device-Based Cardiac Rehabilitation After Myocardial Infarction: A Pilot, Randomized Cross-Over SmartRehab Study","authors":"Peter Wohlfahrt MD, PhD , Dominik Jenča MD , Vojtěch Melenovský MD, PhD , Jolana Mrázková Mgr , Marek Šramko MD, PhD , Martin Kotrč MD , Michael Želízko MD , Věra Adámková MD, PhD , Francisco Lopez-Jimenez MD, MSc, MBA , Jan Piťha MD, PhD , Josef Kautzner MD, PhD","doi":"10.1016/j.mcpdig.2024.06.001","DOIUrl":"https://doi.org/10.1016/j.mcpdig.2024.06.001","url":null,"abstract":"<div><h3>Objective</h3><p>To evaluate the effect of smart device-based telerehabilitation on V<span>o</span><sub>2peak</sub> in patients after myocardial infarction.</p></div><div><h3>Patients and Methods</h3><p>This was a pilot, single-center, randomized, cross-over study with a 3-month intervention. One month after myocardial infarction, patients had cardiopulmonary exercise testing and a 6-minute walking test (6MWT) and were randomly assigned 1:1. In the intervention group, patients received a smartwatch to track the recommended number of steps, which was individualized and derived from the 6MWT. A study nurse telemonitored adherence to the recommended number of steps a day. In the control group, 150 minutes a week of moderate-intensity physical activity was recommended. After 3 months study arms were crossed over, and study procedures were repeated after 3 months.</p></div><div><h3>Results</h3><p>Between June 1, 2019, and February 28, 2023, 64 patients were randomized, of which 61 (aged 51±10 years, 10% women) completed the study. Overall, the smart device-based telerehabilitation led to 2.31 mL/kg/min (95% CI, 1.25-3.37; <em>P</em><.001) V<span>o</span><sub>2peak</sub> increase compared with the control treatment. Furthermore, there was a significant effect on weight (−1.50 kg; 95% CI, −0.39 to −2.70), whereas the effect on the 6MWT distance (4.7 m; 95% CI, −11.8 to 21.1) or Kansas City Quality of Life questionnaire score (0.98; 95% CI, −1.38 to 3.35) was not significant.</p></div><div><h3>Conclusion</h3><p>Smart device-based cardiac rehabilitation may be a promising alternative for patients unable or unwilling to attend in-person cardiac rehabilitation.</p></div><div><h3>Trial Registration</h3><p><span>clinicaltrials.gov</span><svg><path></path></svg> Identifier: <span>NCT03926312</span><svg><path></path></svg></p></div>","PeriodicalId":74127,"journal":{"name":"Mayo Clinic Proceedings. Digital health","volume":"2 3","pages":"Pages 352-360"},"PeriodicalIF":0.0,"publicationDate":"2024-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2949761224000622/pdfft?md5=525707d1f0d92a2cc407d45c17140fef&pid=1-s2.0-S2949761224000622-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141595891","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Irbaz Bin Riaz MD, MS, MBI, PhD , Syed Arsalan Ahmed Naqvi MD , Bashar Hasan MD , Mohammad Hassan Murad MD, MPH
{"title":"Future of Evidence Synthesis: Automated, Living, and Interactive Systematic Reviews and Meta-analyses","authors":"Irbaz Bin Riaz MD, MS, MBI, PhD , Syed Arsalan Ahmed Naqvi MD , Bashar Hasan MD , Mohammad Hassan Murad MD, MPH","doi":"10.1016/j.mcpdig.2024.05.023","DOIUrl":"https://doi.org/10.1016/j.mcpdig.2024.05.023","url":null,"abstract":"","PeriodicalId":74127,"journal":{"name":"Mayo Clinic Proceedings. Digital health","volume":"2 3","pages":"Pages 361-365"},"PeriodicalIF":0.0,"publicationDate":"2024-06-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2949761224000580/pdfft?md5=394feae87eb958f7b2342fa783623323&pid=1-s2.0-S2949761224000580-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141605114","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Adam M. Ostrovsky BS, Joshua R. Chen BS, Vishal N. Shah DO, Babak Abai MD
{"title":"Performance of 5 Prominent Large Language Models in Surgical Knowledge Evaluation: A Comparative Analysis","authors":"Adam M. Ostrovsky BS, Joshua R. Chen BS, Vishal N. Shah DO, Babak Abai MD","doi":"10.1016/j.mcpdig.2024.05.022","DOIUrl":"https://doi.org/10.1016/j.mcpdig.2024.05.022","url":null,"abstract":"","PeriodicalId":74127,"journal":{"name":"Mayo Clinic Proceedings. Digital health","volume":"2 3","pages":"Pages 348-350"},"PeriodicalIF":0.0,"publicationDate":"2024-06-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2949761224000579/pdfft?md5=4ab21ee50f30d05ffdd5242c3c0f5bb9&pid=1-s2.0-S2949761224000579-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141479361","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}